Revisiting Manager READMEs

Elided Branches
www.elidedbranches.com
2025-11-22 19:02:00
Several years ago, I published a critique of manager READMEs that succeeded in stirring up a lot of feelings, pro and con. I’d like to believe it prompted some people to reconsider whether these are actually effective tools.Today, I want to revisit this. Not to encourage you to write a manager READM...
Original Article

Several years ago, I published a critique of manager READMEs that succeeded in stirring up a lot of feelings, pro and con. I’d like to believe it prompted some people to reconsider whether these are actually effective tools.

Today, I want to revisit this. Not to encourage you to write a manager README, but to suggest other ways forward that I have learned in the years since writing the first post.

The Problem

When you become a senior manager or an executive, you face new challenges. Your job involves directing work across many people with different approaches, styles, and opinions. Left to their own devices, each person will develop a slightly different way of communicating with you, one that works for them and that they believe works for you.

With a broad scope of work to oversee, you need to quickly grasp what matters and should be shared upward, outward, and down into different parts of your organization. Now, at most companies, this is a known problem and inevitably someone has already tried to solve it by means of standardized tooling and reporting. Everyone uses Jira for a reason and it’s not that Jira is the best tool ever, but it is malleable to many types of standardization. Companies implement OKR tools and Tableau dashboards, they institute various program management processes, they run quarterly business reviews, and all of these are done in the name of standardizing the information that is passed upward and outward so that people can make better decisions.

Unfortunately, this is typically the lowest common denominator of usefulness to any senior manager. Reporting generated in this way obscures as much as it reveals, and it rarely addresses the things that you really care about¹. So senior managers need other mechanisms for imparting what they want to hear about and see. The README can sometimes be an attempt to impart that cultural overlay: a way of saying, “I care about X, and want you to focus on that when you communicate to me; I don’t care much about Y and Z, and by the way, it’s best if you communicate with me in these ways.”

I remain steadfast that this is not a good approach. It creates a focus on you as the person to be managed up to. Your personality must be accommodated, your preferences honored. I get the desire for this, and I’m certainly not immune to being managed up to, but my preference is to avoid major blind spots. I want to hear what I care about, yes, but I don’t want to live in an information bubble either.

READMEs are also rather lazy. There’s a kernel of truth in their purpose: we want people to focus certain types of communication on what we believe is most valuable. However, doing it in the form of a general README isn’t actually the most effective approach.

So if not READMEs, what then?

The Solution: Appropriate Templates and Ceremonies

Instead of one doc that attempts to communicate all of your preferences and warts and creates a you-focused mindset, it’s time to level up and recognize that a big part of the job of senior/executive management is setting standards for doing certain types of work. The best way to set those standards, in my experience, is lightweight templates and ceremonies for information sharing, discussion, and decision-making.

I think that every good senior manager should have some toolkit of these. You aren’t just going to operate against the lowest common denominator of pre-existing reports and processes in your company, you have to establish a few processes that exist to show what you care about and where you want the organization to focus. One of mine is Wins and Challenges (discussed in my recent book ), which I’ve brought from startups to giant teams and everything in-between. Is it extra work on top of whatever people might be doing in Jira or other tools? Possibly. Does it create far more valuable conversation across my leadership team than those tools? Yes. Does it help me specifically understand things and do my job better? Absolutely.

There is a very lightweight template to follow for my Wins and Challenges, and the process details are owned by the team gathering the information (although I specify opinions about how it should be done, I only check the outcomes). I find that the best templates and processes are lightweight in a way that they show what information should be collected but don’t dictate exactly the process to collect that information.

Developing templates that expose the right useful information is hard. You will both over-do and under-do this as you’re figuring it out, whether it’s your first time in the job, you’ve moved to a different company or team, or your team has just evolved past the usefulness of the old methods. My advice is to start simple and add on new details or processes only when it’s clear you have a widespread gap. A good rhythm for a new job/team is to learn for 90 days, then introduce what you need, and evolve from there with enough time to learn from each iteration (usually, 1-2 quarters).

Don’t Try To Template/Processify Everything

I recently asked an experienced CPO about good product processes, and what they looked like from his perspective. One piece of advice was that not everything should have a fixed process or template. When you need to leave room for discussion, it’s often best to limit the structure; a walkthrough of a prototype might be better done as an open-ended exploration and discussion rather than a formal set of steps.

It’s important not to give into the temptation (or external pressure) to create processes for everything. I personally do not have a fixed format for my 1-1s, and dislike even the expectation of coming with a set of written and shared topics. I don’t want to feel rushed to finish everything on an agenda, and the temptation to immediately jump to conclusions about a topic based on an agenda item often increases miscommunication. Sometimes there’s a need to pre-read and prepare, but sometimes we just need to talk and see where the exploration of current top-of-mind concerns and information takes us.

So, senior leaders, you can tell people how you want them to work with you, but don’t do it via the crude mechanism of a manager README. Drive clarity through templates and processes where needed, resist the urge to create them everywhere, and lead your organization by showing them where to spend their time and focus as a collective good, not just good for you.

¹ Think of it this way, if you could easily see the problems via the pre-existing dashboards, they’d already be on their way to being solved. Dashboards are like alerts and tests in this way, they tend to catch what you know could go wrong, but rarely the surprise problems that lead to big incidents. Necessary, but insufficient.

Enjoy this post? You might like my books: The Manager’s Path , and Platform Engineering: A Guide for Technical, Product, and People Leaders , available on Amazon and Safari Online.

Maybe that's not liquid water on Mars after all

Hacker News
phys.org
2025-11-23 12:58:54
Comments...
Original Article
Maybe that's not liquid water on Mars after all
A highly reflective area over the base of the Martian south pole suggested the potential presence of liquid water. But new radar measurements suggest there may be another explanation. Credit: ESA/DLR/FU Berlin, CC BY-SA 3.0 IGO

Ancient Mars boasted abundant water, but the cold and dry conditions of today make liquid water on the Red Planet seem far less probable. However, the Mars Advanced Radar for Subsurface and Ionosphere Sounding (MARSIS) detected strong radar reflections from a 20-kilometer-wide area over the base of Mars's southern polar ice cap, hinting at the possibility of liquid water below the icy surface. Such a finding would have major implications for the planet's possible habitability.

But sustaining liquid water underneath the ice might not be feasible without very salty brines or localized volcanic heat. Scientists have deliberated about other possible "dry" explanations for the bright reflections detected by MARSIS, such as layers of carbon dioxide and water ices or salty ice and clay causing elevated radar reflectivity.

Aboard the Mars Reconnaissance Orbiter, the Shallow Radar (SHARAD) uses higher frequencies than MARSIS. Until recently, though, SHARAD's signals couldn't reach deep enough into Mars to bounce off the base layer of the ice where the potential water lies—meaning its results couldn't be compared with those from MARSIS.

However, the Mars Reconnaissance Orbiter team recently tested a new maneuver that rolls the spacecraft on its flight axis by 120°—whereas it previously could roll only up to 28° . The new maneuver, termed a "very large roll," or VLR, can increase SHARAD's signal strength and penetration depth, allowing researchers to examine the base of the ice in the enigmatic high-reflectivity zone.

Gareth Morgan and colleagues, for their article published in Geophysical Research Letters , examined 91 SHARAD observations that crossed the high-reflectivity zone. Only when using the VLR maneuver was a SHARAD basal echo detected at the site. In contrast to the MARSIS detection, the SHARAD detection was very weak, meaning it is unlikely that liquid water is present in the high-reflectivity zone.

The researchers suggest that the faint detection returned by SHARAD under this portion of the ice cap is likely due to a localized region of smooth ground beneath the ice. They add that further research is needed to reconcile the differences between the MARSIS and SHARAD findings.

More information: Gareth A. Morgan et al, High Frequency Radar Perspective of Putative Subglacial Liquid Water on Mars, Geophysical Research Letters (2025). DOI: 10.1029/2025gl118537

Provided by Eos

This story is republished courtesy of Eos, hosted by the American Geophysical Union. Read the original story here .

Citation : Maybe that's not liquid water on Mars after all (2025, November 21) retrieved 23 November 2025 from https://phys.org/news/2025-11-liquid-mars.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

X.com Is Gonna Snitch You Out to the Public If You Use a VPN

Hacker News
www.vice.com
2025-11-23 12:46:11
Comments...
Original Article

Regular folks use virtual private networks (VPNs) to keep themselves safe from online data thieves, hackers, snoops, and crooks. Those who live, work, or travel in countries with oppressive human rights rely upon them not to get thrown in jail (or worse). Journalists and activists, likewise, depend on VPNs to keep them safe from the prying eyes of those who’d harm them.

Now it looks like people who use VPN to connect to X will get warning text shown on their profiles. It was hinted at by X last month, but we’re starting to see more evidence that it’s actually set to happen.

Videos by VICE

what it’ll look like

“When you read content on X, you should be able to verify its authenticity. This is critical to getting a pulse on important issues happening in the world,” posted Nikita Bier, Head of Product at X, back on October 14, 2025.

“As part of that, we’re experimenting with displaying new information on profiles, including which country an account is based, among other details. Starting next week, we will surface this on a handful of profiles of X team members to get feedback.”

Online security advocates immediately criticized the news, saying that it was a thinly veiled move to out those who use VPNs, which protect online anonymity by routing internet traffic through a third-party server as a middleman. VPNs are widely available to private citizens in most countries in the world, including the US, where X is based.

“Can users choose to opt out? (It) has potential privacy implications,” asked a user, to which Bier replied, “ There will be privacy toggles. However, if a user configures them, that will likely be highlighted on their profile. More to come.”

Well, we got more details, and it sure looks like X is outing those who use VPNs. “X will show a warning on your account if you try to use a VPN to hide where your account is from when the upcoming ‘About Your Account’ feature launches,” Aaron Perris, an analyst over at MacRumors, posted on X.com on November 15, 2025.

The post accompanied a chunk of (ostensibly X.com’s) code showing the text it’ll display on your X profile to other X users: “‘One of our partners has indicated that you may be connecting via a proxy—such as a VPN—which may change the country or region that is displayed on your profile.’ Your profile will display ‘Country or region may not be accurate’ to other users.”

It’s a shitty move on X’s part. If X really wanted to keep its users safe, it would do something about the severity , depth , and breadth of misinformation that it allows on its platform. But hey, they can monetize that, so I wouldn’t hold my breath.

What does the AGPL require?

Lobsters
runxiyu.org
2025-11-23 12:19:52
Comments...
Original Article
Timed out getting readerview for https://runxiyu.org/blog/agpl/

How Deep is Challenger Deep?

Lobsters
storymaps.arcgis.com
2025-11-23 12:06:25
Comments...

Deepnote (YC S19) is hiring engineers to build a better Jupyter notebook

Hacker News
deepnote.com
2025-11-23 12:03:06
Comments...
Original Article
Timed out getting readerview for https://deepnote.com/join-us

Has Britain become an economic colony?

Guardian
www.theguardian.com
2025-11-23 12:00:10
The UK could’ve been a true tech leader – but it has cheerfully submitted to US dominance in a way that may cost it dear Two and a half centuries ago, the American colonies launched a violent protest against British rule, triggered by parliament’s imposition of a monopoly on the sale of tea and the ...
Original Article

T wo and a half centuries ago, the American colonies launched a violent protest against British rule, triggered by parliament’s imposition of a monopoly on the sale of tea and the antics of a vainglorious king. Today, the tables have turned: it is Great Britain that finds itself at the mercy of major US tech firms – so huge and dominant that they constitute monopolies in their fields – as well as the whims of an erratic president. Yet, to the outside observer, Britain seems curiously at ease with this arrangement – at times even eager to subsidise its own economic dependence. Britain is hardly alone in submitting to the power of American firms, but it offers a clear case study in why nations need to develop a coordinated response to the rise of these hegemonic companies.

The current age of American tech monopoly began in the 2000s, when the UK, like many other countries, became almost entirely dependent on a small number of US platforms – Google, Facebook, Amazon and a handful of others. It was a time of optimism about the internet as a democratising force, characterised by the belief that these platforms would make everyone rich. The dream of the 1990s – naive but appealing – was that anyone with a hobby or talent could go online and make a living from it.

US tech dominance wasn’t the result of a single policy decision. Yet it was still a choice that countries made – as is highlighted by China’s decision to block foreign sites and build its own. While that move was far easier under an authoritarian system, it also amounted to an industrial policy – one that left China as the only other major economy with its own full digital ecosystem.

The pattern was sustained through the 2000s and 2010s. Cloud computing was quickly cornered by Amazon and Microsoft. No serious European or UK competitor emerged to challenge platforms such as Uber or Airbnb. These companies have undoubtedly brought us convenience and entertainment, but the wealth of the internet has not spread as widely as many hoped; instead, US firms took the lion’s share, becoming the most valuable corporations in history. Now the same thing is happening with artificial intelligence. Once more, the big profits look destined for Silicon Valley.

How did all this meet with such little resistance? In short, the UK and Europe followed the logic of free trade and globalisation. Nations, according to this theory, should focus only on what they do best. So just as it made sense for the UK to import French burgundies and Spanish hams, it also seemed logical to rely on American technology rather than trying to do it locally. Better to specialise instead in the UK’s own strengths, such as finance, the creative industries – or making great whisky.

But when it comes to these new platforms, the analogy with regular trade breaks down. There is a vast difference between fine wines and the technologies that underpin the entire online economy. Burgundies can be pricey, but they don’t extract value from every commercial transaction or collect lucrative data. The trade theories of the 1990s masked the distinction between ordinary goods and what are, in effect, pieces of market infrastructure – systems essential to buying and selling. That’s what Google and Amazon represent. A better analogy might be letting a foreign firm build toll roads across the country, charging whatever it likes to use them.

We’re seeing this again with the build-out of artificial intelligence. During President Trump’s state visit in September, the UK proudly celebrated Google and Microsoft ’s investments in “datacentres” – vast warehouses of computer servers that power AI systems. Yet datacentres are the bottom rung of the AI economy, private infrastructure that simply channels profits back to US headquarters.

In another timeline, the UK could have been a true leader in AI. US researchers were once far behind their British and French counterparts. Yet, in a move neither Washington nor Beijing would have permitted, the UK cheerfully allowed the sale of most of its key AI assets and talent over the last decade or so – DeepMind’s purchase by Google being the most famous example. What remains is an AI strategy consisting of the supply of electricity and land for datacentres. It’s like being invited to a party only to discover you’re there to serve the drinks.

If tech platforms are indeed like toll roads, the logical step would be to limit their take – perhaps by capping fees or charging for data extraction. Yet no country has done so: we accept the platforms but fail to regulate their power as we do with other utilities. The European Union has come closest, with its Digital Markets Act , which regulates how dominant platforms treat dependent businesses. The US government, for its part, is also at the mercy of its homegrown tech giants, yet Congress remains paralysed.

If the UK wanted to take a different path, to resist this economic colonisation and extraction, it could partner with the European Union and perhaps Japan in order to develop a joint strategy – one that forces platforms to support local businesses and nurtures alternatives to mature US technologies. So far, though, alongside other nations disadvantaged by American dominance, it has been slow to adapt, instead hoping that the 90s playbook will still work, despite evidence to the contrary.

The truth is that we now live in a more cynical and strategic era. One way or another, the world needs an anti-monopoly framework with far greater force than anything seen so far. Wherever you live, it’s clear the world would be better off with more firms from different countries. The alternative is not only costly but politically dangerous, feeding resentment and dependence. We can do better than a future where what counts as economic freedom is merely a choice between relying on the United States, or relying on China.

Tim Wu is a former special assistant to President Biden and author of The Age of Extraction: How Tech Platforms Conquered the Economy and Threaten Our Future Prosperity (Bodley Head).

Further reading

The Tech Coup by Marietje Schaake (Princeton, £13.99)

Supremacy by Parmy Olson (Pan Macmillan, £10.99)

Chip War by Chris Miller (Simon & Schuster, £10.99)

What are you reading and plan to read next?

Lobsters
lobste.rs
2025-11-23 11:25:03
Also what were you reading? What are you planning to read next? Feel free to ask questions about books you see in the post or comment! I've finished reading the Memory Code by Lynne Kelly — and can highly recommend for unusual point of view on how illiterate society stores information. Still working...
Original Article

Also what were you reading? What are you planning to read next? Feel free to ask questions about books you see in the post or comment!

I've finished reading the Memory Code by Lynne Kelly — and can highly recommend for unusual point of view on how illiterate society stores information. Still working through The Powerful Python by Aaron Maxwell, and it's not bad — it's just not that much of a time right now.

My plans for non-technical books: Ukrainian translation of Memory Called Empire by Arkady Martine (original was perhaps best scifi of my 2024, let's see how much they got into the translation). Also started Patrick Obrian's Master and Commander recommended by fellow lobsterer in similar thread years ago — great reading both in eink and audio!

previous post month ago

The Feds Want to Make It Illegal to Even Possess an Anarchist Zine

Intercept
theintercept.com
2025-11-23 11:00:00
Daniel Sanchez is facing federal charges for what free speech advocates say is a clear attack on the First Amendment. The post The Feds Want to Make It Illegal to Even Possess an Anarchist Zine appeared first on The Intercept....
Original Article
A detail view of the badge worn by Matthew Elliston during an ICE hiring event on Aug. 26, 2025, in Arlington, Texas. Photo: Ron Jenkins/Getty Images

Federal prosecutors have filed a new indictment in response to a July 4 noise demonstration outside the Prairieland ICE detention facility in Alvarado, Texas, during which a police officer was shot .

There are numerous problems with the indictment, but perhaps the most glaring is its inclusion of charges against a Dallas artist who wasn’t even at the protest. Daniel “Des” Sanchez is accused of transporting a box that contained “Antifa materials” after the incident, supposedly to conceal evidence against his wife, Maricela Rueda, who was there.

But the boxed materials aren’t Molotov cocktails, pipe bombs, or whatever MAGA officials claim “Antifa” uses to wage its imaginary war on America. As prosecutors laid out in the July criminal complaint that led to the indictment, they were zines and pamphlets. Some contain controversial ideas — one was titled “Insurrectionary Anarchy” — but they’re fully constitutionally protected free speech. The case demonstrates the administration’s intensifying efforts to criminalize left-wing activists after Donald Trump announced in September that he was designating “Antifa” as a “major terrorist organization” — a legal designation that doesn’t exist for domestic groups — following the killing of Charlie Kirk.

Sanchez was first indicted in October on charges of “corruptly concealing a document or record” as a standalone case, but the new indictment merges his charges with those against the other defendants, likely in hopes of burying the First Amendment problems with the case against him under prosecutors’ claims about the alleged shooting.

It’s an escalation of a familiar tactic. In 2023, Georgia prosecutors listed “zine” distribution as part of the conspiracy charges against 61 Stop Cop City protesters in a sprawling RICO indictment that didn’t bother to explain how each individual defendant was involved in any actual crime. I wrote back then about my concern that this wasn’t just sloppy overreach, but also a blueprint for censorship . Those fears have now been validated by Sanchez’s prosecution solely for possessing similar literature.

Photos of the zines Daniel Sanchez is charged with “corruptly concealing.” Photo: U.S. District Court, Northern District of Texas

There have been other warnings that cops and prosecutors think they’ve found a constitutional loophole — if you can’t punish reporting it, punish transporting it. Los Angeles journalist Maya Lau is suing the LA County Sheriff’s Department for secretly investigating her for conspiracy, theft of government property, unlawful access of a computer, burglary, and receiving stolen property. According to her attorneys , her only offense was reporting on a list of deputies with histories of misconduct for the Los Angeles Times.

If you can’t punish reporting it, punish transporting it.

It’s also reminiscent of the Biden administration’s case against right-wing outlet Project Veritas for possessing and transporting Ashley Biden’s diary, which the organization bought from a Florida woman later convicted of stealing and selling it. The Constitution protects the right to publish materials stolen by others — a right that would be meaningless if they couldn’t possess the materials in the first place.

Despite the collapses of the Cop City prosecution and the Lau investigation — and its own dismissal of the Project Veritas case — the Trump administration has followed those dangerous examples, characterizing lawful activism and ideologies as terrorist conspiracies (a strategy Trump allies also floated during this first term) to seize the power to prosecute pamphlet possession anytime they use the magic word “Antifa.”

That’s a chilling combination for any journalist, activist, or individual who criticizes Trump. National security reporters have long dealt with the specter of prosecution under the archaic Espionage Act for merely obtaining government secrets from sources, particularly after the Biden administration extracted a guilty plea from WikiLeaks founder Julian Assange . But the rest of the press — and everyone else, for that matter — understood that merely possessing written materials, no matter what they said, is not a crime.

Guilt by Literature

At what point does a literary collection or newspaper subscription become prosecutorial evidence under the Trump administration’s logic? Essentially, whenever it’s convenient. The vagueness is a feature, not a bug. When people don’t know which political materials might later be deemed evidence of criminality, the safest course is to avoid engaging with controversial ideas altogether.

The slippery slope from anarchist zines to conventional journalism isn’t hypothetical, and we’re already sliding fast. Journalist Mario Guevara can tell you that from El Salvador, where he was deported in a clear case of retaliation for livestreaming a No Kings protest. So can Tufts doctoral student Rümeysa Öztürk , as she awaits deportation proceedings for co-writing an opinion piece critical of Israel’s wars that the administration considers evidence of support for terrorism.

At least two journalists lawfully in the U.S. — Ya’akub Ira Vijandre and Sami Hamdi — were nabbed by ICE just last month. The case against Vijandre is partially based on his criticism of prosecutorial overreach in the Holy Land Five case and his liking social media posts that quote Quranic verses, raising the question of how far away we are from someone being indicted for transporting a Quran or a news article critical of the war on terror.

Sanchez’s case is prosecutorial overreach stacked on more prosecutorial overreach. The National Lawyers Guild criticized prosecutors’ tenuous dot-connecting to justify holding 18 defendants responsible for one gunshot wound. Some defendants were also charged with supporting terrorism due to their alleged association with “Antifa.” Anarchist zines were cited as evidence against them, too.

Sanchez was charged following a search that ICE proclaimed on social media turned up “literal insurrectionist propaganda” he had allegedly transported from his home to an apartment, noting that “insurrectionary anarchism is regarded as the most serious form of domestic (non-jihadi) terrorist threat.” The tweet also said that Sanchez is a green card holder granted legal status through the Deferred Action for Childhood Arrivals program.

The indictment claims Sanchez was transporting those materials to conceal them because they incriminated his wife. But how can possession of literature incriminate anyone, let alone someone who isn’t even accused of anything but being present when someone else allegedly fired a gun? Zines aren’t contraband; it’s not illegal to be an anarchist or read about anarchism. I don’t know why Sanchez allegedly moved the box of documents, but if it was because he (apparently correctly) feared prosecutors would try to use them against his wife, that’s a commentary on prosecutors’ lawlessness, not Sanchez’s.

Violent rhetoric is subject to punishment only when it constitutes a “ true threat ” of imminent violence. Even then, the speaker is held responsible, not anyone merely in possession of their words.

Government prosecutors haven’t alleged the “Antifa materials” contained any “true threats,” or any other category of speech that falls outside the protection of the First Amendment. Nor did they allege that the materials were used to plan the alleged actions of protesters on July 4 (although they did allege that the materials were “anti-government” and “anti-Trump”).

We don’t need a constitutional right to publish (or possess) only what the government likes.

Even the aforementioned “ Insurrectionary Anarchy: Organizing for Attack ” zine, despite its hyperbolic title, reads like a think piece, not a how-to manual. It advocates for tactics like rent strikes and squatting, not shooting police officers. Critically, it has nothing to do with whether Sanchez’s wife committed crimes on July 4.

Being guilty of possessing literature is a concept fundamentally incompatible with a free society. We don’t need a constitutional right to publish (or possess) only what the government likes, and the “anti-government” literature in Sanchez’s box of zines is exactly what the First Amendment protects. With history and leaders like Vladimir Putin and Viktor Orbán as a guide, we also know it’s highly unlikely that Trump’s censorship crusade will stop with a few radical pamphlets.

The Framers Loved Zines

There’s an irony in a supposedly conservative administration treating anti-government pamphlets as evidence of criminality. Many of the publications the Constitution’s framers had in mind when they authored the First Amendment’s press freedom clause bore far more resemblance to Sanchez’s box of zines than to the output of today’s mainstream news media.

Revolutionary-era America was awash in highly opinionated, politically radical literature. Thomas Paine’s “Common Sense” was designed to inspire revolution against the established government. Newspapers like the Boston Gazette printed inflammatory writings by Samuel Adams and others urging the colonies to prepare for war after the Coercive Acts. The Declaration of Independence itself recognized the right of the people to rise up. It did not assume the revolution of the time would be the last one.

One might call it “literal insurrectionist propaganda” — and some of it was probably transported in boxes.

The framers enshrined press freedom not because they imagined today’s professionally trained journalists maintaining careful neutrality. They protected it because they understood firsthand the need for journalists and writers who believed their government had become tyrannical to espouse revolution.

For all their many faults, the framers were confident enough in their ideas that they were willing to let them be tested. If the government’s conduct didn’t call for radical opposition, then radical ideas wouldn’t catch on. It sure looks like the current administration doesn’t want to make that bet.

Signal knows who you're talking to

Hacker News
sanesecurityguy.com
2025-11-23 10:22:24
Comments...
Original Article

Or, at the very least, they can.

Recently I got a friend to finally join me on Signal. He asked something about whether or not Signal is truly secure and private, like if it was safe from US government surveillance. I told him: “Well, it’s end-to-end encrypted, so they don’t know what we’re talking about, but they definitely know that we’re talking to each other.”

I said that because Signal uses our phone numbers as ID’s. So, Signal would know that Phone Number A is talking to Phone Number B, and if they can figure out that Phone Number A belongs to me, and Phone Number B belongs to my buddy (usually not too hard to figure out with some OSINT or the assistance of certain governments), then Signal would know that my buddy and I are talking, even if they don’t know what we’re talking about.

This is a limit of end-to-end encryption, which I’ve talked about before . End-to-end encryption provides confidentiality of data, but not anonymity or protection from identifying metadata.

However, I was surprised when my friend got back to me saying that, no, Signal actually doesn’t know who’s talking to who because of this feature called “ Sealed Sender “.

“Woah! Seriously?! Cool!” I thought. But then I started reading how Sealed Sender actually works, according to none other than Signal themselves, and I found that this feature is very technically complex, and totally useless .

ʕ ಠ ᴥಠ ʔ: Woah! Seriously?! Not cool!

One-way anonymity for two-way communications

While Sealed Sender is pretty complicated under the hood, the result of it is one-way anonymity. That means that, when Phone Number A sends a message to Phone Number B, Signal won’t know that the message is coming from Phone Number A and will only know that the message is to be delivered to Phone Number B.

It does this in a way that’s very similar to snail mail without a return address: the letter inside the mail envelope might tell the recipient who the sender is, but the mail envelope itself tells the post office only who the recipient is so that it can be delivered to them. If the post office doesn’t or can’t open the envelope to read the letter itself, then they don’t know who the sender is. Later on, when the recipient wants to send a reply to the sender, they can do the same thing.

ʕ·ᴥ·ʔ: Hm, okay. This kind of sounds like it’s anonymous.

Well, yes, it sort of is, but only when there’s only one message to be sent. The problem comes up when multiple messages are being sent back-and-forth like this.

Sticking with the snail mail analogy, what happens when two pen pals keep sending mail to each other from their homes without including return addresses in their envelopes? The postal service might not know who exactly is sending each piece of mail but, over time, they would know that Address A in Lower Manhattan , New York, keeps on getting one-way mail from the post office in 3630 East Tremont Avenue, the Bronx , New York; and Address B in the Bronx keeps on getting one-way mail from the post office in 350 Canal Street, Lower Manhattan .

ʕ´•ᴥ•`ʔ: Oh. Then the postal service would be pretty sure that whoever is living at Address A and Address B are talking to each other.

Exactly. That’s the limitation of one-way anonymity: it works only one way! Once you start doing two-way communications, with replies going back-and-forth, then one-way anonymity is useless.

Two pieces of metadata

With multiple messages being sent back-and-forth over time, and with Signal knowing only the recipient phone number of each message, it would be pretty hard for Signal to figure out who’s talking to who when their servers are getting thousands of messages every second from different senders, with each message being conveyed to thousands of different recipients. But, Signal doesn’t know only the recipient phone number of each message; they also know the IP address of each sender. And this is where the snail mail analogy fails, because IP addresses are much more specific than post offices.

Signal messages, as we all know, get sent over the internet, and the internet sends data around using IP addresses. Sealed Sender only protects the sender’s phone number; it does not protect the sender’s IP address. So, if you’re sending Signal messages to your super secret pen pal from your house, and you aren’t using a VPN or Tor, then Signal knows that the messages being sent to your pen pal’s phone number are coming from your house’s IP address (not a post office, your house).

Even if you are using some method of masking your real IP address, you still have to use some IP address in order to communicate on the internet, and Signal will see that the same IP address keeps on sending messages to the same phone number . That’s enough to easily figure out that all of these different messages meant for the recipient are coming from the same sender. Sure, maybe you’re using the IP address of a VPN server or Tor exit node that has other Signal users sending messages at the same time, but that’s extremely unlikely. More likely: Even when you use a VPN or Tor, Signal can easily tell that every Sealed Sender message you’re sending to your pen pal are coming from one person: you.

And if your pen pal replies, the reply will have his IP address on it (the same IP address Signal sent your messages to) and your phone number on it. And then, when you want to receive the reply, you have to connect to Signal’s servers using, again, your IP address (the same IP address you used to send your messages to your pen pal earlier). Just like that, with two messages, Signal figured out which phone number (yours) is talking to which other phone number (your pen pal’s). If they ever decide to try and figure out who own these two phone numbers, they could ask your telecoms, or simply search Facebook and Twitter.

You can’t avoid using IP addresses on the internet; they are a necessity on the internet. But you can use a VPN or Tor to mask your real IP address with a fake one that’s not tied to your identity. But you can’t do that with phone numbers. A phone number is either tied to your identity or it isn’t; there is no masking possible, unless you use a service like MySudo which isn’t available for most of us (US and Canada only, as of this writing). If you’re fortunate enough to be able to buy a prepaid SIM without ID, then great, all you and your pen pal have to do is buy some SIM cards that aren’t tied to your identities. If buying a prepaid SIM without ID ain’t an option, then your phone number has to be tied to your identity, and Signal can use these unmasked phone numbers, in combination with masked or unmasked IP addresses, to figure out who’s talking to who, despite Sealed Sender’s promises , as long as there’s a two-way conversation going on.

Which brings up an interesting question: Why does Signal require phone numbers?

ʕ´•ᴥ•`ʔ: Hey, that is an interesting question…

Signal works over the internet, and the internet requires IP (internet protocol) addresses in order to figure out where a message should go. But sending messages over the internet does not require phone numbers; that’s a requirement when using SMS or cellular calls or mobile data, but not for using the internet . And yet, the “privacy-protecting” Signal app requires you to use a phone number to send and receive messages…

ʕ⚆ᴥ⚆ʔ: Hmmmm…

It’s always a two-way street

It gets worse. I keep repeating this: two-way communication . Sealed Sender doesn’t work with two-way communication . But, I’ve kind of been lying. The truth is: Signal already knows which phone numbers have been talking to which, even with Sealed Sender and only one-way communication.

ʕ ಠ ᴥಠ ʔ: What?!

Do these check marks look familiar to you? (Forgive the pixelation.)

ʕ·ᴥ·ʔ: Hm, yeah. Aren’t they the check marks that show up for at least a second whenever I send a Signal message? This is what’s shown after the lone check mark, and before they both turn white to indicate that my message was read, right?

That’s right. The lone check mark indicates that your Signal message was sent to Signal’s servers, these two check marks above indicate that your Signal message has been delivered to the recipient, and the two white check marks indicate that the recipient has read your Signal message.

Now, the thing about the two check marks above is that your Signal app only shows them when your phone has received what’s called a “delivery receipt” from the recipient’s phone. Whenever your pen pal gets a message from you, their Signal app sends a delivery receipt from their phone, through Signal’s servers, to your phone. Their Signal app does this automatically and instantly , and neither of you can turn it off. You can turn off read receipts (the two white check marks) and typing indicators, but you can’t turn off the very first reply : delivery receipts.

The delivery receipt is – ahem – also “protected” using Sealed Sender, but what was it that I’ve been saying this whole time is wrong with Sealed Sender?

ʕ·ᴥ·ʔ: It works only one-way…

ʕ   • ᴥ •   ʔ: It works only one-way…

ʕ   º ᴥ º   ʔ: …and the delivery receipt automatically makes it two-way.

Exactly. And you can’t turn it off. Go figure why.

Some alternatives and a work in progress

So if you can’t trust Signal, who can you trust? Well, if all you need is a private text-based communication channel that won’t falsely advertise their privacy guarantees to you, Proton Mail and Tutanota (now called Tuta) are pretty good. But if you want private voice-based communication, then that’s gonna’ be a problem. WhatsApp is even worse than Signal, Telegram is even worse than WhatsApp , Wire requires an email address to use it (another unnecessary requirement), and most of the rest can’t be trusted because they aren’t open-source.

You could use Jitsi for voice communications, but you’d have to use a separate service for text communications. You could use Matrix for both text and voice, but that’s a software and communication protocol, so you’d have to set up your own server running it. You could use Element , which runs Matrix servers, but you’d have to trust Amazon and Cloudflare with your metadata, making this a rather messy solution to a privacy problem.

What that leaves us with is a single service that is still a work in progress: SimpleX . It asks for no global identifiers like phone numbers or email addresses. It at least tries, unlike Signal, to make sure that it doesn’t know who’s talking to who . It does this with the use of proxies that you randomly send your messages through to get to your recipient (the technical details of which are too complicated to get into here). Of course it is open-source and end-to-end encrypted, otherwise I wouldn’t be mentioning it. It even goes so far as to allow you to use Tor with it, or any SOCKS proxy. It’s pretty cool, actually; the most technically amazing communications platform I’ve ever seen.

But, it ain’t perfect. It’s kinda’ slow, and messages sometimes don’t come in in the right order or don’t come in at all. Voice calls are… iffy, particularly when using Tor. It is still a young, developing project, though it has been making great strides in improving itself, including getting a security audit .

Time will tell how it turns out, but at least I can say one thing: we’ve got a viable alternative.

Hey, Kuma!

ʕ •̀ᴥ•́ ʔ: Where have you been for the past 11 months?!

I actually started writing this article months ago and then got busy again.

ʕ ಠ ᴥಠ ʔ: Well at least visit me with some tips and tricks every once in a while.

I’ll try, buddy, but real life comes first before imaginary friends.

ʕ •̀ᴥ•́ ʔ: I know I’m imaginary, but are your subscribers?

I dunno’. Maybe they should give me a hint by signing up below!

Or don’t; my RSS feed’s in the site menu. Unlike Signal, I don’t need you to sign up with a global identifier.

Parallel Threads in Racket v9.0

Lobsters
blog.racket-lang.org
2025-11-23 09:38:03
Comments...
Original Article

posted by Matthew Flatt, Ryan Culpepper, Robby Findler, Gustavo Massaccesi, and Sam Tobin-Hochstadt

With the version 9.0 release, Racket includes support for shared-memory threads that can take advantage of multicore hardware and operating-systems threads to run in parallel— not merely concurrently with other Racket threads, as was the case in versions before 9.0.

Creating a thread that runs in parallel is as simple as adding a flag to the call to thread . To see the effect, try first putting the following code into a file named "thread.rkt" and running racket thread.rkt on the command line:

Racket will find many square roots (tweak N to match your machine), but will keep only one core of your CPU busy. Using time in the shell reports “CPU” (possibly broken into “user” and ‘system”) and “real” times that are similar. To use two cores, add #:pool ' own to the thread call:

In this case, real time should be about half of CPU time, while CPU should remain similar to before. In other words, the parallel version runs twice as fast. On the machine used below :

concurrent

parallel

×1

1011 msec real

979

msec CPU

10

msec CPU for GC

×2

517 msec real

1021

msec CPU

13

msec CPU for GC

Passing the new #:pool argument creates a parallel thread ; create pools via make-parallel-thread-pool to have a group of threads share processor resources or just pass ' own to have the new thread exist in its own parallel thread pool .

As a further addition to thread , a #:keep ' result argument keeps the result of thunk when it returns, instead of discarding the result. Retrieve a thread’s result with thread-wait . So, for example,

runs thunk in parallel to other Racket threads, blocks the current Racket thread (while allowing other Racket threads to continue, even non-parallel ones), and then returns the result value(s) when thunk completes.

To maintain backwards compatibility, the thread function still creates a coroutine thread by default, which is a lightweight thread that is preemptively scheduled and whose execution is interleaved with other coroutine threads. For many tasks that need the organizational benefits of concurrency without the performance benefits of parallelism, such as when managing GUI interactions or orchestrating remote processes, coroutine threads are still the best abstraction. Coroutine threads can use #:keep ' result , too.

Racket’s full thread API works with parallel threads. Follow the links from the thread documentation to see more details on thread pools and for more interesting uses. Of course, just because you put tasks in parallel threads doesn’t mean that they always speed up, as sharing and communication can limit parallelism. Racket’s future visualizer works for parallel threads, tho, and it can help you understand where synchronization in a task limits parallelism. Also, adding parallelism to Racket potentially creates trouble for existing libraries that were not designed to accommodate parallelism. We expect problems to be rare, however.

We’ll explore the performance details and explain why we expect most programs will continue to work well later in this post, but first:

Racket’s Road to Parallelism

Running threads in parallel counts as news in 2025?! Well, it has been a long road.

Racket’s implementation started in the mid-1990s, just as a wave of enthusiasm for parallel programming was winding down. Although operating systems by that point consistently supported within-process threads, computers with multiprocessors were not commonly available. Many language runtime systems from the same era— including Python, Ruby, and OCaml— took advantage of the internal simplicity of a single-threaded runtime system while offering constructs for concurrency at the language level. Racket has always included threads for concurrency, and it was an early adopter of Concurrent ML ’s abstractions for managing concurrency well. But an absence of parallelism was deeply baked into the original implementation.

Over time, to provide support for parallelism, we added places and futures to Racket. Places support coarse-grained parallelism through a message-passing API, effectively running parallel instances of the virtual machine within a single operating-system process; limited sharing makes the implementation easier and safer than arbitrary sharing between parallel threads. Futures provide fine-grained parallelism for restricted computations; a future blocks when it tries to perform any operation that would be difficult for the runtime system to complete safely in parallel. Places and futures are both useful, and they avoid some pitfalls of shared-memory threads. Still, fitting a parallel task into futures or places usually requires special effort.

Meanwhile, single-threaded execution was only one of the problems with the original Racket (a.k.a. PLT Scheme) implementation. To address larger problems with the implementation and to improve performance, we started in 2017 rebuilding Racket on top of Chez Scheme . Rebuilding took some time, and we only gradually deprecated the old “BC” implementation in favor of the new “CS” implementation, but the transition is now complete. Racket BC is still maintained, but as of August 2025, we distribute only Racket CS builds at https://download.racket-lang.org .

Chez Scheme is a much better foundation for improving parallelism in Racket. Part of the Racket-rebuilding effort included improving Chez Scheme’s support for parallelism: we added memory fences as needed for platforms with a weak memory-consistency model, and we parallelized the Chez Scheme garbage collector so that garbage collection itself runs in parallel. There’s still plenty of room for improvement— the garbage collector is only parallel with itself, not the main program, for example— but further improvements are more within reach than before. Equally important, the rebuild included new implementations of the Racket thread scheduler and I/O layer in Racket itself (instead of C). Because of these improvements, Racket’s futures worked better for parallelism from the start in Racket CS than in Racket BC.

With version 9.0, we finally take advantage of new opportunities for parallelism created by the move to Racket CS. Internally, a parallel thread is backed by combination of a future and a coroutine thread. The main extra work was making Racket’s coroutine thread scheduler cooperate more with the future scheduler and making the I/O layer safe for Chez Scheme threads— all while making locks fine-grained enough to enable parallelism, and keeping the cost of needed synchronization as low as possible, including for non-parallel Racket programs.

Performance

Here are some simple benchmarks on an M2 Mac to give a sense of the state of the current implementation. This machine has 8 cores, but 4 big and 4 little, so ×4 speedup is possible with 4-way parallelism but less than ×8 with 8-way parallelism.

As an easy first example, we should expect that a Fibonacci [ code ] run of 1 iteration in each of 4 coroutine threads takes the same time as running it 4 iterations in 1 thread, while 1 iteration in each of 4 parallel threads should take about 1/4th of the time. Also, for such a simple function, using plain old futures should work just as well as parallel threads. That’s what we see in the numbers below.

Times are shown as a speedup over single-threaded, then in real elapsed milliseconds, with CPU milliseconds as the upper smaller number to the right, and CPU milliseconds that are specifically for GC as the lower smaller number to the right. The times are from a single run of the benchmark.

( fib 40 )

real msec

N

sequential

coroutine

parallel

futures

1

×1

511

×1

506

×1

494

×1

495

4

×1

2045

×1

2034

×3.7

554

×3.8

545

8

×1

4154

×1

4154

×5.4

776

×5.2

796

Of course, most programs are not just simple arithmetic. If we change our example to repeatedly convert numbers back and forth to strings as we compute Fibonacci [ code ], then we can see the effects of the more complex conversions. This version also triggers frequent allocation, which lets us see how thread-local allocation and parallel garbage collection scale.

( strfib* 32 )

real msec

N

sequential

coroutine

parallel

futures

1

×1

204

×1

205

×1.1

192

×1

211

4

×1

826

×1

808

×3.7

222

×3.7

221

8

×1

1619

×1

1602

×3.9

419

×4

406

From this table, we still see reasonable scaling up to four cores, but the additional work and the use of the garbage collector limit scaling beyond that point.

That first string variant of Fibonacci includes a slight cheat, however: it goes out of its way to use a string->number* wrapper that carefully calls string->number in a way that avoids evaluating expressions that compute the default values of some arguments. The defaults consult the parameters read-decimal-as-inexact and read-single-flonum which a perfectly fine thing to do in general, but turns out to block a future, because parameter values can depend on the current continuation. In contrast, parallel threads continue to provide a benefit when those kinds of Racket constructs are used. We can see the difference by using plain string->number in place of string->number* , which will fetch parameter values 14 million times in each individual run of ( strfib 32 ) :

( strfib 32 )

real msec

N

sequential

coroutine

parallel

futures

1

×1

772

×1.3

578

×1.1

721

×0.9

873

4

×1

3169

×1.3

2364

×4

797

×0.8

4164

8

×1

6409

×1.4

4730

×4.3

1493

×0.8

8353

The coroutine column here also shows an improvement, surprisingly. That’s because a coroutine thread has a smaller continuation than the one in the sequential column, and the cost of fetching a parameter value can depend (to a limited degree) on continuation size. The effect of parallel threads on this kind of program is more consistent than fine details of a continuation’s shape.

Operations on mutable equal? -based hash tables [ code ] are another case where futures block, but parallel threads can provide performance improvement.

( hash-nums 6 )

real msec

N

sequential

coroutine

parallel

futures

1

×1

193

×1

190

×1

186

×1

191

4

×1

767

×1

763

×3.7

208

×1

763

8

×1

1541

×1

1532

×4.5

346

×1

1539

As an illustration of the current limitations of parallel threads in Racket, let’s try a program that writes data to a byte-string port then hashes it [ code ].

( hash-digs 7 )

real msec

N

sequential

coroutine

parallel

futures

1

×1

127

×1

127

×0.9

135

×1

126

4

×1

503

×0.9

536

×2.5

201

×1

520

8

×1

1022

×0.9

1097

×2.5

403

×1

1049

Here we see that parallel threads do get some speedup, but they do not scale especially well. The fact that separate ports are not contended enables performance improvement from parallelism, but speedup is limited by some general locks in the I/O layer.

Even further in that direction, let’s try a program that hashes all files in the current directory [ code ] and computes a combined hash. When run on the "src" directory of the Racket Git repository, most of the time is reading bytes from files, and locks related to file I/O are currently too coarse-grained to permit much speed-up.

( hash-dir )

real msec

N

sequential

coroutine

parallel

futures

1

×1

170

×1

169

×0.7

256

×1

170

4

×1

692

×1

662

×1.3

515

×1

681

8

×1

1393

×1.1

1293

×1.6

868

×1

1368

Having locks in place for parallel threads can impose a cost on sequential programs, since locks generally have to be taken whether or not any parallel threads are active. Different data structures in Racket use specialized locks to minimize the cost, and most benchmarks reported here run report the same numbers in sequential column in Racket v8.18 (the previous release) and Racket v9.0. The exceptions are the ( hash-nums 6 ) and ( hash-digs 7 ) benchmarks, because those measure very-fine grained actions on mutable hash tables and I/O ports, and the cost is largest for those. Comparing sequential times for those two versions shows that support for parallel thread can cost up to 6-8% for programs that do not use them, although the cost tends to be much less for most programs.

( hash-nums 6 )

real msec

N

v8.18 sequential

v9.0 sequential

1

×1

188

×0.96

195

4

×1

757

×0.98

773

8

×1

1520

×0.98

1546

( hash-digs 7 )

real msec

N

v8.18 sequential

v9.0 sequential

1

×1

118

×0.94

126

4

×1

474

×0.94

506

8

×1

947

×0.92

1025

Overall, parallelizable numerical programs or ones that manipulate unshared data structures can achieve speedup through parallel threads relatively easily, but I/O remains a direction for improvement.

Backward Compatibility

If a library uses mutable variables or objects, either publicly or internally, then it must use locks or some other form of concurrency control to work properly in a multithreaded context. Racket already has concurrency, and the expectation for libraries to work with threads does not change with the introduction of parallel threads. Racket’s semaphores, channels, and other synchronization constructs work the same with parallel threads as concurrent threads. Even programs that use lock-free approaches based on compare-and-swap operation (such as box-cas! ) continue to work, since Racket’s compare-and-swap operations use processor-level primitives.

Still, there are a few concerns:

  • Racket’s coroutine threads offer the guarantee of sequential consistency , which means that effects in one thread cannot be seen out-of-order in another thread. Parallel threads in Racket expose the underlying machine’s memory-consistency model , which may allow reordering of memory effects as observed by other threads. In general, a weak memory model can be an issue for code not intended for use with threads, but Racket— more precisely, Chez Scheme— always guarantees the memory safety of such code using memory fences. That is, Racket code might observe out-of-order writes, but it never observes ill-formed Racket objects. The fences are not new, and they are part of the same write barrier that already supports generational garbage collection and the memory safety of futures. Although sequential consistency supports lock implementations that don’t work with weaker memory models, so they would work with coroutine threads and not parallel threads, we have not found any such implementations in Racket libraries.

  • Some Racket libraries use atomic mode for concurrency control. Atomic mode in Racket prevent coroutine thread swaps, and entering atomic mode is a relatively cheap operation within Racket’s coroutine scheduler. When a parallel thread enters atomic mode, then it prevents other coroutine threads from running, but it does not prevent other parallel threads from running. As long as atomic mode is used consistently to guard a shared resource, then it continues to serve that role with parallel threads.

    Entering atomic mode is a much more expensive operation in a parallel thread than in a coroutine thread; in many cases, Racket core libraries that need finer-grained locking more specifically need to move away from using atomic mode. Still, making atomic mode synchronize a parallel thread with coroutine thread provides a graceful fallback and evolution path.

  • Foreign functions that are called by Racket in a coroutine threads are effectively atomic operations when there are no parallel threads, since a coroutine swap cannot take place during the foreign call. It’s rare that this atomicity implies any kind of lock at the Racket level, however, and the foreign function itself is either adapted to operating-system threads or not. Racket can already create operating systems threads through dynamic-place , and foreign-function bindings have generally been adapted already to that possibility.

The greater degree of concurrency enabled by parallelism exposed some bugs in our existing core libraries that could have been triggered with coroutine threads, but hadn’t been triggered reliably enough to detect and repair the bugs before. Beyond those general improvements, our experience with pre-release Racket is that parallel threads have not created backward-compatibility problems.

Svalboard - Datahand forever

Lobsters
svalboard.com
2025-11-23 09:10:58
Comments...
Original Article

svalboard lightly

Banish pain with the most adaptable ergonomic keyboard and mousing instrument ever made.

  • Magnetic = Light + Tactile

    Lightly brings fifty perfectly light, perfectly tactile magneto-optical keys to your fingertips, with integrated pointing for completely hands-down operation.

    Read about the unique magnetic key mechanism

  • Custom Anatomical Fitment

    The unique Svalboard fitment mechanism gives a perfect fit for any adult hand, accommodating finger length, fingertip size and palm size variation.

    Read about the Fitment System

  • Life-changing Results

    "Over the past six months, the Svalboard has transformed the way I work, helping alleviate my neck and shoulder issues by allowing me to type without strain. Svalboard isn’t just another keyboard—it’s a game-changer for comfort and productivity."

    - Jan Lunge

    Watch Jan's video and more

  • Mount Anywhere

    Built with a ~15° tent angle and M5 tenting screw inserts for extra tenting. It also has 1/4"-20 inserts for additional mounting options: chair-mount, lap desk, keyboard tray and Magic Arm compatible. Pictured here with SmallRig Rosette 11" Magic Arms

  • Multiple Pointing Options

    Trackball, Trackpoint, and now: Touchpad!

    Mix-and-match these integrated pointing devices for maximum comfort.

    Buy now

  • Self-Print Kit Option

    Print and assemble your own with our self-print kit and great community support.

    Buy it here

QWERTY and custom layouts

QWERTY touch typers start with their fingers on the home-row: A,S,D,F for the left hand and J,K,L,; for the right hand. On Svalboard these are exactly the same. Your fingers rest on the round center keys of each key cluster. Then the G and H are simply inward flicks of the index fingers. The remaining four letters: T, Y, B and N are assigned new positions, becoming inward flicks of the middle and ring fingers. The thumb keys switch between different layers: (1) Letters , (2) Numbers , (3) Functions and (4) Navigation .

Alternative layouts like Dvorak or Colemak are easy to set up because the Svalboard Lightly is infinitely customizable. It uses Vial-QMK to change keys in real-time with no flashing required.

Learn more about Layouts

Perfect fit for any hand

Big hands? Small hands? Wonky finger lengths? Broke a pinky when you were three and now it's all janky? We've got you.

Individual 5-axis finger and thumb adjustability for natural hand posture.

3 axis palmrest adjustability for ultimate comfort and hand sizing.

1/4"-20 mounting hardware for tenting, chair and clamp-on mounting using standard camera hardware or just M5 screw inserts for tenting legs.

Buy Now

Light, small, tactile

No more springs or lubed switches. Svalboard's keys are light 20gf magneto-optical keys that have a 100% front-loaded force profile and a clean breakaway unlike anything else on the market. Fingers move just a few mm for any keypress. This reduces hand workload and fatigue by ~90% vs traditional keyboards, even "ergonomic" ones.

No more contortions or reaches to hit the keys at the edge of your keyboard.

Learn More

100% Customizable layout

Built on the amazing QMK engine, with massive community support.

Default QWERTY, Colemak and Dvorak layouts included.

Remap keys, layers, macros, combos, and use tap-dance functions -- all without reflashing thanks to the magic of Vial.

Learn about Vial

From the creator:

"As a Datahand user of 20+ years, I can safely say that Datahand saved my career. I've tried everything over the years, and there's simply no comparison. When the company died, I was gutted. When one of my precious Datahand units got damaged during travel, I got serious about building a replacement.

It was around then I found out about Ben Gruver's amazing lalboard design and built one myself, with lots of help from him.

Lalboard laid the foundation of basic key mechanisms which enabled Svalboard to come to life, but... it was not a production-oriented design. With my background in high volume consumer electronics and input tech development, I decided it was time for me to dive in and make it a reality.

Like Datahand. I hope Svalboard will help thousands of RSI sufferers like me enjoy a faster, safer, more precise, and most of all PAIN FREE typing experience."

Built for the long now

In the spirit of the Svalbard Global Seed Vault, we're committed to a living design which can always be repaired, updated and modified by customers.

All mechanical parts can be produced on a hobby-grade 3D printer, from SLA, or CNC'd from any material you like.

All pinouts are available if you want to hack your own electrical modifications, too.

Join the Discord

Robust and maintainable

In the proud tradition of the Datahand, we've designed Svalboard to be robust, easy-to-clean, and repairable.

Unlike Datahand, Svalboard is totally modular, for easy repairs and modifications.

There are no electromechanical contacts to wear out.

Keys are easily removable for cleaning.

Did we mention you can make your own replacement/custom-fit parts on a hobby-grade 3D printer?

Proudly made in the USA , with a global following

People in over two dozen countries are enjoying pain-free typing with Svalboard Lightly today. Join them on the Discord and learn how Lightly changed their lives:

Join the Discord

After my dad died, we found the love letters

Hacker News
www.jenn.site
2025-11-23 08:40:41
Comments...
Original Article

a few days after dad died , we found the love letters, hidden away among his things. one of them said, i love dota and i love peaches, but i love you more. i will quit smoking and lose weight for you. the happiest days of my life are the ones that start with you across the breakfast table from me.

my parents were not a love match. at 27 and 26, they were embarrassingly old by the standards of their small chinese port town. all four of my grandparents exerted enormous pressure to force them together.

my father fulfilled the familial obligations heaped on his shoulders without complaint. he didn't get along with my mother, or my younger brother, but this wasn't too bad; he often worked away from us (for months and even years on end), mostly in china, more recently in redacted, another canadian city.

the physical distance between us for most of my life has made his passing easier for me to come to terms with. i call him dad here but i didn't lose a dad, i lost someone who was abstractly a father to me. he was more often gone than there, had missed all of my graduations and birthday parties. there was one time he took care of me when i was sick. his hands on me were gentle, and he told me stories from chinese history while i lay feverish in bed. i was seven. this is approximately the only memory i have of him being a dad to me.

still, the two of us were close in our own way. sometimes, the two of us would go on long walks together. after fifteen minutes of silence, or twenty, something would loosen in him and he would start to tell me about the depths of his sadness and the disappointment in the way his life played out. i was good at not taking this personally. i didn't think he ever had a chance to be happy or authentic, his entire life. he sacrificed himself so i could.

i always thought that if he had a chance at happiness, he would be the gentle, funny, and sensitive aesthete that i caught glimpses of sometimes, instead of the bullheaded chinese patriarch others seemed to demand.

except it turns out he did have this chance after all. his lover and i ended up meeting soon after his death. edward lived in redacted, the city that my dad had worked in for the past year and a bit.

edward tells me their story, all in a rush. he and my dad had been seeing each other for three years, and had agreed to go exclusive a year and a half ago. they met while he was in china, and there was an instant spark between them, something special and precious that neither of them had felt before. dad convinced him to apply for a university program here in canada, to eventually get permanent residency in canada. so edward, in his 30s, sold his flourishing business and his house, and came to start over in a foreign land for the sake of being with him.

edward reckons they were engaged, or something like it; they lived together, toured open houses in redacted every weekend with every intent to buy something together, and there was an understanding that dad would soon come out, divorce my mother, and live in the open with edward for the rest of their lives.

edward gave me some photos he had of my dad, and i could scarcely believe that they were of the grim, sad man i knew. he beams in all of them, glowing with joy, his smile more incandescent than i've ever seen in my entire life. i steal glances at edward, the person who took all those impossible photos. the person he was looking at.

my mind keeps stuttering to boskovitch's installation, that single box fan behind plexiglass. i imagine the course of events from edward's point of view: a year living with the love of your life, and then they are suddenly gone in an awful accident and you are too late to see them one last time, to attend the funeral. your own grief is an isolating thing because you are closeted and no one else knew who you were to each other. i wish we had gotten in touch sooner, but edward is grateful to be allowed any affordance, at all.

their life in redacted seemed similarly impossible: a life where my dad splurged on the treats he never did at home (hagen dazs ice cream, honeycrisp apples, nice shoes) and left the house on a regular basis to explore the city with the one he loves. a life where he felt safe enough to ask for kisses and cuddles because he knew they would be provided, even to sa jiao playfully. all i ever knew him to do at home was to sit in a stupor by the television set.

and there was a new hurt, but it was sweet, to imagine the way life could have been in ten years time, a life i've never previously imagined; dad happily with edward in a nice new house where i'd visit every so often, shoulders loose and smiling, and we'd get to talk, actually talk.

according to edward, my dad had known that he had liked men at least since his university years. that makes it almost forty years in the closet, then; just thinking about it makes me feel a sort of dizzying claustrophobia.

i came out to mom years before i came out to dad. when i did, mom told me that coming out to dad was not a good idea, because he was such a traditionalist and she didn't know how he would react. but i came out to him anyways, one quiet afternoon when i visited him in china, because i thought our relationship was good and that he can handle it, and i wanted him to know this about me.

when i did, he took it well. he told me that though the path i am on is a painful one, he would be there for me, and that the most important thing was to find xin fu in life, not to live your life in accordance to the expectations of anyone else. in my staggering relief i did not notice the confusion. i just felt so grateful to have had that understanding, a precious gift that i did not have any expectation of receiving. now, i feel only bereft of the conversations we never managed to have, and grief for the life he never got to live.

dad lives in my living room these days, in a box made of cherry wood, because mom didn't want him in the house after the truth came out. so when edward visited, he got to see him one last time, and say goodbye. he held the box in his arms and wept, spilling more tears and emotions than his biological family managed to, and i escaped to my room for the evening to give them some privacy.

did i mention the shrines? we set them up for the dead in our culture. we had ours, a formal thing in a cabinet, and we had knelt in front of it like we were supposed to, given the correct number of kowtows. edward shared with me pictures of his. it sprawled over the entirety of his dining table. it had packs of playing cards from the brand he liked best and his favourite cuts of meat and the wine he finished off the day with. every morning, he would play my dad's favourite songs to him. i didn't know my dad's favourite cuts of meat. i didn't know he drank wine. i didn't know he listened to music.

so of course i let them say goodbye to each other. when i went out of my room the next morning, he was still fully dressed on my couch, bedding untouched, staring blankly at the box in his lap. it gleamed red in the morning sun. he rose at my approach, put my dad back on the mantle with gentle hands, and then stood quietly at a perfect parade rest in front of him as i managed breakfast for the two of us. his flight back to redacted was that afternoon.

i don't know how to thank you for all this, he says. the chance to say goodbye. he was really proud of you, he spoke about you to me all the time. he never told me that you were gay. edward tells me that dad had plans to go back to redacted in a few weeks time and that he wanted to tell me everything before he left, but he was anxious about how i'd take it. i don't ask edward how many times he'd made the resolution to tell me before.

because you see, my dad was a coward. mom had started asking for divorces by the time i was in my teens, and dad was the one who always said no. he would complain to her mother, a traditionalist, to ensure that she would berate her daughter back into line. his family and his culture had no place for him, so he used her as a shield to make sure that he would be spared the scrutiny. slowly, we found evidence of other affairs, going back decades. of course my mother did not want him in the house.

i sit by my dad sometimes, and i make sure he always has a bowl of fresh fruit. fifty seven years, most of them suffocating and miserable, the last three of them shot through with so much joy his smile absolutely glows.

he wasted his entire life , my mom said to me, the evening we found the love letters. his entire life, and mine as well.

( dissolution )

#diary #longform

qsp: A simple S-Expression parser for Rust TokenStreams

Lobsters
github.com
2025-11-23 08:23:56
Comments...
Original Article

Quick S-Expressions Parser (QSP)

This crate implements as small parser for token streams holding S-Expressions. The S-Expression syntax is very minimal and not conforming to any standard. The only dependency is proc-macro2 , so it should compile very fast.

Why ?

The goal of this library is to make it as convenient as possible to write a whole range of proc macros. Writing a proc macros usually takes a lot of effort. Most of that effort goes into parsing the input, typically using syn , which is incredibly powerful, but very tedious, and slow to compile. If you don't need to bother yourself with parsing, and have an easy time accessing the results of said parsing, most of the trouble of writing proc-macros is gone already.

Additionally, QSP is very simple and only has a single dependency, which should make it compile pretty fast.

Sadly, you will still have to create a macro-crate for your proc-macro. To reach true lisp-macro-convenience, we still require in-package proc-macros

Examples

You can run this by running cargo run --example pipe . It implements a simple pipe macro. If you were to put this into a macro-crate and name it pipe , an invocation in actual code would look just like the input_string except with pipe! before the opening parenthesis. And you would need to turn the result string into a TokenStream again, of course.

use anyhow::Result;
use proc_macro2::TokenStream;
use qsp::Expr;

fn main() -> Result<()> {
    let input_string = r#"
    ( input
        (.strip())
        count_vowels
        { |x| {println!("There are {x} vowels in {input}"); x}}
        ( * 2)
        { |x| println!("you get {x} points for that") }
    )
    "#;

    let token_stream: TokenStream = input_string.parse().unwrap();
    let ast = qsp::parse(token_stream).unwrap();
    let (input, steps) = ast.head_tail_split()?;
    let mut call = input.to_string();
    for step in steps {
        match step {
            Expr::Literal(_) => {
                panic!("steps cannot be literals");
            }
            Expr::Identifier(ident) => {
                call = format!("{ident}({call})");
            }
            Expr::Operator(_) => {
                panic!("steps cannot be operators");
            }
            Expr::RustExpr(token_tree) => {
                call = format!("({token_tree})({call})");
            }
            elems @ Expr::List(_) => call = format!("({call}) {elems}"),
        }
    }

    println!("Resulting call:\n{call}");
    Ok(())
}
  

The following functions are defined on the Expr type, which make it very easy to use. The error messages contain as much information as possible, and should make it easy to catch mistakes, even if you just liberally use ? everywhere.

  • as_literal(&self) -> CastResult<&Literal>
  • as_str_lit(&self) -> CastResult<StrLit>
  • as_identifier(&self) -> CastResult<&Ident>
  • as_operator(&self) -> CastResult<&TokenTree>
  • as_rust_expr(&self) -> CastResult<&TokenTree>
  • as_slice(&self) -> CastResult<&[Expr]>
  • head_tail_split(&self) -> Result<(&Expr, BorrowedList<'_>), HeadTailSplitError>
  • pair_split(&self) -> Result<(&Expr, &Expr), PairSplitError>
  • try_flat_map<F, T, E, R>(&self, f: F) -> Result<Vec<T>, TryFlatMapError<E>

BorrowedList reimplements all list-related functions.

State

This is still a proof of concept. I intend to use it the next time I need a proc-macro, but that hasn't happened yet. It currently serves as an example of an idea.

Google Revisits JPEG XL in Chromium After Earlier Removal

Hacker News
windowsreport.com
2025-11-23 06:05:51
Comments...
Original Article

X INSTALL BY CLICKING THE DOWNLOAD FILE

Three years ago, Google removed JPEG XL support from Chrome, stating there wasn’t enough interest at the time. That position has now changed.

In a recent note to developers, a Chrome team representative confirmed that work has restarted to bring JPEG XL to Chromium and said Google “would ship it in Chrome” once long-term maintenance and the usual launch requirements are met.

The team explained that other platforms moved ahead. Safari supports JPEG XL, and Windows 11 users can add native support through an image extension from Microsoft Store. The format is also confirmed for use in PDF documents. There has been continuous demand from developers and users who ask for its return.

Before Google ships the feature in Chrome, the company wants the integration to be secure and supported over time.

Chrome JPEG XL implementation adds animation support

A developer has submitted new code that reintroduces JPEG XL to Chromium. This version is marked as feature complete. The developer said it also “includes animation support,” which earlier implementations did not offer. The code passes most of Chrome’s automated testing, but it remains under review and is not available to users.

The featured image is taken from an unlisted developer demo created for testing purposes.

JPEG XL is a newer image format intended as a replacement for traditional JPEG files. It can reduce file size without loss in visual quality. This may help web pages load faster and reduce data usage. More details are available on the official JPEG XL website .

Google has not provided a timeline for JPEG XL support in Chrome. Users cannot enable the format today, but development has restarted after years without progress.


Venkat

Venkat Shield

Tech Journalist

Venkat is a tech writer with over 15 years of experience, known for spotting new browser features and tech changes before they go public. Based in India, he breaks down under-the-radar browser updates to help readers stay ahead.


Readers help support Windows Report. We may get a commission if you buy through our links. Tooltip Icon

Read our disclosure page to find out how can you help Windows Report sustain the editorial team. Read more

trifold is a tool to quickly & cheaply host static websites using a CDN

Lobsters
jpt.sh
2025-11-23 05:28:58
I built this to replace my Netlify* workflow for dozens of small static sites & thought others might find it useful. A single config file and you can trifold publish to your heart's content. Unlike free options** it requires a bunny.net CDN account, but you can host as many of your static sites ...
Original Article

trifold is a tool to serve static websites using a content delivery network (CDN).

Source: https://codeberg.org/jpt/trifold/

This allows painless deployment of sites consisting entirely of static assets (HTML, CSS, JS, images) for pennies a month. It is the perfect companion to deploy sites built with static-site generators like Hugo , Zola , Quarto , or zensical .

The tool provides a simple CLI that allows:

  • initializing new projects without touching the CDN web interface
  • syncing local HTML/CSS/JS/etc. to the CDN & clearing the cache
  • configuring a custom domain name to point at your files, with SSL enabled
  • setting a maximum monthly cost to avoid surprise bills
  • using CDN edge functions to support redirects

This project grew out of frustration with services making their free tier less friendly to indie devs and students that just need a cheap & reliable place they can host things. trifold offers an easy alternative to services like Cloudflare Pages, Netlify, and GitHub pages.

Instead of relying on a free service it is hopefully going to be more stable to rely on a paid service with a reasonable price point and the ability to set billing limits.

Installation

If you have uv installed, you can run uvx trifold without a separate installation step.

This is the recommended method, uvx is a built-in alias for uv tool run . (See https://docs.astral.sh/uv/guides/tools/#running-tools if you're curious to learn more.)

To upgrade: uv tool upgrade trifold

You can also use any Python package manager you prefer, pip , pipx , poetry , etc. to install in your preferred manner.

bunny.net & pricing

At the moment bunny.net 1 is the only supported provider, but others can be added.

bunny.net is a professional-grade CDN that is also very affordable.

Like most hosts, they charge for both storage & bandwidth. Both starts at $0.01/GB/mo.

The typical static webpage is under 1GB, meaning storage costs will be negligible unless you decide to host audio/video. And if you do, the rates are far cheaper than most competitors, see their pricing for details.

In terms of bandwidth, let's say your page size is 2MB (a moderate-sized page) and hits the front page of a popular website, driving a surge of 25,000 hits. ( Congrats! )

Not only will your site handle the traffic just fine, your total bill will be $0.50 for the 50GB of bandwidth used. (You could serve a million hits, ~2TB, for $20.)

Of course, most sites will only get a fraction of this traffic.

It is possible to host dozens of low-traffic sites for the $1/monthly minimum bill.

Cryptographers Held an Election. They Can't Decrypt the Results

Hacker News
www.nytimes.com
2025-11-23 03:47:17
Comments...
Original Article

Please enable JS and disable any ad blocker

MCP Apps just dropped (OpenAI and Anthropic collab) and I think this is huge

Hacker News
blog.modelcontextprotocol.io
2025-11-23 03:27:04
Comments...
Original Article

Today we’re introducing the proposal for the MCP Apps Extension ( SEP-1865 ) to standardize support for interactive user interfaces in the Model Context Protocol.

This extension addresses one of the most requested features from the MCP community and builds on proven work from MCP-UI and OpenAI Apps SDK - the ability for MCP servers to deliver interactive user interfaces to hosts .

MCP Apps Extension introduces a standardized pattern for declaring UI resources, linking them to tools, and enabling bidirectional communication between embedded interfaces and the host application.

Example of an inline chat app with interactive UI for permission management

The SEP was authored by MCP Core Maintainers at OpenAI and Anthropic, together with the MCP-UI creators and lead maintainers of the MCP UI Community Working Group.

Standardization for interactive interfaces

Currently, MCP servers are limited to exchanging text and structured data with hosts. While this works well for many use cases, it creates friction when tools need to present visual information or gather complex user input.

For example, consider a data visualization MCP server that returns chart data as JSON. The host application must interpret that data and render it. Handling all kinds of specialized data in this scenario translates to a significant burden for client developers, who would need to build their own logic to render the UI. As more UI requirements come up, like the need to collect multiple related settings from users, the complexity balloons. Alternatively, without UI support, these interactions become awkward exchanges of text prompts and responses.

The MCP community has been creative in working around these limitations, but different implementations using varying conventions and architectures make it harder for servers to work consistently across clients. This lack of standardization creates a real risk of ecosystem fragmentation - something we’re working to proactively prevent.

Building together

The MCP-UI project , created by Ido Salomon and Liad Yosef and maintained by a dedicated community, spearheaded the vision of agentic apps with interactive interfaces. The project developed patterns for delivering rich user interfaces as first-class MCP resources, proving that agentic apps fit naturally within the MCP architecture. The project is backed by a large community and provides rich SDKs , adopted at leading companies and projects such as Postman, Shopify, Hugging Face, Goose, and ElevenLabs.

The OpenAI Apps SDK further validated the demand for rich UI experiences within conversational AI interfaces. The SDK enables developers to build rich, interactive applications inside ChatGPT using MCP as its backbone. To ensure interoperability and establish consistent security and usage patterns across the ecosystem, Anthropic, OpenAI, and MCP-UI are collaborating to create an official MCP extension for interactive interfaces.

Example of a fullscreen app with a rich data table interface

MCP Apps Extension specification

We’re proposing a specification for UI resources in MCP, but the implications go further than just a set of schema changes. The MCP Apps Extension is starting to look like an agentic app runtime: a foundation for novel interactions between AI models, users, and applications. The proposal is intentionally lean, starting with core patterns that we plan on expanding over time.

Key design decisions

Pre-declared resources

UI templates are resources with the ui:// URI scheme, referenced in tool metadata.

// Server registers UI resource
{
  uri: "ui://charts/bar-chart",
  name: "Bar Chart Viewer",
  mimeType: "text/html+mcp"
}

// Tool references it in metadata
{
  name: "visualize_data_as_bar_chart",
  description: "Plots some data as a bar chart",
  inputSchema: {
    type: "object",
    properties: {
      series: { type: "array", items: .... }
    }
  },
  _meta: {
    "ui/resourceUri": "ui://charts/bar-chart",
  }
}

This approach enables hosts to prefetch and review templates before tool execution, improving both performance and security. It also separates static presentation (the template) from dynamic data (tool results), enabling better caching.

MCP transport for communication

Instead of inventing a custom message protocol, UI components communicate with hosts using existing MCP JSON-RPC base protocol over postMessage . This means that:

  • UI developers can use the standard @modelcontextprotocol/sdk to build their applications
  • All communication is structured and auditable
  • Future MCP features automatically work with the UI extension

Starting with HTML

The initial extension specification supports only text/html content, rendered in sandboxed iframes . This provides:

  • Universal browser support
  • Well-understood security model
  • Screenshot and preview generation capabilities
  • A clear baseline for future extensions

Other content types such as external URLs, remote DOM, and native widgets are explicitly deferred to future iterations.

Security-first

Hosting interactive content from MCP servers requires careful security consideration. The proposal addresses this through multiple layers:

  1. Iframe sandboxing : All UI content runs in sandboxed iframes with restricted permissions
  2. Predeclared templates : Hosts can review HTML content before rendering
  3. Auditable messages : All UI-to-host communication goes through loggable JSON-RPC
  4. User consent : Hosts can require explicit approval for UI-initiated tool calls

These mitigations create defense in depth against malicious servers while preserving the flexibility developers need.

Backward compatibility

MCP Apps is an optional extension. Existing implementations continue working without changes, and hosts can gradually adopt UI support at their own pace. Servers should provide text-only fallback for all UI-enabled tools and return meaningful content even when UI is unavailable, so they can serve both UI-capable and text-only hosts.

What’s next

The UI Community Working Group has been instrumental in shaping this proposal through extensive feedback and discussion. We have built an early access SDK to demonstrate the patterns and types described in the specification proposal. The MCP-UI client and server SDKs support these patterns.

If you are interested in contributing to this effort, we invite you to:

Acknowledgements

This proposal wouldn’t exist without the work of the maintainers at MCP-UI, OpenAI, and Anthropic.

Ido Salomon and Liad Yosef , through MCP-UI and moderation of #ui-wg , incubated and championed many of the patterns that MCP Apps now standardizes, and together with contributors demonstrated that UI resources can be a natural part of MCP.

Sean Strong , Olivier Chafik , Anton Pidkuiko , and Jerome Swannack from Anthropic helped steer the initiative and drive the collaboration.

Nick Cooper , Alexei Christakis , and Bryan Ashley from OpenAI have provided valuable direction from their experience building the Apps SDK.

Special thanks to the UI Community Working Group members and everyone who contributed to the discussions that shaped this proposal.

Enough With All The Raft

Lobsters
transactional.blog
2025-11-23 02:54:26
Comments...
Original Article

This talk is an extension of my earlier Data Replication Design Spectrum blog post. The blog post was the analysis of the various replication algorithms, which concludes with showing that Raft has no particular advantage along any easy analyze/theoretical dimension. This builds on that argument to try and persuade you out of using Raft and to supply suggestions on how to work around the downsides of quorum-based or reconfiguration-based replication which makes people shy away from them.

Video

Transcript

slide 1

Hi folks. I’m here to try and convince you to consider options other than Raft.


slide 2

Raft, or just leadered consensus in general and I’m using the two interchangeably in this talk, has emphatically won both on actual usage in databases by my somewhat haphazard survey…


slide 3

And even more subjectively it’s won by mindshare. Any discussion I see of replication is always about raft. (and this is edited, throughout this whole talk, I’m not trying to subtweet any one person/project/whatever) But it’s always Raft. Or multi-paxos. Or that viewstamped replication should be the one true replication algorithm. And this grates on me, because if you’re choosing between three options, those aren’t even the right three to be considering.


slide 4

I claim there’s three classes of replication algorithms [1] : Quorums, Reconfiguration, and leadered consensus as a hybrid of the two, and that all replication algorithms can be placed along a single axis which classifies them based upon how they handle failures. With quorums, the loss of any member of the replication group can be tolerated, and replication continues on. Think Cassandra. With reconfiguration, the write-all-read-one replication halts on a failure, and continues once the failed node has been automatically replaced. Historically, this is like MySQL with failover. And finally our overused Raft exists as a hybrid of the two: the followers act like quorum replication, but having a leader bumps it one tick towards reconfiguration. [1]: This is the one slide summary of what Data Replication Design Spectrum tries to pitch in terms of classification.


slide 5

And so this talk is framed as trying to argue my hypothetical arch-nemesis out their mental model here that Raft is the absolute best and always the correct default option, and anything else should only be used begrudgingly in some very specific cases. I’m actually trying to get to the argument of: please just use the best suited replication algorithm, but that’s going to involve some Raft bashing while sprinkling in advice on how to succeed in a non-raft world.

So let’s get started.


slide 6

We’re going to first tackle the broad argument that raft is just uniformly superior. And if you tell me it’s best, I want to know, it’s best at…​ what?


slide 7

If it’s the best at something, I should be able to sit down, and do the math of how it acts along some dimensions versus the alternatives, and show, inarguably, that raft delivers better something than the alternatives. But I’ve done that math. I have a blog post which calculates Quorums, Raft, and Reconfiguration along these dimensions, with every notable variant or proposed raft optimization factored in.


slide 8

And that post shows: Raft isn’t better. In every category, it’s at best tied, and at worst, it’s the worst. Most distributed database deployments I’ve worked with have been storage bound, and that 40% higher storage efficiency for reconfiguration can mean a lot of money. Or if you care about availability, on paper, leaderless Paxos gives you better tail latencies with less availability blips than Raft. So the math isn’t justifying Raft’s absurd popularity.


slide 9

There’s also this draw to Raft that it’s great because of its simplicity. It’s simpler than Multi-Paxos, for sure, it did a great job at that.


slide 10

But in the in the broader picture, Raft isn’t simpler. Quorums have different replicas with different states and different orders of operations causing an explosion of states to check for correctness. But once you’ve handled that, all distributed systems problems of slowness, failures, partitions, what-have-you all look the same.

Reconfiguration is the opposite. I’ve worked on FoundationDB, a very reconfiguration-based databases, and whenever some code sends an RPC, either it gets a reply or everyone gets killed and the system resets. All the code is happy-path only, as all failures get pushed through one reconfiguration process. It’s beautifully simple. But gray failures are hard, and having to precisely answer “is this other replica sufficiently alive?” is the challenge that Reconfiguration gains instead.

And Raft is both of these things, so not only do you have to have a well-integrated failure detector for the leader, but you also have a tremendous state space to search in which bugs could be hiding from the quorum of followers. It’s not simpler.


slide 11

One could argue "Raft is better than Reconfiguration because Reconfiguration has unavailability!"

This is the reconfiguration counter-argument I have encountered the most often, and this is my least favorite argument, because it’s like a matryoshka of misunderstandings.


slide 12

First, If you’re so upset about unavailability, what happens when the leader dies in raft? Request processing halts, there’s a timeout, a reconfiguration process (leader election), and requests resume.


slide 13

What happens when you use reconfiguration and a replica dies? Request processing halts, there’s a timeout, a reconfiguration process, and requests resume. It’s literally the same diagram. I just deleted some nodes. If you’re upset about this slide, you have to be equally upset about the last slide too.


slide 14

Furthermore, if we’re talking about replicating partitions of data, then leadership gets distributed across every machine to balance resource usage as leaders do more work. So when a machine fails, some percentage of your data is going to be "unavailable", we’re only arguing about exactly what that percent is. So, no.


slide 15

Furthermore, it’s an argument based out of a bad definition of the word availability. Unavailability is when requests have latency above a given threshold. If the reconfiguration process happens within your latency threshold, it’s not unavailability.


slide 16

The Huawei Taurus paper has an argument for reconfiguration-based replication in this vein, which is a bold argument and I love it.


slide 17

They’re building replication for a write ahead log, and are making a case here about their write availability for appending a new log segment.

They say:

  • We can identify a failure quickly.

  • Our reconfiguration process is fast.

  • The chance of us being unable to find 3 new working nodes is effectively 0.

  • Therefore our change of being unavailable is effectively 0%.

And that’s the correct way to look at availability. You can hate this argument, you can still poke some minor holes in it, but they’re not wrong.


slide 18

There is a correct counter-argument here, and it’s that you cannot solve consensus with two failures using three nodes. So when raft is electing a new leader or changing its replicas, it can do that itself. Reconfiguration-based replication needs some external consensus service to lean on. But the options of what you can use for that are ever more plentiful. With S3 supporting compare-and-swap now, you can even use S3 as your consensus service. But this is a design requirement difference from Raft.


slide 19

For concrete advice on how to build systems using an external consensus service to manage membership, the PacificA paper gives a very nice description of how to do this, and how manage an automatic failover and reconfiguration process safely. It has already been directly adopted Elasticsearch, and Kafka’s replication is very similar in spirit.


slide 20

Moving onto the Quorums side, one could argue "Raft is better than Quorums because Quorums livelock on contention!"

Simple majority quorums doesn’t livelock, so we’re talking about leaderless consensus here only, and this is a known concern. But there’s ways to minimize or work around this issue. [2] [2]: Unmentioned in this talk is "just put the replicas closer together", like Tencent’s PaxosStore , because that’s not as general of advice.


slide 21

First, don’t keep the raft mental model that operations need to go into a log, and all operations need to go into one log. Target your operations to the specific entity or entities that you’re modifying, so that you contend only on what you actually need to.


slide 22

You don’t even need to materialize a log if you don’t need a log. Compare-and-Swap Paxos , just models evolving your entity from one state to the new state with no “put things into a log” step in-between. And it’s a great example of being simpler than Raft — Denis’s example implementation with membership changes is 500 lines of code.

If you’re looking for a weekend implement consensus project, this is what I’d recommend doing.


slide 23

Second, and this is the trick I see applied the least often, but remember that even when modifying the same entity, you don’t need to have all replicas agree on an ordering for commutative operations — those which yield the same result regardless of what order they’re performed in. Increments are the easiest example. Every replica agrees that at the end it’s a net plus six here, and this is safe to do as long as no one sees an intermediate result.


slide 24

Permitting commutative operations to commit concurrently while banning reads requires cooperation from your concurrency control layer too. You can read about increment locks in database textbooks, but escrow transactions is the most fun. If I try to deposit $100 and withdraw $100 from my bank account, those might be commutative operations. If I have zero dollars, it matters if the withdrawal gets ordered before the deposit. If I’m a billionaire, it doesn’t matter. Escrow Transactions pitches how to handle even these sorts of "conditionally commutative" situations so that you can get your contention down as low as possible.


slide 25

Lastly, the livelock stems from inconsistent ordering of requests across replicas, and you can also take a dependency on physical clocks to help consistently order requests instead. There’s an E-Paxos Revisited [3] paper which gives a focused pitch on this idea as well, but I’d strongly suggest checking out Accord , Cassandra’s new strictly serializable transaction protocol, that’s an industry implementation of leaderless consensus, and avoiding livelock by leaning on a physical time based ordering. [3]: E-Paxos is the classic example of targeting only the entities one wishes to modify within paxos, but there’s aspects of it which haven’t been fully scoped out for real-world implementation. Most of these are centered around that E-Paxos maintains a DAG of operations (where edges are conflicts) which makes a number of aspects of a real system (e.g. replica catchup or garbage collection) significantly harder to do efficiently. I only know of Cassandra having an implementation of it which was never merged, and they ended up going towards extending E-Paxos into Accord instead.


slide 26

So to wrap this up, I’m not here to pitch you that Raft never has a use. Going through these arguments was to show that there are limitations to Quorums and Reconfiguration, and talk about how you can best work around those limitations. But each side has a critical flaw, and the one advantage that Raft uniquely has, is its unrelenting, unwavering mediocrity. It is less efficient, it is less “available”, and it is more complicated, but there’s no situation in which Raft isn’t an “okay” solution. It’s a safe choice. But, broadly, categorically, and littered with minor factual issues, not using raft gets you a system thats’s better at something.


slide 27

So the mental model I’d like to leave you with is:

  • Use Quorums or Raft if you can’t have any other supporting service to help with group membership.

  • Use Reconfiguration or Raft if you must handle high, single-item contention.

If you need both of these things, then you might have to use Raft. But using Raft is your punishment. You’re forced to use a resource in-efficient, complex solution, because your design constraints left you with no wiggle room.


slide 28

Please use the replication algorithm that best fits your use case. It’s possible that is Raft. That’s fine. But reconfiguration is 40% cheaper by instance count than Raft. If I go to your database’s users and ask if they’re fine with slightly higher tail latency in exchange for 40% off their hardware cost, how many are going to say no? Or if tail latency is really that important to them, would they not be happier with Quourms? Use what fits your users' needs the best.


slide 29

If you’re interested in some further food for thought here, looking at disaggregated OLTP systems is a really interesting replication case study. Each of the major vendors chose a completely different replication solution, and so if you read through the series of papers you see what effects those choices had, and get to read the criticisms that the later papers had of the earlier ones' decisions.


See discussion of this page on Reddit , Hacker News , and Lobsters .

Flux Keyboard

Lobsters
fluxkeyboard.com
2025-11-23 02:49:17
Comments...
Original Article

Pre-orders on this website will ship according to your selected batch. In stock orders will become available after pre-orders have been fulfilled.

Pre-orders on this website will ship according to your selected batch. In stock orders will become after pre-orders have been fulfilled.

The keyboard features a USB-C port on both sides of the device to add on any peripheral such as an external numpad if desired. We plan to release a matching numpad add-on in the future.

Any pre-existing hotkey bindings will function in any software and a custom layout can be made to support it. By default the following layouts will be included at launch with more to come over time:

  • Adobe Photoshop
  • Adobe Illustrator
  • Adobe After Effects
  • Adobe Premiere Pro
  • Adobe Lightroom
  • Ableton Live
  • Autodesk 3ds Max
  • Autodesk Fusion 360
  • Autodesk Maya
  • Blender
  • Cinema 4D
  • Davinci Resolve
  • Discord
  • Figma
  • Final Cut Pro
  • FL Studio
  • Jetbrains IntelliJ
  • Logic Pro
  • Microsoft Excel
  • Microsoft Outlook
  • Microsoft Powerpoint
  • OBS Studio
  • SolidWorks
  • Steinberg Cubase
  • Unity
  • Unreal Engine 4/5
  • Visual Studio Code
  • Microsoft Visual Studio
  • Windows 10/11
  • macOS 11+ (Note: application volume mixing not available for macOS)
  • Linux / Android / iOS – Limited compatibility (see FAQ regarding operation without installed software)
  • Full Linux compatibility is planned but may not be available at launch.

In order to customise and configure the keyboard, the provided Flux Polymath software must be used. However, once a configuration is loaded onto the keyboard’s on-board memory it can be used even on computers that do not have the Flux Polymath software installed. Shortcuts and custom macros consisting of combination keypresses will also still function as well as manual switching between profiles. However, more advanced functionality such as automatic layout switching based on the active window and certain application specific features such as album art and track information for music will not be available if the utility is not installed.

Due to the position of the sensors ortho layouts will not be available. We hope to do a separate ortho split model in the future and the best way to make that happen is to let people know about this product.

Analog hall effect sensors are the best type of sensors for gaming as they allow for adjustable actuation point and rapid trigger functionality which provides some advantages for rapid direction changes or in rhythm games. Polling rate is 1000hz with 1-2ms latency, however the performance may exceed higher polling keyboards as most real world latency comes from the key travel time and debounce.

The keys contain magnets along their perimeter which are attracted by magnets in the frame which surrounds them. This magnetic attraction suspends them in place and provides the return force which makes the key bounce back after depressing – similar to a spring.

No, the keyboard does not function as a touchscreen. This means there is no accidental actuation from resting fingers on the keys.

No, the keyboard is not recognised as a display device by the host computer operating system. It is driven by its own independent efficient embedded microprocessor to provide a more seamless user experience and compatibility with more devices. Also your mouse cursor will never get lost on your keyboard.

No, there is a mechanical end stop built into the frame which prevents keys from hitting the screen.

You will need 1x USB-A or USB-C 2.0 with a minimum power delivery of 5V 1.5A. If you plan to use additional peripherals connected to the keyboard 5V 3A is recommended. The keyboard features a USB-C receptacle for removable cable. If your computer’s USB ports are unable to provide this, a 5V 3A USB-C power supply/charger (sold separately) can be used along with the included USB Y cable.

Yes you can play video files on this, there is 8GB of inbuilt storage for wallpapers and icons. You can easily fit a few 1080p feature length films.

The removable frame allows easy access to the gaps under the keys where dirt generally gathers on a normal keyboard, making cleaning much easier. The frame itself is also totally passive with no electronics within it.

The Flux Keyboard is infinitely customisable, any image or video can be used as a wallpaper with some interactive styles as well. The key legends/symbols and mapping are also completely customisable with support for macros.

Frames will be available in:

  • ANSI 84 Key layout
  • ISO 85 Key layout

The keyboard’s base is compatible with any frame, so it is possible to swap between these layouts if you wanted to.

At launch two switch types will be available:

  • Tactile 55g
  • Linear 45g

The weight can be modified by changing the magnets to magnets of a different strength grade and more varieties are planned to be made available in the future.

GCC SC approves inclusion of Algol 68 Front End

Hacker News
gcc.gnu.org
2025-11-23 02:18:03
Comments...
Original Article
David Edelsohn dje.gcc@gmail.com
Sat Nov 22 20:38:25 GMT 2025
The GCC Steering Committee has agreed to include the Algol 68 Front End in
trunk designated as experimental with stipulations:

1. Algol 68 is not part of all languages built by default.
2. Algol 68 is not part of the GCC release criteria.
3. All GCC developers who are not responsible for the Algol 68 Front End
may decline to work on issues related to the Algol 68 Front End.
4. If the Algol 68 Front End bit rots or is not maintained, it will be
removed.

To permit Algol 68 development to continue, Jose E. Marchesi is appointed
as Algol 68 front end maintainer.

If maintaining a Front End outside of trunk is difficult, the Algol 68
developers are invited to propose concrete changes to make it easier to
maintain front ends in a branch of the GCC repository outside of trunk.

Happy Hacking!
David


More information about the Gcc mailing list

An Economy of AI Agents

Hacker News
arxiv.org
2025-11-23 02:08:00
Comments...
Original Article

View PDF HTML (experimental)

Abstract: In the coming decade, artificially intelligent agents with the ability to plan and execute complex tasks over long time horizons with little direct oversight from humans may be deployed across the economy. This chapter surveys recent developments and highlights open questions for economists around how AI agents might interact with humans and with each other, shape markets and organizations, and what institutions might be required for well-functioning markets.

Submission history

From: Andrew Koh [ view email ]
[v1] Mon, 1 Sep 2025 02:07:39 UTC (56 KB)

dingo: A meta-language for Go

Lobsters
github.com
2025-11-23 01:22:25
Comments...
Original Article

Dingo

Dingo mascot

Go that escaped.

Go Version License Development Status PRs Welcome

Features Quick Start Examples Status Roadmap Contributing


At a Glance

Sum Types: Working | Pattern Matching: Working | Error Propagation: Working | Functional Utils: Working | Playground for Go's Future | v1.0 Target: Late 2025


Look, I love Go. But...

You know that feeling when you're writing Go and you type if err != nil for the 47th time in a single file?

Or when you forget to check for nil and your production server learns what a panic feels like?

Or when you're explaining to a Rust developer why Go doesn't have sum types and they look at you like you just said "we don't believe in seatbelts"?

Yeah. That's why Dingo exists.

What's Dingo?

Think TypeScript, but for Go.

Dingo is a language that compiles to clean, idiomatic Go code. Not some franken-runtime or a whole new ecosystem. Just better syntax that becomes regular Go.

The pitch: Write code with Result types, pattern matching, and null safety. Get back perfect Go code that your team can read, your tools can process, and your production servers can run at exactly the same speed.

Zero runtime overhead. Zero new dependencies. Zero "what's this weird thing in my transpiled code?"

Is this proven to work? Yes. Borgo (4.5k stars) already proved you can transpile to Go successfully. Dingo builds on that foundation with better IDE integration, source maps, and a pure Go implementation.


Why "Dingo"?

Ever wonder what a dingo actually is?

Thousands of years ago, they were domesticated dogs. Well-behaved. Following commands. Controlled.

Then they escaped to the Australian wild and evolved into something science couldn't categorize. Not quite dog. Not quite wolf. Ungovernable.

The Go Gopher? Created at Google. Lives by the rules. Does what it's told.

Dingo broke free.

Here's the beautiful part: dingos are still canines. They didn't reject their DNA—they just refused to be controlled. Same with our language.

Every Go feature still works. Go 1.24 adds something? You get it in Dingo. Day one. Disable all plugins? You're running pure Go.

You're not losing anything. You're gaining freedom without asking permission.

Want pattern matching? Enable it. Want sum types? Already working. Think you can do it better? Fork the plugin and prove it.

Your language. Your rules. No committee required.

See MANIFESTO.md for why this terrifies the establishment.


Quick Start

Note: Dingo is in active development. Phase V Complete - Infrastructure ready for v1.0 with comprehensive documentation, workspace builds, CI/CD enhancements, and 3/4 external model approval.

Latest (2025-11-22) : ✅ LSP Integration Complete - Post-AST source maps (100% accurate position mapping) + auto-rebuild on save. Edit → Save → IDE features work instantly! Hover, go-to-definition, and automatic transpilation fully functional. See CHANGELOG.md for details.

Installation

# Clone the repository
git clone https://github.com/MadAppGang/dingo.git
cd dingo

# Build the compiler
go build -o dingo ./cmd/dingo

# Add to PATH (optional)
export PATH=$PATH:$(pwd)

Your First Dingo Program

Create hello.dingo :

package main

import "fmt"

func main() {
    let message = "Hello from Dingo!"
    fmt.Println(message)
}

Build and run:

# Transpile to Go
dingo build hello.dingo

# Or compile and run in one step
dingo run hello.dingo

Try Working Features Now

Sum Types with Pattern Matching:

enum Result {
    Ok(value: int),
    Error(message: string)
}

func divide(a: int, b: int) Result {
    if b == 0 {
        return Error("division by zero")
    }
    return Ok(a / b)
}

let result = divide(10, 2)
match result {
    Ok(value) => fmt.Printf("Success: %d\n", value),
    Error(msg) => fmt.Printf("Error: %s\n", msg)
}

Safe Navigation and Null Coalescing (Phase 7 ✅):

// Property access with safe navigation
let city = user?.address?.city?.name ?? "Unknown"

// Method calls with safe navigation
let email = user?.getProfile()?.email ?? "noreply@example.com"

// Works with Go pointers too!
let timeout = config?.database?.timeout ?? 30

// Chained defaults
let theme = user?.theme ?? project?.theme ?? global?.theme ?? "light"

Functional Utilities:

let numbers = []int{1, 2, 3, 4, 5}
let doubled = numbers.map(func(x int) int { return x * 2 })
let evens = numbers.filter(func(x int) bool { return x % 2 == 0 })
let sum = numbers.reduce(0, func(acc int, x int) int { return acc + x })

See examples/ and docs/features/ for more working code.


Why Dingo?

The Go Pain Points

  • 47 if err != nil blocks per file
  • Nil pointer panics in production
  • No sum types after 15 years of requests
  • Verbose error handling drowning business logic
  • No null safety operators
  • Boilerplate enums requiring manual type guards

The Dingo Solution

  • ? operator propagates errors cleanly
  • Option<T> makes nil checks compile-time safe
  • enum keyword with full sum type support
  • Pattern matching with exhaustiveness checking
  • ?. and ?? for safe navigation (coming soon)
  • Zero overhead - transpiles to clean Go

Key Insight: Dingo doesn't change Go. It transpiles to it. Your team gets modern ergonomics, your production gets pure Go performance.


The Hidden Superpower: Use Dingo Selfishly, Help Go Evolve Naturally

Here's what makes Dingo special — you get two massive benefits simultaneously :

1. Revolutionize YOUR Codebase TODAY

This is why you'll actually use Dingo:

  • 67% less error handling boilerplate ? operator instead of 47 if err != nil blocks
  • 78% code reduction with sum types — Rust-style enums that just work
  • Zero nil pointer panics — Option types that the compiler enforces
  • Pattern matching — Exhaustive, type-safe, impossible to mess up
  • Same performance — Transpiles to clean Go, zero runtime overhead

Your code becomes cleaner, safer, and more maintainable. Immediately.

This is the selfish reason to use Dingo. And it's a damn good reason.

2. Shape Go's Future (As a Natural Side Effect)

Here's the beautiful part you get for free:

While you're using Dingo to make YOUR code better, you're automatically contributing to Go's evolution:

  • 📊 Your metrics become data — "67% reduction in error handling code across 50 real projects"
  • 🐛 Your bugs find edge cases — Real problems that theoretical debates miss
  • Your usage validates ideas — Proof that features work in production
  • 📚 Your code becomes examples — Concrete demonstrations for Go proposals

You don't have to think about this. It just happens.

The TypeScript Parallel (This Is EXACTLY What Happened)

Developers didn't adopt TypeScript to "help JavaScript evolve."

They adopted it because it made their codebases better.

  • TypeScript added types → Developers used them → JavaScript saw it worked → JavaScript added types
  • TypeScript added async/await → Millions used it → JavaScript saw the value → JavaScript adopted it
  • TypeScript added optional chaining → Everyone loved it → JavaScript added it officially

Developers used TypeScript selfishly. JavaScript evolved as a natural consequence.

Same thing is happening with Dingo and Go:

  1. You use Dingo because it makes error handling less painful
  2. 50,000 other developers do the same thing
  3. Go team sees 2 years of production data showing it works
  4. Go proposal now has concrete evidence instead of theoretical debate
  5. Everyone wins

You're not doing charity work. You're writing better code. Helping Go evolve is just a happy side effect.

Why This Matters More Than You Think

For decades, programming language evolution has been broken:

The old way: Community → Proposal → Years of debate → Maybe no → Frustration

The Dingo way: Developers → Use features → Data emerges → Go team decides with evidence

This is how TypeScript revolutionized JavaScript. Not through proposals, but through proving ideas in production .

The Win-Win-Win Scenario

🎯 You win: Better code today, zero waiting 🎯 Go team wins: Real data for decisions, reduced risk 🎯 Go ecosystem wins: Faster evolution, battle-tested features

Example: Sum Types

Imagine if before Go Proposal #19412 (sum types - 996+ 👍 but rejected), there was:

  • ✅ 50,000 developers using it for 2 years
  • ✅ Concrete metrics: 78% code reduction
  • ✅ Known edge cases documented
  • ✅ Production validation across 5,000+ projects

The proposal would have been impossible to reject with that evidence.

That's what Dingo enables. Every feature you use contributes data that could reshape Go's future.

This Is Not Hypothetical — TypeScript Proved It Works

Here's what actually happened with TypeScript and JavaScript:

Feature TypeScript Added Developers Used It JavaScript Adopted Timeline
Async/Await 2015 Millions of codebases ES2017 2 years
Optional Chaining 2019 Massive adoption ES2020 1 year
Nullish Coalescing 2019 Widespread use ES2020 1 year
Class Fields 2017 Standard in TS code ES2022 5 years
Decorators 2015 Widely used Stage 3 proposal Still evolving

Notice the pattern:

  1. TypeScript adds feature
  2. Developers use it (for selfish reasons - better code)
  3. Real-world data proves it works
  4. JavaScript adopts it with evidence-based confidence

Dingo enables the exact same cycle for Go.

You're not choosing between "make my code better" OR "help Go evolve."

You get both. Automatically. Simultaneously.


Why should you care?

Here's what the Go community has been begging for since 2009:

What developers want How badly What Dingo gives you
Stop typing if err != nil every 3 lines ⭐⭐⭐⭐⭐ Result<T, E> + the ? operator
Stop shipping nil pointer panics ⭐⭐⭐⭐⭐ Option<T> type that the compiler actually checks
Sum types (seriously, it's been 15 years) ⭐⭐⭐⭐⭐ 996+ 👍 enum with pattern matching
Enums that can't be invalid ⭐⭐⭐⭐⭐ 900+ 👍 Type-safe enums with exhaustiveness
Lambda functions that don't take 4 lines ⭐⭐⭐⭐ 750+ 👍 |x| x * 2 like a normal language

The Go team has valid reasons for rejecting these features. They're not wrong about simplicity.

But here's the thing: Dingo doesn't change Go. We just compile to it.

Want sum types? Great. They become tagged structs in Go. Want the ? operator? Cool. It becomes if err != nil checks. Want pattern matching? Done. It's a switch statement underneath.

Your Go code stays pure. Your Dingo code stays sane.


Show Me Code or I'm Leaving

Fair enough.

Code Reduction in Action

Metric Traditional Go With Dingo Savings
Sum Type Definition 33 lines 7 lines 79% less code
Enum with Data 46 lines 10 lines 78% less code
Error Handling Pipeline 85 lines 28 lines 67% less code
API Handler 42 lines 15 lines 64% less code

All numbers from real examples in our test suite

The "if err != nil" problem

What you write in Go today:

func processOrder(orderID string) (*Order, error) {
    order, err := fetchOrder(orderID)
    if err != nil {
        return nil, fmt.Errorf("fetch failed: %w", err)
    }

    validated, err := validateOrder(order)
    if err != nil {
        return nil, fmt.Errorf("validation failed: %w", err)
    }

    payment, err := processPayment(validated)
    if err != nil {
        return nil, fmt.Errorf("payment failed: %w", err)
    }

    return payment, nil
}

75% of this function is error handling ceremony. The actual logic is hiding somewhere in there.

What you write in Dingo:

func processOrder(orderID: string) -> Result<Order, Error> {
    let order = fetchOrder(orderID)?
    let validated = validateOrder(order)?
    let payment = processPayment(validated)?
    return Ok(payment)
}

Same safety. Same error handling. 60% less code.

The ? just means "if this is an error, return it. Otherwise, unwrap the value and keep going." That's it. That's the entire feature.

Rust developers have been using this for 8 years. They love it so much they put it on t-shirts.


Real Working Examples (From Our Test Suite)

These are actual examples from Dingo's test suite that transpile and run today.

Example 1: Simple Sum Type

What You Write (Dingo)

package main

enum Status {
    Pending,
    Active,
    Complete,
}

What You Get (Generated Go)

package main

type StatusTag uint8

const (
    StatusTagPending StatusTag = iota
    StatusTagActive
    StatusTagComplete
)

type Status struct {
    tag StatusTag
}

func Status_Pending() Status {
    return Status{tag: StatusTagPending}
}

func Status_Active() Status {
    return Status{tag: StatusTagActive}
}

func Status_Complete() Status {
    return Status{tag: StatusTagComplete}
}

func (e Status) IsPending() bool {
    return e.tag == StatusTagPending
}

func (e Status) IsActive() bool {
    return e.tag == StatusTagActive
}

func (e Status) IsComplete() bool {
    return e.tag == StatusTagComplete
}

7 lines of Dingo → 33 lines of idiomatic, type-safe Go. All the boilerplate written for you.


Example 2: Sum Type with Associated Data

What You Write (Dingo)

package main

enum Shape {
    Point,
    Circle { radius: float64 },
    Rectangle {
        width: float64,
        height: float64
    },
}

What You Get (Generated Go)

package main

type ShapeTag uint8

const (
    ShapeTagPoint ShapeTag = iota
    ShapeTagCircle
    ShapeTagRectangle
)

type Shape struct {
    tag             ShapeTag
    circleRadius    *float64
    rectangleWidth  *float64
    rectangleHeight *float64
}

func Shape_Point() Shape {
    return Shape{tag: ShapeTagPoint}
}

func Shape_Circle(radius float64) Shape {
    return Shape{
        tag: ShapeTagCircle,
        circleRadius: &radius,
    }
}

func Shape_Rectangle(width, height float64) Shape {
    return Shape{
        tag: ShapeTagRectangle,
        rectangleWidth: &width,
        rectangleHeight: &height,
    }
}

func (e Shape) IsPoint() bool {
    return e.tag == ShapeTagPoint
}

func (e Shape) IsCircle() bool {
    return e.tag == ShapeTagCircle
}

func (e Shape) IsRectangle() bool {
    return e.tag == ShapeTagRectangle
}

10 lines of Dingo → 46 lines of type-safe Go. Rust-style enums without the Rust learning curve.


Example 3: Basic Syntax Sugar

What You Write (Dingo)

package main

func main() {
    let message = "Hello, Dingo!"
    println(message)
    return
}

func add(a: int, b: int) int {
    return a + b
}

What You Get (Generated Go)

package main

func main() {
    var message = "Hello, Dingo!"
    println(message)
    return
}

func add(a int, b int) int {
    return a + b
}

Clean parameter syntax with : separators. Transpiles to standard Go that any gopher can read.


The "Holy Crap" Example

Want to see something beautiful? Here's real-world Go code I found in production:

Go (85 lines of pain):

func ProcessUserDataPipeline(userID string, options *ProcessOptions) (*UserReport, error) {
    // Fetch user
    user, err := db.GetUser(userID)
    if err != nil {
        return nil, fmt.Errorf("failed to get user: %w", err)
    }
    if user == nil {
        return nil, errors.New("user not found")
    }

    // Get user's orders
    orders, err := db.GetOrdersForUser(user.ID)
    if err != nil {
        return nil, fmt.Errorf("failed to get orders: %w", err)
    }

    // Filter valid orders
    var validOrders []Order
    for _, order := range orders {
        if order.Status != "cancelled" && order.Total > 0 {
            validOrders = append(validOrders, order)
        }
    }

    // Calculate totals
    var totalSpent float64
    for _, order := range validOrders {
        totalSpent += order.Total
    }

    // Get user preferences
    prefs, err := db.GetPreferences(user.ID)
    if err != nil {
        return nil, fmt.Errorf("failed to get preferences: %w", err)
    }

    // Apply discount if premium
    discount := 0.0
    if prefs != nil && prefs.IsPremium {
        discount = totalSpent * 0.1
    }

    // Get shipping address
    address, err := db.GetShippingAddress(user.ID)
    if err != nil {
        return nil, fmt.Errorf("failed to get address: %w", err)
    }

    // Format city name
    cityName := "Unknown"
    if address != nil && address.City != nil {
        cityName = *address.City
    }

    // Get payment methods
    payments, err := db.GetPaymentMethods(user.ID)
    if err != nil {
        return nil, fmt.Errorf("failed to get payment methods: %w", err)
    }

    // Find default payment
    var defaultPayment *PaymentMethod
    for i := range payments {
        if payments[i].IsDefault {
            defaultPayment = &payments[i]
            break
        }
    }

    // Get recommendation score
    score, err := analytics.GetRecommendationScore(user.ID)
    if err != nil {
        // Non-critical, use default
        score = 0.0
    }

    // Build report
    report := &UserReport{
        UserID:         user.ID,
        Email:          user.Email,
        TotalSpent:     totalSpent,
        Discount:       discount,
        OrderCount:     len(validOrders),
        City:           cityName,
        HasPayment:     defaultPayment != nil,
        RecommendScore: score,
    }

    return report, nil
}

Dingo (28 lines of clarity):

func ProcessUserDataPipeline(userID: string, options: ProcessOptions) -> Result<UserReport, Error> {
    let user = db.GetUser(userID)?.okOr("user not found")?

    let orders = db.GetOrdersForUser(user.ID)?
    let validOrders = orders.filter(|o| o.status != "cancelled" && o.total > 0)
    let totalSpent = validOrders.map(|o| o.total).sum()

    let prefs = db.GetPreferences(user.ID)?
    let discount = prefs.isPremium ? totalSpent * 0.1 : 0.0

    let address = db.GetShippingAddress(user.ID)?
    let cityName = address?.city?.name ?? "Unknown"

    let payments = db.GetPaymentMethods(user.ID)?
    let defaultPayment = payments.find(|p| p.isDefault)

    let score = analytics.GetRecommendationScore(user.ID).unwrapOr(0.0)

    return Ok(UserReport{
        userID: user.id,
        email: user.email,
        totalSpent: totalSpent,
        discount: discount,
        orderCount: validOrders.len(),
        city: cityName,
        hasPayment: defaultPayment.isSome(),
        recommendScore: score,
    })
}

67% less code. Same functionality. Infinitely more readable.

Look at what just happened:

  • ✅ Error propagation with ? eliminated 12 if err != nil blocks
  • ✅ Lambda functions turned 8-line loops into one-liners
  • ✅ Optional chaining ?. replaced nested nil checks
  • ✅ Ternary operator cleaned up conditional assignments
  • .filter() , .map() , .sum() made collection operations obvious

The business logic literally jumps off the screen now. You can see what it's doing instead of drowning in error handling boilerplate.

This is what Dingo does. It takes your Go code and makes it readable .


Features That Make Dingo Special

Feature Status Description
Sum Types ✅ Working Rust-style enums with associated data
Pattern Matching ✅ Working Exhaustive match expressions with destructuring
Error Propagation ✅ Working The ? operator for clean error handling
Functional Utilities ✅ Working map , filter , reduce with zero overhead
Result & Option ✅ Infrastructure Ready Type-safe error and null handling (integration pending)
Safe Navigation ✅ Complete (Phase 7) ?. operator for properties and methods
Null Coalescing ✅ Complete (Phase 7) ?? operator for default values
Lambda Syntax ✅ Complete (Phase 6) TypeScript arrows and Rust pipes (configurable)
Language Server ✅ Working (Phase 10) Full IDE support via gopls proxy with source maps

Feature Deep Dive

1. Result Type — Error handling for grown-ups

Stop returning (value, error) tuples and hoping callers remember to check both.

func fetchUser(id: string) -> Result<User, DatabaseError> {
    if !isValidID(id) {
        return Err(DatabaseError.invalidID(id))
    }

    let user = database.query(id)
    return Ok(user)
}

// The compiler forces you to handle both cases
match fetchUser("123") {
    Ok(user) => println("Found: ${user.name}"),
    Err(error) => println("Database said no: ${error.message}")
}

Your function signature now tells you exactly what can go wrong. No surprises. No "oh, I didn't know this could return an error."

2. Error Propagation ( ? ) — Because life's too short

This one's from Rust, and it's honestly genius.

func getUserProfile(userID: string) -> Result<Profile, Error> {
    let user = fetchUser(userID)?          // Returns error if this fails
    let posts = fetchPosts(user.ID)?       // Or this
    let comments = fetchComments(user.ID)? // Or this

    return Ok(Profile{user, posts, comments})
}

Every ? is an escape hatch. Hit an error? Jump straight to the return. Got a value? Keep going.

It's like try/catch but without the invisible control flow that makes you question your life choices.

3. Option Type — Nil pointers are a billion-dollar mistake

Tony Hoare (the guy who invented null references) literally apologized for it. Called it his "billion-dollar mistake."

We don't have to keep living with that mistake.

func findUser(email: string) -> Option<User> {
    let users = db.query("SELECT * FROM users WHERE email = ?", email)
    if users.isEmpty() {
        return None
    }
    return Some(users[0])
}

// Safe navigation like it's 2024
let city = user?.address?.city?.name ?? "Unknown"

// Compiler won't let you forget to check
match findUser("test@example.com") {
    Some(user) => sendEmail(user),
    None => println("Who dis?")
}

Can't accidentally dereference None . The type system literally won't let you compile if you forget to handle it.

How many production panics would this have prevented in your codebase? I'll wait.

4. Pattern Matching — Switch statements grew up

Go's switch is fine. But imagine if it could do this :

enum HttpResponse {
    Ok(body: string),
    NotFound,
    ServerError{code: int, message: string},
    Redirect(url: string)
}

func handleResponse(resp: HttpResponse) -> string {
    match resp {
        Ok(body) => "Success: ${body}",
        NotFound => "404: Not found",
        ServerError{code, message} => "Error ${code}: ${message}",
        Redirect(url) => "Redirecting to ${url}"
        // Forget a case? Compiler yells at you
    }
}

Exhaustiveness checking means the compiler knows every possible case. Miss one? Won't compile.

No more "oh crap, we didn't handle the timeout case" at 2 AM.

5. Sum Types — The #1 requested feature in Go history

996 upvotes on the Go proposal. That's not just popular, that's "the entire community is screaming for this."

enum State {
    Idle,
    Loading{progress: float},
    Success{data: string},
    Error{message: string}
}

func render(state: State) -> string {
    match state {
        Idle => "Ready when you are",
        Loading{progress} if progress < 50 => "Just getting started...",
        Loading{progress} => "Loading: ${progress}%",
        Success{data} => showSuccess(data),
        Error{message} => showError(message)
    }
}

A value can be one of several types. The compiler tracks which one. You can't mess it up.

This is how Rust does enums. How Swift does enums. How Kotlin does sealed classes. How TypeScript does discriminated unions.

Everyone has this except Go. Until now.

6. Lambda Functions — Choose your style (TypeScript or Rust)

Dingo supports two lambda syntax styles (configurable in dingo.toml ), giving you the conciseness of modern languages without the verbosity of Go's function literals.

TypeScript/JavaScript arrow functions (default):

// Single parameter (no parens needed)
users.filter(u => u.age > 18)
    .map(u => u.name)
    .sorted()

// Multiple parameters (parens required)
numbers.reduce((acc, x) => acc + x)

// With explicit types when needed
let parser = (s: string): int => parseInt(s)

Rust style with pipes:

// Single or multiple parameters
users.filter(|u| u.age > 18)
    .map(|u| u.name)
    .sorted()

numbers.reduce(|acc, x| acc + x)

// With explicit types when needed
let parser = |s: string| -> int { parseInt(s) }

Configuration ( dingo.toml ):

[syntax]
lambda_style = "typescript"  # or "rust"

Type inference: Dingo uses go/types to infer parameter types from context. When inference fails, just add explicit type annotations:

// ✅ Type inferred from filter signature
users.filter(u => u.age > 18)

// ❌ No context - inference fails
let standalone = x => x * 2

// ✅ Fix with explicit type
let standalone = (x: int) => x * 2       // TypeScript style
let standalone = |x: int| x * 2          // Rust style

Compare that to Go's verbose function literals:

filteredUsers := make([]User, 0)
for _, u := range users {
    if u.Age > 18 {
        filteredUsers = append(filteredUsers, u)
    }
}

names := make([]string, 0, len(filteredUsers))
for _, u := range filteredUsers {
    names = append(names, u.Name)
}

sort.Strings(names)

60-70% code reduction for simple callbacks. The business logic stands out instead of being buried in ceremony.

Why two styles?

  • TypeScript arrows : Largest developer community familiarity (JavaScript/TypeScript devs)
  • Rust pipes : Clear, explicit, familiar to Rust developers
  • Configuration-driven : Pick one per project, no confusion

Why no currying? Basic lambdas solve 95%+ of real use cases. Currying ( |x| |y| x + y ) is rare even in Rust (10-15% usage), doesn't fit Go's pragmatic culture, and adds complexity for minimal benefit. See features/lambdas.md for details.

7. Null Safety Operators — Chain nil checks like a human

The Go way:

var city string
if user != nil && user.Address != nil && user.Address.City != nil {
    city = *user.Address.City
} else {
    city = "Unknown"
}

The Dingo way:

let city = user?.address?.city?.name ?? "Unknown"

One line. Same safety. Your eyes will thank you.

What's working now:

  • ✅ Safe navigation ( ?. ) for properties: user?.name
  • ✅ Safe navigation for methods: user?.getName()
  • ✅ Method arguments: user?.process(arg1, arg2)
  • ✅ Chaining: user?.getProfile()?.email
  • ✅ Dual type support: Option AND Go pointers (*T)
  • ✅ Null coalescing ( ?? ): value ?? default
  • ✅ Chained defaults: a ?? b ?? c
  • ✅ Integration: user?.name ?? "Guest"

See docs/features/safe-navigation.md and docs/features/null-coalescing.md for complete documentation.

8. Ternary Operator — Yes, we're going there

Go rejected this. We're adding it anyway.

let max = a > b ? a : b
let status = isActive ? "online" : "offline"
println("You have ${count} item${count == 1 ? "" : "s"}")

Every. Single. Modern. Language. Has. This.

C has it. Java has it. JavaScript has it. Python has it (kinda). Swift has it. Even PHP has it.

Go's reason for not having it? "We only need one conditional construct."

Cool. You do you, Go. We'll be over here with our one-liners.


More Real-World Examples

API Handler: Before and After

Go (42 lines):

func HandleUserUpdate(w http.ResponseWriter, r *http.Request) {
    userID := r.URL.Query().Get("id")
    if userID == "" {
        http.Error(w, "missing user ID", http.StatusBadRequest)
        return
    }

    var updateReq UpdateRequest
    if err := json.NewDecoder(r.Body).Decode(&updateReq); err != nil {
        http.Error(w, "invalid JSON", http.StatusBadRequest)
        return
    }

    user, err := db.GetUser(userID)
    if err != nil {
        http.Error(w, "database error", http.StatusInternalServerError)
        return
    }
    if user == nil {
        http.Error(w, "user not found", http.StatusNotFound)
        return
    }

    if err := validateUpdate(&updateReq); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    if err := db.UpdateUser(user.ID, &updateReq); err != nil {
        http.Error(w, "update failed", http.StatusInternalServerError)
        return
    }

    updated, err := db.GetUser(user.ID)
    if err != nil {
        http.Error(w, "failed to fetch updated user", http.StatusInternalServerError)
        return
    }

    json.NewEncoder(w).Encode(updated)
}

Dingo (15 lines):

func HandleUserUpdate(w: http.ResponseWriter, r: http.Request) {
    let result = processUpdate(r)

    match result {
        Ok(user) => json.NewEncoder(w).Encode(user),
        Err(ApiError.BadRequest(msg)) => http.Error(w, msg, 400),
        Err(ApiError.NotFound(msg)) => http.Error(w, msg, 404),
        Err(ApiError.Internal(msg)) => http.Error(w, msg, 500),
    }
}

func processUpdate(r: http.Request) -> Result<User, ApiError> {
    let userID = r.URL.Query().Get("id").filter(|s| !s.isEmpty()).okOr(ApiError.BadRequest("missing user ID"))?
    let updateReq = json.NewDecoder(r.Body).Decode::<UpdateRequest>().mapErr(|_| ApiError.BadRequest("invalid JSON"))?
    let user = db.GetUser(userID)?.okOr(ApiError.NotFound("user not found"))?

    validateUpdate(updateReq)?
    db.UpdateUser(user.id, updateReq)?
    db.GetUser(user.id)
}

Data Processing Pipeline

Go (complex nested loops and error handling):

func ProcessDataBatch(items []Item) ([]ProcessedItem, error) {
    var processed []ProcessedItem

    for _, item := range items {
        if item.IsValid() {
            enriched, err := enrichItem(item)
            if err != nil {
                log.Printf("failed to enrich item %s: %v", item.ID, err)
                continue
            }

            validated, err := validateItem(enriched)
            if err != nil {
                log.Printf("validation failed for item %s: %v", item.ID, err)
                continue
            }

            transformed, err := transformItem(validated)
            if err != nil {
                return nil, fmt.Errorf("transform failed: %w", err)
            }

            processed = append(processed, transformed)
        }
    }

    if len(processed) == 0 {
        return nil, errors.New("no items processed")
    }

    return processed, nil
}

Dingo (functional pipeline):

func ProcessDataBatch(items: []Item) -> Result<[]ProcessedItem, Error> {
    let processed = items
        .filter { it.isValid() }
        .mapWithLog { enrichItem(it) }
        .mapWithLog { validateItem(it) }
        .map { transformItem(it) }
        .collect()?

    processed.isEmpty()
        ? Err(Error.new("no items processed"))
        : Ok(processed)
}

The functional style makes the data flow obvious: filter → enrich → validate → transform → collect.


How does this actually work?

Two-part system, just like TypeScript:

The Transpiler ( dingo build ) - Two-Stage Architecture

Stage 1: Preprocessor (Text-based transformations)

  • Converts Dingo-specific syntax to valid Go using regex-based pattern matching
  • TypeAnnotProcessor : Transforms param: Type param Type
  • ErrorPropProcessor : Expands x? → proper error handling code
  • EnumProcessor : Converts enum Name { Variant } → Go tagged union structs
  • KeywordProcessor : Handles other Dingo keywords
  • Output: Valid Go code (no Dingo syntax remains)

Stage 2: AST Processing (Structural transformations)

  • Uses native go/parser to parse the preprocessed Go code
  • Plugin pipeline (Discovery → Transform → Inject) modifies the AST
  • Result type plugin transforms Ok()/Err() constructors
  • Generates clean .go files and source maps using go/printer

Why this approach?

  • Preprocessors handle syntax that go/parser can't understand ( enum , : , etc.)
  • Then go/parser does the heavy lifting (no custom parser needed!)
  • Plugins add semantic transformations on valid Go AST
  • Simple, maintainable, leverages Go's own tooling

The generated Go code looks like what you'd write by hand. Not some machine-generated nightmare.

The Language Server ( dingo-lsp ) - ✅ Working

Wraps gopls (Go's language server). Intercepts LSP requests. Translates positions using source maps. Forwards to gopls.

You get autocomplete, go-to-definition, diagnostics, hover info — everything gopls does. Working in VS Code right now.

Your editor thinks it's editing Go. Your terminal thinks it's compiling Go. Only you know you're actually writing Dingo.


What the generated code looks like

Let's see what actually comes out of the transpiler.

You write this Dingo:

func findUser(email: string) -> Option<User> {
    let users = db.query("SELECT * FROM users WHERE email = ?", email)
    if users.isEmpty() {
        return None
    }
    return Some(users[0])
}

match findUser("test@example.com") {
    Some(user) => sendEmail(user),
    None => println("Not found")
}

Dingo generates this Go:

type OptionUser struct {
    value *User
    isSet bool
}

func findUser(email string) OptionUser {
    users := db.query("SELECT * FROM users WHERE email = ?", email)
    if users.isEmpty() {
        return OptionUser{isSet: false}
    }
    return OptionUser{value: &users[0], isSet: true}
}

opt := findUser("test@example.com")
if opt.isSet {
    user := *opt.value
    sendEmail(user)
} else {
    fmt.Println("Not found")
}

Clean. Readable. Exactly what you'd write if you were implementing Option types by hand in Go.

No magic. No runtime library. Just structs and if statements.


Your questions, answered

Is this ready to use right now?

Partially. We're in active development with several features already working:

Working Today:

  • Sum types with enum keyword
  • Pattern matching with match expressions
  • Error propagation with ? operator
  • Functional utilities ( map , filter , reduce , etc.)
  • Beautiful CLI tooling ( dingo build , dingo run )
  • Clean, idiomatic code generation

Coming Soon:

  • Result<T, E> and Option<T> integration
  • Lambda syntax sugar
  • Safe navigation operators
  • Language server and IDE support

Ready to experiment? Clone the repo, build the compiler, and try it out. The transpiler works, generates clean Go code, and Phase 2.7 is complete.

Ready for production? Not yet. Wait for v1.0 (10-12 months).

Want to follow along? Star the repo and watch the releases.

Will my existing Go code work with this?

Yes. 100%.

Import any Go package into Dingo. Call any Go function. Use any Go library. It all just works.

You can even mix .go and .dingo files in the same project. Migrate gradually. No big-bang rewrite required.

What about performance?

Dingo compiles to Go. Go compiles to machine code. Same machine code.

Zero runtime overhead. No wrapper library. No reflection tricks. No performance penalty.

If your Go code runs fast, your Dingo code runs at exactly the same speed. Because it is Go code.

Why not just wait for Go to add these features?

The Go team has been saying no to sum types for 15 years. They're not changing their minds.

And honestly? They're probably right for Go . These features add complexity. Go values simplicity.

But Dingo isn't Go. We're a meta-language. We can add features that transpile to simple Go underneath.

Best of both worlds.

How is this different from Borgo?

First: Borgo deserves massive respect.

Borgo ( github.com/borgo-lang/borgo ) proved this entire concept works. 4.5k stars. Real production users. They showed the world that transpiling to Go is viable.

What Borgo proved:

  • ✅ Transpiling modern syntax to Go actually works
  • ✅ Result/Option/pattern matching can run on Go's runtime
  • ✅ There's massive demand for Go with better ergonomics
  • ✅ Zero runtime overhead is achievable
  • ✅ You can have sum types without changing Go itself

Borgo taught us WHAT to build. Now Dingo is building it BETTER.

Why Dingo is taking a different path:

Aspect Borgo Dingo Why It Matters
Syntax Rust-like Go-like + enhancements Go developers don't need to learn Rust
Implementation Written in Rust Pure Go No Rust toolchain required
Architecture Monolithic transpiler Plugin-based system Extensible, community can add features
IDE Support Basic/none Full gopls proxy via LSP Autocomplete, refactoring, diagnostics
Source Maps No Yes Error messages point to .dingo files, not generated Go
Target Audience Rust devs wanting Go runtime Go devs wanting modern features Different user base
Maintenance Last commit: 2023 Active development Community support and updates
Lambda Syntax Rust closures only 4 styles (Rust/TS/Kotlin/Swift) Pick what feels natural
Error Messages Point to generated Go Point to original Dingo Debugging actually works

The honest truth:

Borgo is for Rust developers who need Go's deployment/runtime but don't want to give up Rust's syntax.

Dingo is for Go developers who love Go but are tired of typing if err != nil 47 times per file.

Different problems. Different solutions. Both valid.

What we're learning from Borgo:

  • Their transpilation patterns for sum types (brilliant)
  • How they handle pattern matching (rock solid)
  • Their approach to zero-cost abstractions (perfect)

What we're doing differently:

  • Building in Go so Go developers can contribute
  • Adding gopls integration so your IDE actually works
  • Supporting multiple lambda styles (not everyone loves Rust syntax)
  • Active maintenance and community building
  • Plugin architecture for extensibility

Use Borgo if: You're a Rust developer stuck using Go for deployment/company reasons.

Use Dingo if: You're a Go developer who wants Result types without learning Rust.

Both projects make Go better. That's a win for everyone.


Roadmap (the realistic version)

Phase 1: Core Transpiler — 8-10 weeks

  • ✅ Research complete
  • 🔨 Building sum types right now
  • 🔜 Result and Option types
  • 🔜 Pattern matching
  • 🔜 The ? operator

Goal: Ship something you can actually use to solve Go's error handling problem.

Phase 2: Ergonomics — 6-8 weeks

  • Null safety operators ( ?. , ?? )
  • Lambda functions (all styles)
  • Map/filter/reduce
  • Tuples
  • Type-safe enums

Goal: Make writing Dingo feel modern without being weird.

Phase 3: Advanced Features — 4-6 weeks

  • Immutability
  • Ternary operator
  • Pattern guards
  • Advanced destructuring

Goal: Feature parity with Swift/Kotlin for type safety.

Phase 4: Language Server — ✅ Complete (2025-11-20)

  • ✅ gopls proxy working
  • ✅ Source maps translating with high accuracy
  • ✅ VSCode IDE integration complete

Goal: Developer experience that matches or beats regular Go. ✅ Achieved

Phase 5: Polish & Ship — 4-6 weeks

  • Documentation you'll actually read
  • Examples that don't suck
  • Real-world testing
  • Community feedback

Target: v1.0 in about 12-15 months .


What Dingo adds (and what it doesn't)

We're adding:

✅ Type safety that prevents bugs at compile time ✅ Modern ergonomics that reduce boilerplate ✅ Features that are proven in Rust/Swift/Kotlin ✅ Zero-cost abstractions (transpiles to clean Go)

We're NOT adding:

❌ Runtime dependencies (it's just Go underneath) ❌ Magic behavior (no operator overloading, fight me) ❌ Ecosystem fragmentation (works with all Go packages) ❌ Unnecessary complexity (if Go's way is fine, we keep it)


Standing on the shoulders of giants

Dingo exists because these languages and projects proved it's possible:

TypeScript — The blueprint for meta-languages.

You can add type safety to an existing language without breaking the world. TypeScript didn't replace JavaScript, it enhanced it. Millions of developers use it daily. The entire pattern of "compile to a simpler language" comes from TypeScript's success.

Rust — The gold standard for type safety.

Result, Option, pattern matching, and the ? operator are genuinely brilliant. We're not reinventing this wheel—we're copying Rust's homework because they got an A+. Every language that adds these features becomes better.

Swift — Proof that nil safety works.

Optional types and safe navigation ( ?. ) made nil pointer bugs almost extinct in Swift codebases. Apple bet their entire platform on this approach. It works. We're bringing it to Go.

Kotlin — Pragmatism over purity.

Kotlin showed you can add modern features to a language ecosystem (JVM) without destroying compatibility. Null safety, sealed classes, smart casts—all transpile to regular Java bytecode. Same playbook, different target.

Borgo — The trailblazer that proved Go transpilation works.

Borgo (4.5k stars) was the first to prove you can transpile Rust-like syntax to Go successfully. They showed:

  • ✅ Result/Option types work on Go's runtime
  • ✅ Pattern matching transpiles cleanly
  • ✅ Zero runtime overhead is achievable
  • ✅ Real production users want this

Borgo validated the entire concept. Without Borgo, Dingo wouldn't exist. We're building on their shoulders, improving the architecture (gopls integration, source maps, pure Go implementation), but the core idea? That's Borgo's genius.

templ — The LSP proxy pattern.

templ showed how to wrap gopls as a language server proxy. They figured out the hard parts: bidirectional position mapping, protocol translation, maintaining gopls compatibility. We're using their playbook for Dingo's LSP.


The common thread:

Every one of these projects proved that enhancing a language WITHOUT forking it is not only possible—it's the right approach. TypeScript didn't fork JavaScript. Kotlin didn't fork Java. Borgo didn't fork Go.

Dingo won't either. We're making Go better by building on top of it.


Can I help?

Yes. Here's how:

🌟 Star the repo — Shows us people actually want this

💡 Open issues — Got ideas? Complaints? Weird edge cases? Tell us.

📖 Improve docs — If something's confusing, it's our fault. Help us fix it.

🔨 Write code — Check issues tagged "good first issue"

Before you write code, open an issue first. Let's chat about the approach. Saves everyone time.


Project structure (for the curious)

dingo/
├── cmd/dingo/                # CLI tool (dingo build, run, version)
├── pkg/
│   ├── preprocessor/         # Stage 1: Text transformations (Dingo → valid Go)
│   │   ├── preprocessor.go   # Pipeline coordinator
│   │   ├── typeannotation.go # param: Type → param Type
│   │   ├── errorprop.go      # x? → error handling
│   │   ├── enum.go           # enum → Go tagged unions
│   │   └── keyword.go        # Other Dingo keywords
│   ├── plugin/               # Stage 2: AST transformations
│   │   ├── plugin.go         # 3-phase pipeline (Discovery/Transform/Inject)
│   │   └── builtin/
│   │       └── result_type.go # Result<T,E> plugin
│   ├── generator/            # Code generation (go/printer)
│   └── config/               # Configuration
├── tests/golden/             # Golden file tests
├── features/                 # Feature proposals (read INDEX.md)
├── docs/                     # Documentation
└── examples/                 # Example code

Important files:

  • features/INDEX.md — Every planned feature with priorities and complexity
  • CLAUDE.md — Project context (yes, we use AI for help)
  • ai-docs/ — Research notes and architecture decisions

Implementation Status

Core Completed

  • Transpiler Pipeline

    • Stage 1: Preprocessor (regex-based text transforms + unique markers)
    • Stage 2: go/parser + plugin pipeline (AST transforms)
    • Stage 3: Post-AST source maps (100% accurate position mapping)
    • Clean, idiomatic code generation
    • Full IDE support via precise source maps
  • CLI Tooling

    • dingo build - Transpile files
    • dingo run - Compile and execute
    • dingo version - Version info
    • Beautiful terminal UI (lipgloss)
  • Plugin Architecture

    • Modular transformation system
    • Dependency resolution
    • Extensible design

Features Implemented

Type System (Phase 2.5-2.7)

  • Sum Types with enum
  • Pattern Matching with match
  • Pattern destructuring (struct/tuple)
  • IIFE expression wrapping
  • Type inference engine
  • Configurable nil safety (off/on/debug)

Error Handling (Phase 2.6)

  • Error Propagation ( ? operator)
  • Statement context transformation
  • Expression lifting
  • Error message wrapping
  • Multi-pass AST processing

Functional Programming (Phase 2.7)

  • map() - Transform collections
  • filter() - Predicate selection
  • reduce() - Aggregation
  • sum() , count() , all() , any()
  • Method chaining support
  • Zero-overhead IIFE transpilation

Infrastructure & Tooling (Phase V ✅)

Documentation (Complete)

  • Getting Started Guide
  • 5 Feature Guides (8,000+ lines)
  • Migration from Go Guide
  • Package Management Strategy
  • CI/CD Setup Guide

Development Tools

  • Source map validation (98.7% accuracy)
  • Workspace builds ( dingo build ./... )
  • Golden test diff visualizer
  • Performance tracking
  • GitHub Actions CI/CD

Package Management

  • Hybrid strategy (libs→.go, apps→.dingo)
  • 3 working example projects
  • Dependency resolution
  • Incremental build caching

Recently Completed

Phase 7: Null Safety Operators

  • ✅ Safe navigation ( ?. ) - properties and methods
  • ✅ Null coalescing ( ?? ) - default values
  • ✅ Dual type support (Option + *T pointers)
  • ✅ Chaining and integration

Planned Features

Operators & Syntax

  • Ternary operator ( ? : )
  • Lambda functions (4 syntax styles)

Advanced Features

  • Immutability
  • Tuples
  • Pattern guards
  • Type inference enhancements

Future Tooling

  • Language server (gopls proxy)
  • VS Code extension
  • Advanced IDE integration

Development Progress

✅ Completed (v0.1.0-alpha)

Phase Status Features Results
Phase 0 ✅ Complete Research & Architecture N/A
Phase 1 ✅ Complete Transpiler + CLI 100%
Phase 2.5 ✅ Complete Sum Types + Pattern Matching 52/52 tests passing
Phase 2.6 ✅ Complete Result/Option Foundation 3/8 golden tests
Phase 2.7 ✅ Complete Functional Utilities 8/8 tests passing
Phase 2.16 ✅ Complete Parser Fix + Result Integration 48/48 preprocessor tests
Phase 3 ✅ Complete Fix A4/A5 + Option 261/267 tests (97.8%)
Phase 4 ✅ Complete Pattern Matching Enhancements 57/57 tests passing
Phase V ✅ Complete Infrastructure & Developer Experience 3/4 external approval
Phase 6 ✅ Complete Lambda Functions (TypeScript/Rust styles) 105/105 tests, 9/9 golden tests
Phase 7 ✅ Complete Null Safety Operators ( ?. , ?? ) 37/37 tests passing

Current Capabilities: Result<T,E>, Option, sum types (enum), pattern matching (Rust/Swift syntax), error propagation (?), functional utilities (map/filter/reduce), lambda functions (TypeScript/Rust styles) , safe navigation (?.) , null coalescing (??) , exhaustiveness checking, workspace builds, source maps (98.7% accuracy)

🚧 Planned for v1.0 (Q1 2026)

Phase Priority Features Timeline Status
Phase 6 P1 Lambda Functions (2 syntax styles) 2-3 weeks ✅ Complete
Phase 7 P1 Null Safety Operators ( ?. , ?? ) 2 weeks ✅ Complete
Phase 8 P2 Tuples (single-level, single-line) 1-2 weeks ✅ Complete (Scope Reduction)*
Phase 9 P2 Ternary Operator ( ? : ) 2-3 days ✅ Complete (Implemented in Phase 6)
Phase 10 P0 Language Server (gopls proxy) 8-10 weeks ✅ Complete (2025-11-20)
Phase 11 P0 IDE Integration (VS Code, Neovim) 4-6 weeks ✅ Complete (VSCode)

v1.0 Goals: Production-ready transpiler with full IDE support, comprehensive documentation, and battle-tested core features

🔮 Post-v1.0 Roadmap

Phase Priority Features Timeline Community Demand
v1.1 P2 Immutability (const tracking) 4+ weeks ⭐⭐⭐
v1.1 P3 Default Parameters 2 weeks ⭐⭐
v1.2 P4 Function Overloading 3 weeks ⭐⭐ (specialized use)
v1.2 P4 Operator Overloading 2 weeks ⭐⭐ (math/DSL users)
Future P5 Async/await sugar TBD ⭐⭐⭐
Future P5 Macros/metaprogramming TBD ⭐⭐

Legend: 🔴 Not Started | 🟡 In Progress | 🟢 In Development | ✅ Complete

Phase V Deliverables (2025-11-19):

  • 📚 12 comprehensive documentation guides (8,000+ lines)
  • 📦 Package management strategy + 3 example projects
  • 🛠️ Workspace builds with dependency resolution
  • ✅ Source map validation (98.7% accuracy)
  • 🚀 CI/CD enhancements (GitHub Actions, diff visualizer, performance tracking)
  • ⭐ External validation: Grok 4 Fast (9/10), Gemini 3 Pro (8.8/10), Claude Opus 4 (9.5/10)

Current Status: Infrastructure ready for v1.0. Core language features complete. Targeting v1.0 release Q1 2026.


One more thing

Go is an amazing language. This isn't a "Go sucks" project.

But languages evolve. JavaScript got classes. C++ got lambdas. Even Python added type hints.

Go's evolving too — they added generics in 1.18.

Dingo is just evolution they won't add themselves. Sum types. Pattern matching. Result types. The stuff the community keeps asking for.

We're not changing Go. We're building on top of it.

And maybe, just maybe, making your codebase a little bit nicer to work with.


Get Started Today

Try Dingo in 3 Steps

# 1. Clone and build
git clone https://github.com/MadAppGang/dingo.git && cd dingo && go build -o dingo ./cmd/dingo

# 2. Create hello.dingo
echo 'package main
enum Result { Ok, Error }
func main() { println("Dingo works!") }' > hello.dingo

# 3. Run it
./dingo run hello.dingo

View Examples Read Features Check Roadmap Star on GitHub


Join the Community

We're building Dingo in public. Here's how to get involved:

Star the Repo

Show your support and stay updated

GitHub

Try It Out

Clone, build, experiment

Quick Start

Report Issues

Found a bug? Have an idea?

Open an Issue

Contribute Code

Check out good first issues

Contributing


One Final Thing

Go is an amazing language. This isn't a "Go sucks" project.

But languages evolve. JavaScript got classes. C++ got lambdas. Even Python added type hints.

Go's evolving too — they added generics in 1.18.

Dingo is just evolution they won't add themselves. Sum types. Pattern matching. Result types. The stuff the community keeps asking for.

We're not changing Go. We're building on top of it.

And maybe, just maybe, making your codebase a little bit nicer to work with.


Built by developers who love Go but also love sum types.

Dingo: Because you can love Go and still want better error handling.


License: TBD (probably MIT or Apache 2.0) • Version: 0.1.0-alpha • Status: Active Development

Website GitHub Issues Changelog Features

Not ready for production • Star to follow progress • v1.0 target: Late 2025

Experimenting with Robin Hood hashing

Lobsters
twdev.blog
2025-11-23 01:19:49
Comments...
Original Article

I’ve recently discovered a paper 1 about Robin Hood hashing and decided to perform some simple experiments and check how a trivial, custom implementation of a hash table employing this algorithm stacks against unordered_map .

In general, collisions in hash tables are handled using either linear probing (meaning that the colliding element will be inserted in next available slot in the table if the one that it’s hashed to is already occupied) or element chaining, in which case, the elements with the same hash are just chained together into a linked list.

Robin Hood hashing is a technique applied to implementations using linear probing that aims to minimise the probing sequence length to optimise lookups. So, based on that, it’s safe to assume it’s optimising read-heavy applications.

In hash tables employing linear probing, psl describe the distance between elements hashed index and its actual index. These two can be far away if the hash table is heavily populated and there are a lot of elements hashed to the same index.

I’ve copied this pseudo-code from the original paper verbatim. vpsl is a bit a strange name for a variable but in this case, it just means psl of v , the latter describing the variable name for the element being inserted. In short, the while loop, runs until we find a free slot in the table. If there’s no collisions, then it doesn’t execute at all. The psl of v is zero and the element is inserted straight away into its designated hashed index. Therefore, psl == 0 indicates that a given element is inserted with no collisions.

In case the hashed index of v is already occupied, the loop will linearly look for next free slot in the table increasing the vpsl with each iteration.

Which swaps the current value of v with the one at current index in order to equalise the overall elements psl in the table.

I’ve come up with this quick implementation sketch just to get some benchmark numbers:

The insert algorithm is re-implemented just like in the paper. There are no special quirks or features added on top of it.

With the help of LLMs, I’ve generated the following benchmarks which compare the performance of my implementation against unordered_map .

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
std::vector<int> generateTestData(size_t count, int seed = 42) {
  std::mt19937 gen(seed);
  std::uniform_int_distribution<int> dis(0, 1000000);

  std::vector<int> data;
  data.reserve(count);
  for (size_t i = 0; i < count; ++i) {
    data.push_back(dis(gen));
  }
  return data;
}

template <size_t Size>
static void BM_RobinHood_Insert(benchmark::State &state) {
  auto data = generateTestData(Size);

  for (auto _ : state) {
    RHHT<int, Size> ht;
    for (const auto &val : data) {
      benchmark::DoNotOptimize(ht.insert(val));
    }
    benchmark::ClobberMemory();
  }

  state.SetItemsProcessed(state.iterations() * Size);
}

static void BM_UnorderedMap_Insert(benchmark::State &state) {
  size_t size = state.range(0);
  auto data = generateTestData(size);

  for (auto _ : state) {
    std::unordered_map<int, int> map;
    for (const auto &val : data) {
      benchmark::DoNotOptimize(map.insert({val, val}));
    }
    benchmark::ClobberMemory();
  }

  state.SetItemsProcessed(state.iterations() * size);
}

template <size_t Size, int LoadFactorPercent>
static void BM_RobinHood_Lookup_Sequential(benchmark::State &state) {
  size_t numElements = (Size * LoadFactorPercent) / 100;
  auto data = generateTestData(numElements);

  // Pre-populate
  RHHT<int, Size> ht;
  for (const auto &val : data) {
    ht.insert(val);
  }

  size_t idx = 0;
  for (auto _ : state) {
    bool found = ht.has(data[idx % data.size()]);
    benchmark::DoNotOptimize(found);
    idx++;
  }

  state.SetItemsProcessed(state.iterations());
}

template <size_t Size, int LoadFactorPercent>
static void BM_UnorderedMap_Lookup_Sequential(benchmark::State &state) {
  size_t numElements = (Size * LoadFactorPercent) / 100;
  auto data = generateTestData(numElements);

  // Pre-populate
  std::unordered_map<int, int> map;
  for (const auto &val : data) {
    map[val] = val;
  }

  size_t idx = 0;
  for (auto _ : state) {
    bool found = map.find(data[idx % data.size()]) != map.end();
    benchmark::DoNotOptimize(found);
    idx++;
  }

  state.SetItemsProcessed(state.iterations());
}

// Random lookup pattern (more realistic)
template <size_t Size, int LoadFactorPercent>
static void BM_RobinHood_Lookup_Random(benchmark::State &state) {
  size_t numElements = (Size * LoadFactorPercent) / 100;
  auto data = generateTestData(numElements);
  auto lookupKeys =
      generateTestData(numElements, 123); // Different seed for lookups

  // Pre-populate
  RHHT<int, Size> ht;
  for (const auto &val : data) {
    ht.insert(val);
  }

  size_t idx = 0;
  for (auto _ : state) {
    bool found = ht.has(lookupKeys[idx % lookupKeys.size()]);
    benchmark::DoNotOptimize(found);
    idx++;
  }

  state.SetItemsProcessed(state.iterations());
}

template <size_t Size, int LoadFactorPercent>
static void BM_UnorderedMap_Lookup_Random(benchmark::State &state) {
  size_t numElements = (Size * LoadFactorPercent) / 100;
  auto data = generateTestData(numElements);
  auto lookupKeys = generateTestData(numElements, 123);

  // Pre-populate
  std::unordered_map<int, int> map;
  for (const auto &val : data) {
    map[val] = val;
  }

  size_t idx = 0;
  for (auto _ : state) {
    bool found = map.find(lookupKeys[idx % lookupKeys.size()]) != map.end();
    benchmark::DoNotOptimize(found);
    idx++;
  }

  state.SetItemsProcessed(state.iterations());
}

// ============================================================================
// MIXED WORKLOAD (90% reads, 10% writes)
// ============================================================================

template <size_t Size>
static void BM_RobinHood_Mixed_90Read(benchmark::State &state) {
  size_t numElements = (Size * 80) / 100;
  auto data = generateTestData(numElements);
  auto lookupKeys = generateTestData(numElements, 123);

  for (auto _ : state) {
    state.PauseTiming();
    RHHT<int, Size> ht;
    for (const auto &val : data) {
      ht.insert(val);
    }
    state.ResumeTiming();

    for (size_t i = 0; i < 1000; ++i) {
      if (i % 10 == 0) {
        // 10% writes
        benchmark::DoNotOptimize(ht.insert(lookupKeys[i % lookupKeys.size()]));
      } else {
        // 90% reads
        benchmark::DoNotOptimize(ht.has(lookupKeys[i % lookupKeys.size()]));
      }
    }
  }

  state.SetItemsProcessed(state.iterations() * 1000);
}

static void BM_UnorderedMap_Mixed_90Read(benchmark::State &state) {
  size_t size = state.range(0);
  size_t numElements = (size * 80) / 100;
  auto data = generateTestData(numElements);
  auto lookupKeys = generateTestData(numElements, 123);

  for (auto _ : state) {
    state.PauseTiming();
    std::unordered_map<int, int> map;
    for (const auto &val : data) {
      map[val] = val;
    }
    state.ResumeTiming();

    for (size_t i = 0; i < 1000; ++i) {
      if (i % 10 == 0) {
        benchmark::DoNotOptimize(map[lookupKeys[i % lookupKeys.size()]] = i);
      } else {
        benchmark::DoNotOptimize(map.find(lookupKeys[i % lookupKeys.size()]) !=
                                 map.end());
      }
    }
  }

  state.SetItemsProcessed(state.iterations() * 1000);
}

// ============================================================================
// CACHE BEHAVIOR TEST (Many lookups on same small set)
// ============================================================================

template <size_t Size>
static void BM_RobinHood_HotCache(benchmark::State &state) {
  size_t numElements = (Size * 80) / 100;
  auto data = generateTestData(numElements);

  RHHT<int, Size> ht;
  for (const auto &val : data) {
    ht.insert(val);
  }

  // Hot set: only 100 keys accessed repeatedly
  std::vector<int> hotKeys(data.begin(),
                           data.begin() + std::min(size_t(100), data.size()));

  size_t idx = 0;
  for (auto _ : state) {
    bool found = ht.has(hotKeys[idx % hotKeys.size()]);
    benchmark::DoNotOptimize(found);
    idx++;
  }

  state.SetItemsProcessed(state.iterations());
}

template <size_t Size>
static void BM_UnorderedMap_HotCache(benchmark::State &state) {
  size_t numElements = (Size * 80) / 100;
  auto data = generateTestData(numElements);

  std::unordered_map<int, int> map;
  for (const auto &val : data) {
    map[val] = val;
  }

  std::vector<int> hotKeys(data.begin(),
                           data.begin() + std::min(size_t(100), data.size()));

  size_t idx = 0;
  for (auto _ : state) {
    bool found = map.find(hotKeys[idx % hotKeys.size()]) != map.end();
    benchmark::DoNotOptimize(found);
    idx++;
  }

  state.SetItemsProcessed(state.iterations());
}

// ============================================================================
// REGISTER BENCHMARKS
// ============================================================================

// Small size (1K elements)
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Sequential, 1024, 75);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Sequential, 1024, 75);
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Sequential, 1024, 90);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Sequential, 1024, 90);

BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Random, 1024, 75);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Random, 1024, 75);
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Random, 1024, 90);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Random, 1024, 90);

// Medium size (10K elements)
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Sequential, 10240, 75);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Sequential, 10240, 75);
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Sequential, 10240, 90);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Sequential, 10240, 90);

BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Random, 10240, 75);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Random, 10240, 75);
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Random, 10240, 90);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Random, 10240, 90);

// Large size (100K elements)
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Sequential, 102400, 75);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Sequential, 102400, 75);
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Sequential, 102400, 90);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Sequential, 102400, 90);

BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Random, 102400, 75);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Random, 102400, 75);
BENCHMARK_TEMPLATE(BM_RobinHood_Lookup_Random, 102400, 90);
BENCHMARK_TEMPLATE(BM_UnorderedMap_Lookup_Random, 102400, 90);

// Insertion benchmarks
BENCHMARK_TEMPLATE(BM_RobinHood_Insert, 1024);
BENCHMARK(BM_UnorderedMap_Insert)->Arg(1024);
BENCHMARK_TEMPLATE(BM_RobinHood_Insert, 10240);
BENCHMARK(BM_UnorderedMap_Insert)->Arg(10240);
BENCHMARK_TEMPLATE(BM_RobinHood_Insert, 102400);
BENCHMARK(BM_UnorderedMap_Insert)->Arg(102400);

// Mixed workload
BENCHMARK_TEMPLATE(BM_RobinHood_Mixed_90Read, 10240);
BENCHMARK(BM_UnorderedMap_Mixed_90Read)->Arg(10240);

// Hot cache
BENCHMARK_TEMPLATE(BM_RobinHood_HotCache, 10240);
BENCHMARK_TEMPLATE(BM_UnorderedMap_HotCache, 10240);
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
2025-11-16T18:06:17+00:00
Running ./bld/rh
Run on (16 X 1624.14 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1280 KiB (x8)
  L3 Unified 18432 KiB (x1)
Load Average: 0.65, 0.85, 0.68
--------------------------------------------------------------------------------------------------------
Benchmark                                              Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------------------------------
BM_RobinHood_Lookup_Sequential<1024, 75>            23.2 ns         23.2 ns     29431364 items_per_second=43.0989M/s
BM_UnorderedMap_Lookup_Sequential<1024, 75>         47.7 ns         47.7 ns     14715163 items_per_second=20.9762M/s
BM_RobinHood_Lookup_Sequential<1024, 90>            30.1 ns         30.0 ns     22890742 items_per_second=33.301M/s
BM_UnorderedMap_Lookup_Sequential<1024, 90>         49.0 ns         48.9 ns     14332389 items_per_second=20.4436M/s
BM_RobinHood_Lookup_Random<1024, 75>                23.3 ns         23.2 ns     30824388 items_per_second=43.0181M/s
BM_UnorderedMap_Lookup_Random<1024, 75>             50.0 ns         49.9 ns     14074009 items_per_second=20.0314M/s
BM_RobinHood_Lookup_Random<1024, 90>                30.8 ns         30.8 ns     22487564 items_per_second=32.5128M/s
BM_UnorderedMap_Lookup_Random<1024, 90>             53.3 ns         53.2 ns     13227582 items_per_second=18.7909M/s
BM_RobinHood_Lookup_Sequential<10240, 75>           15.1 ns         15.1 ns     46823053 items_per_second=66.3617M/s
BM_UnorderedMap_Lookup_Sequential<10240, 75>        51.5 ns         51.5 ns     13923980 items_per_second=19.4305M/s
BM_RobinHood_Lookup_Sequential<10240, 90>           19.4 ns         19.4 ns     37017310 items_per_second=51.5001M/s
BM_UnorderedMap_Lookup_Sequential<10240, 90>        54.2 ns         54.2 ns     12995774 items_per_second=18.4631M/s
BM_RobinHood_Lookup_Random<10240, 75>               14.1 ns         14.1 ns     49210793 items_per_second=71.0811M/s
BM_UnorderedMap_Lookup_Random<10240, 75>            56.7 ns         56.7 ns     12466727 items_per_second=17.6515M/s
BM_RobinHood_Lookup_Random<10240, 90>               19.0 ns         18.9 ns     36736116 items_per_second=52.8209M/s
BM_UnorderedMap_Lookup_Random<10240, 90>            59.8 ns         59.8 ns     11552573 items_per_second=16.734M/s
BM_RobinHood_Lookup_Sequential<102400, 75>          24.5 ns         24.5 ns     28520988 items_per_second=40.7975M/s
BM_UnorderedMap_Lookup_Sequential<102400, 75>       61.6 ns         61.5 ns     11358184 items_per_second=16.2709M/s
BM_RobinHood_Lookup_Sequential<102400, 90>          33.4 ns         33.3 ns     21381782 items_per_second=30.0186M/s
BM_UnorderedMap_Lookup_Sequential<102400, 90>       56.5 ns         56.4 ns     12453230 items_per_second=17.7316M/s
BM_RobinHood_Lookup_Random<102400, 75>              23.1 ns         23.0 ns     28817244 items_per_second=43.4076M/s
BM_UnorderedMap_Lookup_Random<102400, 75>           67.5 ns         67.3 ns     10439771 items_per_second=14.8492M/s
BM_RobinHood_Lookup_Random<102400, 90>              33.2 ns         33.2 ns     21677768 items_per_second=30.1504M/s
BM_UnorderedMap_Lookup_Random<102400, 90>           57.8 ns         57.7 ns     12323824 items_per_second=17.3335M/s
BM_RobinHood_Insert<1024>                          95319 ns        95212 ns         7270 items_per_second=10.755M/s
BM_UnorderedMap_Insert/1024                       112268 ns       112161 ns         6246 items_per_second=9.12976M/s
BM_RobinHood_Insert<10240>                       4249993 ns      4245093 ns          164 items_per_second=2.4122M/s
BM_UnorderedMap_Insert/10240                     1162218 ns      1161018 ns          604 items_per_second=8.81984M/s
BM_RobinHood_Insert<102400>                    826079369 ns    824980437 ns            1 items_per_second=124.124k/s
BM_UnorderedMap_Insert/102400                   15235647 ns     15212770 ns           46 items_per_second=6.73119M/s
BM_RobinHood_Mixed_90Read<10240>                   20910 ns        20882 ns        33439 items_per_second=47.889M/s
BM_UnorderedMap_Mixed_90Read/10240                239302 ns       239000 ns         2929 items_per_second=4.1841M/s
BM_RobinHood_HotCache<10240>                        9.40 ns         9.39 ns     73922140 items_per_second=106.523M/s
BM_UnorderedMap_HotCache<10240>                     49.3 ns         49.3 ns     14228030 items_per_second=20.2833M/s

Clearly, there’s at least a 2x advantage using RobinHood hashing vs default unordered_map . This is perhaps better visible on the graphs:

The access times advantages quickly diminishes when switching to actual release build ( -O3 ):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
2025-11-16T18:04:25+00:00
Running ./bld/rh
Run on (16 X 400 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x8)
  L1 Instruction 32 KiB (x8)
  L2 Unified 1280 KiB (x8)
  L3 Unified 18432 KiB (x1)
Load Average: 2.59, 1.05, 0.70
--------------------------------------------------------------------------------------------------------
Benchmark                                              Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------------------------------
BM_RobinHood_Lookup_Sequential<1024, 75>            5.40 ns         5.39 ns    133449597 items_per_second=185.489M/s
BM_UnorderedMap_Lookup_Sequential<1024, 75>         5.27 ns         5.27 ns    132876156 items_per_second=189.839M/s
BM_RobinHood_Lookup_Sequential<1024, 90>            8.55 ns         8.54 ns     83301857 items_per_second=117.098M/s
BM_UnorderedMap_Lookup_Sequential<1024, 90>         5.39 ns         5.39 ns    129537377 items_per_second=185.468M/s
BM_RobinHood_Lookup_Random<1024, 75>                5.89 ns         5.88 ns    119774512 items_per_second=170.068M/s
BM_UnorderedMap_Lookup_Random<1024, 75>             5.90 ns         5.90 ns    118744624 items_per_second=169.57M/s
BM_RobinHood_Lookup_Random<1024, 90>                8.85 ns         8.85 ns     80458434 items_per_second=113.045M/s
BM_UnorderedMap_Lookup_Random<1024, 90>             6.27 ns         6.26 ns    111711716 items_per_second=159.653M/s
BM_RobinHood_Lookup_Sequential<10240, 75>           4.83 ns         4.82 ns    144810114 items_per_second=207.391M/s
BM_UnorderedMap_Lookup_Sequential<10240, 75>        5.48 ns         5.48 ns    126778145 items_per_second=182.631M/s
BM_RobinHood_Lookup_Sequential<10240, 90>           7.54 ns         7.53 ns     93302180 items_per_second=132.745M/s
BM_UnorderedMap_Lookup_Sequential<10240, 90>        7.45 ns         7.44 ns     93659417 items_per_second=134.361M/s
BM_RobinHood_Lookup_Random<10240, 75>               4.39 ns         4.39 ns    159898091 items_per_second=227.92M/s
BM_UnorderedMap_Lookup_Random<10240, 75>            14.4 ns         14.4 ns     49152677 items_per_second=69.4271M/s
BM_RobinHood_Lookup_Random<10240, 90>               6.32 ns         6.31 ns    111408154 items_per_second=158.355M/s
BM_UnorderedMap_Lookup_Random<10240, 90>            15.9 ns         15.9 ns     43591477 items_per_second=62.957M/s
BM_RobinHood_Lookup_Sequential<102400, 75>          15.5 ns         15.5 ns     45106080 items_per_second=64.451M/s
BM_UnorderedMap_Lookup_Sequential<102400, 75>       14.5 ns         14.5 ns     48171299 items_per_second=69.1551M/s
BM_RobinHood_Lookup_Sequential<102400, 90>          18.5 ns         18.5 ns     37614565 items_per_second=54.039M/s
BM_UnorderedMap_Lookup_Sequential<102400, 90>       10.9 ns         10.9 ns     63764135 items_per_second=91.5571M/s
BM_RobinHood_Lookup_Random<102400, 75>              14.8 ns         14.8 ns     47009683 items_per_second=67.4697M/s
BM_UnorderedMap_Lookup_Random<102400, 75>           24.1 ns         24.0 ns     29192357 items_per_second=41.6372M/s
BM_RobinHood_Lookup_Random<102400, 90>              18.3 ns         18.3 ns     38073755 items_per_second=54.6051M/s
BM_UnorderedMap_Lookup_Random<102400, 90>           18.6 ns         18.6 ns     37694527 items_per_second=53.7428M/s
BM_RobinHood_Insert<1024>                            236 ns          236 ns      2962331 items_per_second=4.33331G/s
BM_UnorderedMap_Insert/1024                        30669 ns        30636 ns        22923 items_per_second=33.4252M/s
BM_RobinHood_Insert<10240>                          2292 ns         2291 ns       305683 items_per_second=4.47005G/s
BM_UnorderedMap_Insert/10240                      458507 ns       457909 ns         1533 items_per_second=22.3625M/s
BM_RobinHood_Insert<102400>                        22836 ns        22828 ns        30668 items_per_second=4.48567G/s
BM_UnorderedMap_Insert/102400                    7463094 ns      7448520 ns           95 items_per_second=13.7477M/s
BM_RobinHood_Mixed_90Read<10240>                    9574 ns         9551 ns        73202 items_per_second=104.697M/s
BM_UnorderedMap_Mixed_90Read/10240                123672 ns       123521 ns         5690 items_per_second=8.09582M/s
BM_RobinHood_HotCache<10240>                        2.35 ns         2.34 ns    298672432 items_per_second=426.647M/s
BM_UnorderedMap_HotCache<10240>                     5.42 ns         5.41 ns    129175740 items_per_second=184.684M/s

Still though, there’s a very noticeable performance advantage when performing random lookups which for sure is influenced by the CPU cache and data locality in case of my implementation. Which makes this approach viable when used as some sort of lookup cache.

Meta buried 'causal' evidence of social media harm, US court filings allege

Hacker News
www.reuters.com
2025-11-23 01:09:47
Comments...
Original Article

Please enable JS and disable any ad blocker

Germany to classify date rape drugs as weapons to ensure justice for survivors

Hacker News
www.theguardian.com
2025-11-23 01:00:04
Comments...
Original Article

Germany plans to treat the use of date rape drugs like the use of weapons in prosecutions as part of measures to ensure justice for survivors of domestic violence and sexual assault.

“We classify date rape drugs, which are increasingly used as a widespread tool in crimes, as weapons. This creates the basis for significantly stricter prosecutions,” Alexander Dobrindt, the interior minister, said on Friday. “We are committed to clear consequences and consistent enforcement. Women should feel safe and be able to move freely everywhere.”

Nearly 54,000 women and girls were the victims of sexual offences in Germany in 2024 – an increase of 2.1% on the previous year – of which nearly 36% were victims of rape and sexual assault.

A parliamentary debate over a bill calling for a mandatory minimum five-year prison sentence for the use of date rape drugs in sexual assaults was postponed last month.

Police last year registered a record high of almost 266,000 domestic violence victims, a 3.8% increase on the previous year. However, there were a high number of unreported cases, the head of the federal criminal police office, Holger Münch, said.

“We must work to ensure that more victims find the courage to report crimes in order to improve the protection and support for victims,” he added.

Although there is no uniform federal definition for femicide in Germany, nearly 1,200 women and girls were recorded as victims of homicide or attempted homicide in 2024, a slight decrease year on year.

The German government this week approved a draft law allowing the use of electronic ankle monitors to track perpetrators of domestic violence. Victims can also request a separate device to warn them when the perpetrator is nearby.

Dobrindt said the government had also secured funding for the national rollout of an app allowing domestic abuse victims to secretly document abuse incidents to be used later in court.

A Year Without Caffeine

Hacker News
bryanalexander.org
2025-11-23 00:50:49
Comments...
Original Article

(This is the first of a two-part series.  The second part can be found here .)

I used to drink more caffeine than you do.

Me drinking coffee.  Photo by Cogdog.

Note the bared teeth, the delighted eyes, and the paw wrapped protectively wrapped around that frail cup.

That is almost certainly true.  From my college days (1985ff) through the end of 2011, my caffeine consumption was extravagant.

Epic.  Other people speak of the number of cups of coffee they drink per day; they are pikers.  For me, the number was pots .  Two, three, or four pots between grimly waking up and falling asleep.  From the first thing in the morning through multiple after-dinner imbibings, the blessed black bean brew was my brain’s constant companion.

Jolt Cola.

Ah, Jolt.

Along with coffee, soda was my parallel caffeine delivery system.  I still recall the glorious days of Jolt Cola (more sugar, and twice the caffeine!), two-liters of which saw me through my sophomore and junior years.  Coke was too basic for me, but doable when nothing else was available.  Mello Yellow was fine, but hard to obtain.  Although it had a splendidly lying name: lots of caffeine, so nothing mellow; green, not yellow color.

Mountain Dew was my drink of choice, sweet and fiercely caffeinated.  One year my housemates and I purchased enough Mountain Dew cans in bulk to make a six foot tall stack.  It nearly replaced water for us. I was quaffing a can with breakfast, bottles during the day, cups in the evening, plus a final can in bed, just to relax.

Gunpower tea.

Gunpowder tea, preferably.

Other caffeine mechanisms also supplied my needs.  Chocolate, especially chocolate-covered espresso beans, helped.  Black tea sometimes sufficed when I was among Brits, or just wanted the taste.  Hot chocolate was fine in winter.  But Turkish coffee, ah, that was the sublime caffeine delivery system.  I fell in love with the potent stuff in Bosnia during the 1990s war, and sought it out ever afterwards.  I visited an academic in Mostar whose house had taken a hit from a shell or missile.  In its ruins, on a half-shattered gas-powered stove, the prof and his wife brewed Turkish coffee every day.  I recognized my fellows, members of the worldwide society of caffeine devotees.  That concentrated bolt of coffee was like neutronium, or anti-Kryptonite for Superman, an outrageously heavy distillate for my gleeful brain.

Turkish coffee

The ur-coffee.

I could also combine caffeination systems.  During a long drive I’d load up with Mountain Dew and a giant cup of coffee.  After a couple of hours I’d stop to replenish those sources, buy some Water Joe , then add a couple of doses of Stok to the steaming coffee.  (At home my wife forbid me from brewing coffee with Water Joe, lest my chest simply explode)

Once a chemist friend gave me a small container of pure caffeine.  She warned me not to just snarf the white power straight down, so I took to dabbing a finger in it.

That was peppy.

Too Much Coffee Man, my ideal superhero.

Too Much Coffee Man, my ideal superhero.

Why did I drink so much caffeine?  it wasn’t simply chemical or behavioral addiction.  My habit began in college as a way of providing enough energy to do both my studies and jobs.

I took heavy courseloads (double or triple majoring), while working.  After graduation that overload of work never went away.  I did my M.A. in a single year, while working at a bookstore.  For my PhD I was teaching nearly the entire time.  As a professor I taught four (4) classes per term, while conducting research, plus doing lots of service (committees, technology, advising, etc), plus consulting on the side.  I also married, and we had two children.  With such long days (and nights), the caffeine was essential.

After a while caffeine no longer provided stimulus.  Instead it became a way of recovering some basic energy level from a pit of exhaustion.  A strong dose stopped making me sparky and manic, as far as I can tell, but powered me up just enough to get things done over the course of a very long, but well fueled, day.  I suppose this is another way of saying “maintenance level”.

I would have continued along this glorious, bean-strewn path until my body failed, and it nearly did.  Throughout 2010 and 2011 I suffered frequent bouts of gut pain.  This wasn’t indigestion, but fiery shocks, enough to wake me up at night or knock me off track during the day.  The pains increased in frequency, duration, and intensity, ultimately coming several times a day, and leading to regular nausea.  Besides being painful and disgusting, these attacks were debilitating.  I took to chewing antacids many times a day. Ultimately I decided to seek medical advice.  Well, “I decided” really means “I gave into my wife’s patient, well-informed concern”.

On December 22, 2011, we arrived at the family clinic we saw for most medical questions.  I described my symptoms to the doctor, who looked concerned.  He asked me to describe my caffeine intake, and his facial expressions were quite entertaining.  He demonstrated incredulity, dismay, outrage, amusement, followed by iron determination.  When I finished, the doc laid it out for me.

“Either I hospitalize you tomorrow, or you go cold turkey on caffeine.  Immediately.”

( onward to part 2 )

(photos by Cogdog , 7 Bits of Truth , Akuppa , Nate Steiner , Wikipedia )

Liked it? Take a second to support Bryan Alexander on Patreon!

Become a patron at Patreon!

Agent design is still hard

Simon Willison
simonwillison.net
2025-11-23 00:49:39
Agent design is still hard Armin Ronacher presents a cornucopia of lessons learned from building agents over the past few months. There are several agent abstraction libraries available now (my own LLM library is edging into that territory with its tools feature) but Armin has found that the abstrac...
Original Article

Agent design is still hard ( via ) Armin Ronacher presents a cornucopia of lessons learned from building agents over the past few months.

There are several agent abstraction libraries available now (my own LLM library is edging into that territory with its tools feature ) but Armin has found that the abstractions are not worth adopting yet:

[…] the differences between models are significant enough that you will need to build your own agent abstraction. We have not found any of the solutions from these SDKs that build the right abstraction for an agent. I think this is partly because, despite the basic agent design being just a loop, there are subtle differences based on the tools you provide. These differences affect how easy or hard it is to find the right abstraction (cache control, different requirements for reinforcement, tool prompts, provider-side tools, etc.). Because the right abstraction is not yet clear, using the original SDKs from the dedicated platforms keeps you fully in control. […]

This might change, but right now we would probably not use an abstraction when building an agent, at least until things have settled down a bit. The benefits do not yet outweigh the costs for us.

Armin introduces the new-to-me term reinforcement , where you remind the agent of things as it goes along:

Every time the agent runs a tool you have the opportunity to not just return data that the tool produces, but also to feed more information back into the loop. For instance, you can remind the agent about the overall objective and the status of individual tasks. […] Another use of reinforcement is to inform the system about state changes that happened in the background.

Claude Code’s TODO list is another example of this pattern in action.

Testing and evals remains the single hardest problem in AI engineering:

We find testing and evals to be the hardest problem here. This is not entirely surprising, but the agentic nature makes it even harder. Unlike prompts, you cannot just do the evals in some external system because there’s too much you need to feed into it. This means you want to do evals based on observability data or instrumenting your actual test runs. So far none of the solutions we have tried have convinced us that they found the right approach here.

Armin also has a follow-up post, LLM APIs are a Synchronization Problem , which argues that the shape of current APIs hides too many details from us as developers, and the core challenge here is in synchronizing state between the tokens fed through the GPUs and our client applications - something that may benefit from alternative approaches developed by the local-first movement.

A monopoly ISP refuses to fix upstream infrastructure

Hacker News
sacbear.com
2025-11-23 00:46:53
Comments...
Original Article

A documented case of infrastructure failure, failed escalation, and a company that refuses to investigate.

Here’s the situation: I have outages. My neighbor has the same outages. Xfinity won’t fix it.

I bought Xfinity internet in June 2024. Immediately, my connection started dropping. Multiple times a day. Every single day. After troubleshooting every piece of equipment I had and questioning my sanity my neighbor complained about the same thing which led me to understand this was not my equipment.

I set up an uptime monitor and found that these outages happen 6-7 times per day for 125 seconds.

Over 17 months of my service term that’s approximately 3,387 outage incidents totaling 117+ hours of cumulative downtime.

This outage pattern has recurred thousands of times. It is consistent, predictable, and it follows an automated schedule.

My neighbor has the same problem. Different house. Different line from a different junction box. Same 125-second outages happening at the same times of day.

PING Uptime Log
2025-11-21T14:42:31-08:00 Warning dpinger exiting on signal 15
2025-11-21T07:01:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 21.4 ms RTTd: 14.8 ms Loss: 10.0 %)
2025-11-21T07:01:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 21.7 ms RTTd: 15.8 ms Loss: 20.0 %)
2025-11-21T06:59:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.1 ms RTTd: 1.7 ms Loss: 21.0 %)
2025-11-21T06:59:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.1 ms RTTd: 1.6 ms Loss: 12.0 %)
2025-11-21T00:01:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.5 ms RTTd: 2.7 ms Loss: 10.0 %)
2025-11-21T00:01:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.6 ms RTTd: 2.8 ms Loss: 20.0 %)
2025-11-20T23:59:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 20.1 ms RTTd: 3.1 ms Loss: 21.0 %)
2025-11-20T23:59:30-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 20.0 ms RTTd: 3.0 ms Loss: 12.0 %)
2025-11-20T15:01:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 21.3 ms RTTd: 3.9 ms Loss: 10.0 %)
2025-11-20T15:01:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 21.4 ms RTTd: 4.1 ms Loss: 20.0 %)
2025-11-20T14:59:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 21.0 ms RTTd: 3.9 ms Loss: 21.0 %)
2025-11-20T14:59:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 21.3 ms RTTd: 4.3 ms Loss: 12.0 %)
2025-11-20T13:46:40-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.4 ms RTTd: 2.9 ms Loss: 10.0 %)
2025-11-20T13:46:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.2 ms RTTd: 3.0 ms Loss: 20.0 %)
2025-11-20T13:44:38-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.9 ms RTTd: 1.7 ms Loss: 21.0 %)
2025-11-20T13:44:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.7 ms RTTd: 1.7 ms Loss: 12.0 %)
2025-11-20T12:46:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.3 ms RTTd: 1.2 ms Loss: 10.0 %)
2025-11-20T12:46:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.2 ms RTTd: 1.2 ms Loss: 20.0 %)
2025-11-20T12:44:39-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.2 ms RTTd: 0.9 ms Loss: 21.0 %)
2025-11-20T12:44:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.3 ms RTTd: 1.0 ms Loss: 12.0 %)
2025-11-20T05:16:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.5 ms RTTd: 1.4 ms Loss: 10.0 %)
2025-11-20T05:16:27-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.5 ms RTTd: 1.4 ms Loss: 20.0 %)
2025-11-20T05:14:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.4 ms RTTd: 1.2 ms Loss: 21.0 %)
2025-11-20T05:14:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.3 ms RTTd: 1.2 ms Loss: 12.0 %)
2025-11-20T03:31:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 18.9 ms RTTd: 1.3 ms Loss: 10.0 %)
2025-11-20T03:31:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 18.8 ms RTTd: 1.1 ms Loss: 20.0 %)
2025-11-20T03:29:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.3 ms RTTd: 1.6 ms Loss: 21.0 %)
2025-11-20T03:29:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.4 ms RTTd: 1.6 ms Loss: 14.0 %)
2025-11-19T20:01:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 18.9 ms RTTd: 1.2 ms Loss: 10.0 %)
2025-11-19T20:01:26-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 18.8 ms RTTd: 1.2 ms Loss: 20.0 %)
2025-11-19T19:59:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.2 ms RTTd: 2.2 ms Loss: 21.0 %)
2025-11-19T19:59:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.1 ms RTTd: 2.1 ms Loss: 12.0 %)
2025-11-19T16:31:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.2 ms RTTd: 2.7 ms Loss: 10.0 %)
2025-11-19T16:31:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.2 ms RTTd: 2.8 ms Loss: 20.0 %)
2025-11-19T16:29:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.6 ms RTTd: 2.1 ms Loss: 21.0 %)
2025-11-19T16:29:30-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.8 ms RTTd: 2.3 ms Loss: 12.0 %)
2025-11-19T13:31:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 17.7 ms RTTd: 2.5 ms Loss: 10.0 %)
2025-11-19T13:31:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 17.9 ms RTTd: 2.6 ms Loss: 20.0 %)
2025-11-19T13:29:38-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 16.7 ms RTTd: 2.0 ms Loss: 21.0 %)
2025-11-19T13:29:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 16.6 ms RTTd: 2.0 ms Loss: 12.0 %)
2025-11-19T10:31:39-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.6 ms RTTd: 1.4 ms Loss: 10.0 %)
2025-11-19T10:31:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.5 ms RTTd: 1.3 ms Loss: 20.0 %)
2025-11-19T10:29:39-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 18.9 ms RTTd: 1.2 ms Loss: 21.0 %)
2025-11-19T10:29:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.1 ms RTTd: 1.3 ms Loss: 12.0 %)
2025-11-19T03:46:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.6 ms RTTd: 2.6 ms Loss: 10.0 %)
2025-11-19T03:46:25-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.6 ms RTTd: 2.7 ms Loss: 20.0 %)
2025-11-19T03:44:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.4 ms RTTd: 1.2 ms Loss: 21.0 %)
2025-11-19T03:44:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.4 ms RTTd: 1.2 ms Loss: 12.0 %)
2025-11-18T22:31:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.9 ms RTTd: 2.6 ms Loss: 10.0 %)
2025-11-18T22:31:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.8 ms RTTd: 2.5 ms Loss: 20.0 %)
2025-11-18T22:29:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 20.1 ms RTTd: 2.5 ms Loss: 21.0 %)
2025-11-18T22:29:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 20.0 ms RTTd: 2.4 ms Loss: 12.0 %)
2025-11-18T12:19:41-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 21.1 ms RTTd: 4.2 ms Loss: 10.0 %)
2025-11-18T12:19:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 21.1 ms RTTd: 4.3 ms Loss: 20.0 %)
2025-11-18T12:17:42-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.5 ms RTTd: 1.7 ms Loss: 21.0 %)
2025-11-18T12:17:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.8 ms RTTd: 1.8 ms Loss: 12.0 %)
2025-11-18T12:16:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.3 ms RTTd: 1.6 ms Loss: 10.0 %)
2025-11-18T12:16:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.2 ms RTTd: 1.7 ms Loss: 20.0 %)
2025-11-18T12:14:38-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.1 ms RTTd: 1.2 ms Loss: 21.0 %)
2025-11-18T12:14:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.1 ms RTTd: 1.1 ms Loss: 12.0 %)
2025-11-18T09:31:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.4 ms RTTd: 1.0 ms Loss: 10.0 %)
2025-11-18T09:31:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.4 ms RTTd: 1.0 ms Loss: 20.0 %)
2025-11-18T09:29:38-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.6 ms RTTd: 1.3 ms Loss: 21.0 %)
2025-11-18T09:29:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.6 ms RTTd: 1.2 ms Loss: 12.0 %)
2025-11-18T07:46:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 21.1 ms RTTd: 15.0 ms Loss: 10.0 %)
2025-11-18T07:46:26-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 21.4 ms RTTd: 15.9 ms Loss: 20.0 %)
2025-11-18T07:44:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.7 ms RTTd: 2.1 ms Loss: 21.0 %)
2025-11-18T07:44:27-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.6 ms RTTd: 2.0 ms Loss: 12.0 %)
2025-11-18T02:46:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.0 ms RTTd: 1.5 ms Loss: 10.0 %)
2025-11-18T02:46:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.1 ms RTTd: 1.6 ms Loss: 20.0 %)
2025-11-18T02:44:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.0 ms RTTd: 1.0 ms Loss: 21.0 %)
2025-11-18T02:44:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.0 ms RTTd: 1.0 ms Loss: 12.0 %)
2025-11-17T23:01:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 18.6 ms RTTd: 0.9 ms Loss: 10.0 %)
2025-11-17T23:01:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 18.7 ms RTTd: 0.9 ms Loss: 20.0 %)
2025-11-17T22:59:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 18.8 ms RTTd: 1.0 ms Loss: 21.0 %)
2025-11-17T22:59:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 18.9 ms RTTd: 1.0 ms Loss: 12.0 %)
2025-11-17T17:01:35-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.5 ms RTTd: 3.3 ms Loss: 10.0 %)
2025-11-17T17:01:29-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.4 ms RTTd: 3.0 ms Loss: 20.0 %)
2025-11-17T16:59:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.4 ms RTTd: 1.5 ms Loss: 21.0 %)
2025-11-17T16:59:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.4 ms RTTd: 1.6 ms Loss: 12.0 %)
2025-11-17T13:31:36-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 20.1 ms RTTd: 3.3 ms Loss: 10.0 %)
2025-11-17T13:31:30-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 20.2 ms RTTd: 3.4 ms Loss: 20.0 %)
2025-11-17T13:29:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.3 ms RTTd: 1.1 ms Loss: 21.0 %)
2025-11-17T13:29:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.5 ms RTTd: 1.5 ms Loss: 12.0 %)
2025-11-17T12:46:37-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.0 ms RTTd: 3.4 ms Loss: 10.0 %)
2025-11-17T12:46:31-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.3 ms RTTd: 3.5 ms Loss: 20.0 %)
2025-11-17T12:44:40-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 17.8 ms RTTd: 3.1 ms Loss: 22.0 %)
2025-11-17T12:44:34-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 17.8 ms RTTd: 3.0 ms Loss: 12.0 %)
2025-11-17T02:46:33-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> none RTT: 19.8 ms RTTd: 2.6 ms Loss: 10.0 %)
2025-11-17T02:46:27-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> loss RTT: 19.8 ms RTTd: 2.7 ms Loss: 20.0 %)
2025-11-17T02:44:32-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: loss -> down RTT: 19.0 ms RTTd: 0.9 ms Loss: 21.0 %)
2025-11-17T02:44:28-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: none -> loss RTT: 19.2 ms RTTd: 1.5 ms Loss: 12.0 %)
2025-11-17T01:15:08-08:00 Notice dpinger MONITOR: WAN_GW (Addr: 8.8.8.8 Alarm: down -> none RTT: 19.0 ms RTTd: 1.1 ms Loss: 0.0 %)
2025-11-17T01:14:57-08:00 Warning dpinger send_interval 1000ms loss_interval 4000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 0ms loss_alarm 0% alarm_hold 10000ms dest_addr 8.8.8.8 bind_addr 24.2.60.72 identifier "WAN_GW "
DOCSIS Event Log from MODEM (MAC Obfuscated)

00:00:36

Fri Nov 21 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

07:00:37

Fri Nov 21 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

14:48:44

Fri Nov 21 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

15:30:40

Fri Nov 21 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

03:00:36

Sat Nov 22 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

04:00:39

Sat Nov 22 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

11:00:30

Sat Nov 22 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

11:00:30

Sat Nov 22 2025 Critical (3) Started Unicast Maintenance Ranging – No Response received – T3 time-out;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

11:00:35

Sat Nov 22 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

11:45:35

Sat Nov 22 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

13:15:31

Sat Nov 22 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) SYNC Timing Synchronization failure – Failed to acquire QAM/QPSK symbol timing;;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:00:00:00:00:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

Time Not Established

Notice (6) Honoring MDD; IP provisioning mode = IPv6

18:15:32

Sat Nov 22 2025 Notice (6) DS profile assignment change. DS Chan ID: 32; Previous Profile: ; New Profile: 1 2 3.;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

18:15:39

Sat Nov 22 2025 Critical (3) UCD invalid or channel unusable;CM-MAC=00:XX:XX:XX:XX:ac;CMTS-MAC=00:XX:XX:XX:XX:00;CM-QOS=1.1;CM-VER=3.1;

My Attempts to Fix It

I called support. Multiple times. They blamed my WiFi (I’m hardwired). They blamed my modem (an Xfinity Approved MB8611 that worked without fault on Wave Broadband and Spectrum). They blamed my router (I tested with multiple devices). They sent a subcontracted technician. Then another subcontracted technician. Then a subcontracted crew. The crew ran a new coax line and grounded it to a water pipe that turns into PVC when it enters the ground (this doesn’t ground anything). Then they sent an Xfinity technician to look at the line.

The problem never changed. The only thing that changed is my download speed dropped from advertised 1200Mbps to <500Mbps.

I escalated to retention. They offered me nothing. I provided detailed technical documentation showing the exact pattern of the outages, the minute markers they occur at, the exact duration every single time. They didn’t understand it. They couldn’t escalate it.

I was transferred to technical support. The person did not care and put me on speaker phone with so much background noise I couldn’t hear myself think. I imagine he was rolling his eyes while trying his utmost to care less.

My Neighbor’s Attempts to Fix It

He called Xfinity about his TV cutting out repeatedly. The technician told him his UPS grounding his coax cable was causing the problem. So he ungrounded the cable, pocketed the spare cable, and my neighbor kept having the same issues.

What I Found

Just in the last 72 hours I have documented 20 consecutive outages using OPNSense’s built in gateway uptime monitor. Here’s what they show:

Every single outage lasted 124.8 ± 1.3 seconds. That’s not random hardware failure. That’s a timeout value hardcoded into something in Xfinity’s infrastructure.

The outages cluster at specific minute markers. 35% start at minute :44. 35% start at minute :29. This is scheduled automation. This is likely a cron job or automated task running at those exact times every hour.

The outages peak at specific hours. Most happen between noon and 1 PM. Others cluster in early morning around 2-3 AM. This is not random.

This is an infrastructure problem on Xfinity’s network. Not on my end. Not on my neighbor’s end. Upstream. Somewhere on their equipment something is failing for exactly 125 seconds multiple times per day.

I have the data. I have the patterns. I have another customer (my neighbor) on a different line experiencing the exact same thing.

Xfinity has all this information. They know about the problem. They just won’t investigate it.

Why They Won’t Fix It

  1. Support can’t understand technical data. They follow scripts. When I attempted to explain monitoring logs they had no framework for discussing it. They blamed my equipment because that’s what they’re trained to do.
  2. Nobody has authority to escalate. Retention transferred me to tech support. Tech support couldn’t care nor help. They dug up my yard and placed a new line which did nothing to solve the problem. Nobody would actually order an investigation into the upstream infrastructure.
  3. There’s no pressure to fix it. Xfinity is the only gigabit provider in this area. No competition. No alternatives. I can’t leave. So they don’t have to care.
  4. A 2-minute outage every few hours is “tolerable.” It’s annoying enough to frustrate customers but not enough to make them quit (since they have nowhere else to go). It’s cheap to ignore compared to actually investigating and fixing it.

There’s Also a Security Problem

About half of the Xfinity junction boxes in my neighborhood are unlocked or broken. Anyone can walk up and disconnect whomever they want.

If your home security system is on Xfinity with no wireless backup, someone can just walk to the street and physically disconnect your internet, rob your house, and your security system won’t notify you.

I’m Out of Options

I’ve done everything I can do as a customer:

  • Documented the problem professionally
  • Escalated through all available channels
  • Provided technical evidence
  • Been ignored at every level

The problem is real. My neighbor confirms it. Everyone downstream of whatever is broken on their infrastructure probably has it too.

I can’t fix this. Only Xfinity can. And they won’t.

So I’m publishing this hoping someone with actual authority, perhaps someone at a regulatory agency, someone at a news outlet, someone who has power over Xfinity sees this and decides to actually investigate.

Because I’m out of options. My neighbors are out of options. And Xfinity’s counting on us staying out of options. Because this is the reality of my neighborhood ( source ):

If you’re in Sacramento County and have Xfinity internet, check two things:

1. Walk to your junction box. Is it locked? If not, you have a physical security problem.

2. Look at where your cable grounds. Does it go to your electrical panel? Or to a water pipe? If it’s a water pipe or PVC, that’s wrong.

3. Have you noticed your connection drop briefly multiple times a day? Same times of day? If you see a pattern, document it. You might have the same problem.

If you want to report this:


Discover more from The Sacramento Bear

Subscribe to get the latest posts sent to your email.

NTSB report: Decryption of images from the Titan submersible camera [pdf]

Hacker News
data.ntsb.gov
2025-11-23 00:35:27
Comments...
Original Article
No preview for link for known binary extension (.pdf), Link: https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&FileExtension=pdf&FileName=Underwater%20Camera%20-%20Specialist%27s%20Factual%20Report-Rel.pdf.

Olmo 3 is a fully open LLM

Simon Willison
simonwillison.net
2025-11-22 23:59:46
Olmo is the LLM series from Ai2 - the Allen institute for AI. Unlike most open weight models these are notable for including the full training data, training process and checkpoints along with those releases. The new Olmo 3 claims to be "the best fully open 32B-scale thinking model" and has a strong...
Original Article

22nd November 2025

Olmo is the LLM series from Ai2—the Allen institute for AI . Unlike most open weight models these are notable for including the full training data, training process and checkpoints along with those releases.

The new Olmo 3 claims to be “the best fully open 32B-scale thinking model” and has a strong focus on interpretability:

At its center is Olmo 3-Think (32B) , the best fully open 32B-scale thinking model that for the first time lets you inspect intermediate reasoning traces and trace those behaviors back to the data and training decisions that produced them.

They’ve released four 7B models—Olmo 3-Base, Olmo 3-Instruct, Olmo 3-Think and Olmo 3-RL Zero, plus 32B variants of the 3-Think and 3-Base models.

Having full access to the training data is really useful. Here’s how they describe that:

Olmo 3 is pretrained on Dolma 3 , a new ~9.3-trillion-token corpus drawn from web pages, science PDFs processed with olmOCR , codebases, math problems and solutions, and encyclopedic text. From this pool, we construct Dolma 3 Mix , a 5.9-trillion-token (~6T) pretraining mix with a higher proportion of coding and mathematical data than earlier Dolma releases, plus much stronger decontamination via extensive deduplication, quality filtering, and careful control over data mixing. We follow established web standards in collecting training data and don’t collect from sites that explicitly disallow it, including paywalled content.

They also highlight that they are training on fewer tokens than their competition:

[...] it’s the strongest fully open thinking model we’re aware of, narrowing the gap to the best open-weight models of similar scale – such as Qwen 3 32B – while training on roughly 6x fewer tokens.

If you’re continuing to hold out hope for a model trained entirely on licensed data this one sadly won’t fit the bill—a lot of that data still comes from a crawl of the web.

I tried out the 32B Think model and the 7B Instruct model using LM Studio . The 7B model is a 4.16GB download, the 32B one is 18.14GB.

The 32B model is absolutely an over-thinker! I asked it to “Generate an SVG of a pelican riding a bicycle” and it thought for 14 minutes 43 seconds , outputting 8,437 tokens total most of which was this epic thinking trace .

I don’t usually quote the full SVG in these write-ups, but in this case it’s short enough that I think it’s worth sharing. The SVG comments give a great impression of what it was trying to do—it has a Bicycle, Bike frame, Pelican, Left and Right wings and even “Feet on pedals”.

<svg width="200" height="200" viewBox="0 0 100 100">
  <!-- Bicycle -->
  <circle cx="30" cy="60" r="15" stroke="black" fill="none"/>
  <circle cx="70" cy="60" r="15" stroke="black" fill="none"/>
  
  <!-- Bike frame -->
  <rect x="35" y="25" width="30" height="10" fill="saddlebrown"/>
  <line x1="35" y1="40" x2="30" y2="60" stroke="black" stroke-width="3"/>
  <line x1="65" y1="40" x2="70" y2="60" stroke="black" stroke-width="3"/>
  
  <!-- Pelican -->
  <ellipse cx="55" cy="65" rx="20" ry="15" fill="white"/>
  <polygon points="52 50,57 35,62 50" fill="black"/> <!-- Head/beak -->
  <circle cx="55" cy="45" r="2" fill="white"/>
  <circle cx="60" cy="45" r="2" fill="white"/>
  
  <polygon points="45 60,50 70,55 60" fill="lightgrey"/> <!-- Left wing -->
  <polygon points="65 60,70 70,55 60" fill="lightgrey"/> <!-- Right wing -->
  
  <!-- Feet on pedals -->
  <polygon points="25 75,30 85,35 75" fill="black"/>
  <polygon points="75 75,70 85,65 75" fill="black"/>
</svg>

Rendered it looks like this:

Two circles, each with a triangle sticking out from the bottom. They have bars leading up to a brown box. Overlapping them is a black triangle with white circles for eyes and two grey triangles that are probably meant to be wings. It is not recognizable as a pelican or a bicycle.

I tested OLMo 2 32B 4bit back in March and got something that, while pleasingly abstract, didn’t come close to resembling a pelican or a bicycle:

Blue and black wiggly lines looking more like a circuit diagram than a pelican riding a bicycle

To be fair 32B models generally don’t do great with this. Here’s Qwen 3 32B’s attempt (I ran that just now using OpenRouter ):

The bicycle is two black circles joined by two lines, with a weird rectangular saddle perched on top The pelican is a blue oval, a white circles with a yellow triangle in it and a weird eye shaped oval overlapping the blue one.

OlmoTrace

I was particularly keen on trying out the ability to “inspect intermediate reasoning traces”. Here’s how that’s described later in the announcement:

A core goal of Olmo 3 is not just to open the model flow, but to make it actionable for people who want to understand and improve model behavior. Olmo 3 integrates with OlmoTrace , our tool for tracing model outputs back to training data in real time.

For example, in the Ai2 Playground, you can ask Olmo 3-Think (32B) to answer a general-knowledge question, then use OlmoTrace to inspect where and how the model may have learned to generate parts of its response. This closes the gap between training data and model behavior: you can see not only what the model is doing, but why---and adjust data or training decisions accordingly.

You can access OlmoTrace via playground.allenai.org , by first running a prompt and then clicking the “Show OlmoTrace” button below the output.

I tried that on “Generate a conference bio for Simon Willison” (an ego-prompt I use to see how much the models have picked up about me from their training data) and got back a result that looked like this:

Screenshot of AI2 Playground interface with Olmo 3 32B Think model selected. User prompt reads "Generate a conference bio for Simon Willison". AI response shows expanded "Thoughts" section: "Okay, the user wants me to generate a conference bio for Simon Willison. Let me start by recalling who Simon is. He's a well-known developer, entrepreneur, and author in the tech community. I remember he's the co-founder of co:here and now works at Anthropic. He's famous for his work with SQLite, especially the datasette project. Also, he writes a blog called SimonWillison.net which is quite popular. First, I need to structure a typical conference bio. These usually include his current role, past positions, notable projects, and maybe some achievements or recognitions. I should highlight his key contributions like datasette, which allows easy exploration of SQLite databases via a web interface. That's a big deal in data journalism and open data circles." Right panel shows "OlmoTrace" feature described as "Documents from the training data that have exact text matches with the model response. Powered by infini-gram". First document excerpt discusses technology and innovation, with highlighted match text "societal implications of technology, emphasizing the" shown in bold, surrounded by text about responsibility and merging innovation with intellect. Second document excerpt about Matt Hall has highlighted match "is a software engineer and entrepreneur based in" shown in bold, describing someone in New York City who co-founded a PFP collection and works at Google Creative Lab. Note indicates "Document repeated 2 times in result" with "View all repeated documents" link.

It thinks I co-founded co:here and work at Anthropic, both of which are incorrect—but that’s not uncommon with LLMs, I frequently see them suggest that I’m the CTO of GitHub and other such inaccuracies.

I found the OlmoTrace panel on the right disappointing. None of the training documents it highlighted had anything to do with me—it appears to be looking for phrase matches (powered by Ai2’s infini-gram ) but the documents it found had nothing to do with me at all.

Can open training data address concerns of backdoors?

Ai2 claim that Olmo 3 is “the best fully open 32B-scale thinking model”, which I think holds up provided you define “fully open” as including open training data. There’s not a great deal of competition in that space though—Ai2 compare themselves to Stanford’s Marin and Swiss AI’s Apertus , neither of which I’d heard about before.

A big disadvantage of other open weight models is that it’s impossible to audit their training data. Anthropic published a paper last month showing that a small number of samples can poison LLMs of any size —it can take just “250 poisoned documents” to add a backdoor to a large model that triggers undesired behavior based on a short carefully crafted prompt.

This makes fully open training data an even bigger deal.

Ai2 researcher Nathan Lambert included this note about the importance of transparent training data in his detailed post about the release :

In particular, we’re excited about the future of RL Zero research on Olmo 3 precisely because everything is open. Researchers can study the interaction between the reasoning traces we include at midtraining and the downstream model behavior (qualitative and quantitative).

This helps answer questions that have plagued RLVR results on Qwen models, hinting at forms of data contamination particularly on math and reasoning benchmarks (see Shao, Rulin, et al. “Spurious rewards: Rethinking training signals in rlvr.” arXiv preprint arXiv:2506.10947 (2025). or Wu, Mingqi, et al. “Reasoning or memorization? unreliable results of reinforcement learning due to data contamination.” arXiv preprint arXiv:2507.10532 (2025).)

I hope we see more competition in this space, including further models in the Olmo series. The improvements from Olmo 1 (in February 2024 ) and Olmo 2 (in March 2025 ) have been significant. I’m hoping that trend continues!

How to Spot a Counterfeit Lithium-Ion Battery

Hacker News
spectrum.ieee.org
2025-11-22 23:08:57
Comments...
Original Article

As an auditor of battery manufacturers around the world, University of Maryland mechanical engineer Michael Pecht frequently finds himself touring spotless production floors. They’re akin to “the cleanest hospital that you could imagine–it’s semiconductor-type cleanliness,” he says. But he’s also seen the opposite, and plenty of it. Pecht estimates he’s audited dozens of battery factories where he found employees watering plants next to a production line or smoking cigarettes where particulates and contaminants can get into battery components and compromise their performance and safety.

Unfortunately, those kinds of scenes are just the tip of the iceberg. Pecht says he’s seen poorly assembled lithium-ion cells with little or no safety features and, worse, outright counterfeits . These phonies may be home-built or factory-built and masquerade as those from well-known global brands. They’ve been found in scooters , vape pens , e-bikes , and other devices, and have caused fires and explosions with lethal consequences.

The prevalence of fakes is on the rise, causing growing concern in the global battery market. In fact, after a rash of fires in New York City over the past few years caused by faulty batteries , including many powering e-bikes used by the city’s delivery cyclists , New York banned the sale of uncertified batteries. The city is currently setting up what will be its first e-bike battery-swapping stations as an alternative to home charging, in an effort to coax delivery riders to swap their depleted batteries for a fresh one rather than charging at home, where a bad battery could be a fire hazard.

Compared with certified batteries , whose public safety risks may be overblown, the dangers of counterfeit batteries may be underrated. “It is probably an order of magnitude worse with these counterfeits,” Pecht says.

Counterfeit Lithium-Ion Battery Risks

There are a few ways to build a counterfeit battery. Scammers often relabel old or scrap batteries built by legitimate manufacturers like LG , Panasonic , or Samsung and sell them as new. “It’s so simple to make a new label and put it on,” Pecht says. To fetch a higher price, they sometimes rebadge real batteries with labels that claim more capability than the cells actually have.

But the most prevalent fake batteries, Pecht says, are homemade creations. Counterfeiters can do this in make-shift environments because building a lithium-ion cell is fairly straightforward. With an anode, cathode, separator, electrolyte, and other electrical elements, even fly-by-night battery makers can get the cells to work.

What they don’t do is make them as safe and reliable as tested, certified batteries. Counterfeiters skimp on safety mechanisms that prevent issues that lead to fire. For example, certified batteries are built to stop thermal runaway , the chain reaction that can start because of an electrical short or mechanical damage to the battery and lead to the temperature increasing out of control.

Judy Jeevarajan , the vice president and executive director of Houston-based Electrochemical Safety Research Institute, which is part of Underwriters Laboratories (UL) Research Institutes , led a study of fake batteries in 2023. In the study, Jeevarajan and her colleagues gathered both real and fake lithium batteries from three manufacturers (whose names were withheld), and pushed them to their limits to demonstrate the differences.

One test, called a destructive physical analysis, involved dismantling small cylindrical batteries. This immediately revealed differences in quality. The legitimate, higher quality examples contained thick plastic insulators at the top and bottom of the cylinders, as well as axially and radially placed tape to hold the “jelly roll” core of the battery. But illegitimate examples had thinner insulators or none at all, and little or no safety tape.

“This is a major concern from a safety perspective as the original products are made with certain features to reduce the risk associated with the high energy density that li-ion cells offer,” Jeevarajan says.

Jeevarajan’s team also subjected batteries to overcharging and to electrical shorts. A legitimately tested and certified battery, like the iconic 18650 lithium-ion cylinder , counters these threats with internal safety features such as positive temperature coefficient, where a material gains electrical resistance as it gets hotter, and a current interrupt device (CID), which automatically disconnects the battery’s electrical circuit if the internal pressure rises too high. The legit lithium battery in Jeevarajan’s test had the best insulators and internal construction. It also had a high-quality CID that prevented overcharging, reducing risk a fire. Neither of the other cells had one.

Despite the gross lack of safety parts in the batteries, great care had clearly gone into making sure the counterfeit labels had the exact same shade and markings as the original manufacturer’s, Jeevarajan says.

How to Spot a Counterfeit Battery

Because counterfeiters are so skilled at duplicating manufacturers’ labels, it can be hard to know for sure whether the lithium batteries that come with a consumer electronics device, or the replacements that can be purchased on sites like eBay or Amazon , are in fact the genuine article. It’s not just individual consumers who struggle with this. Pecht says he knows of instances where device makers have bought what they thought were LG or Samsung batteries for their machines but failed to verify that the batteries were the real thing.

“One cannot tell from visually inspecting it,” Jeevarajan says. But companies don’t have to dismantle the cells to do their due diligence. “The lack of safety devices internal to the cell can be determined by carrying out tests that verify their presence,” she says. A simple way, Pecht says, is to have a comparison standard on hand – a known, legitimate battery whose labeling, performance, or other characteristics can be compared to a questionable cell. His team will even go as far as doing a CT scan to see inside a battery and find out whether it is built correctly.

Of course, most consumers don’t have the equipment on hand to test the veracity of all the rechargeable batteries in their homes. To shop smart, then, Pecht advises people to think about what kind of batteries and devices they’re using. The units in our smartphones and the large, high-capacity batteries found in electric vehicles aren’t the problem; they are subject to strict quality control and very unlikely to be fake. By far, he says, the more likely places to find counterfeits are the cylindrical batteries found in small, inexpensive devices.

“They are mostly found as energy and power sources for portable applications that can vary from your cameras , camcorders , cell phones, power banks , power tools, e-bikes and e-scooters,” adds Jeevarajan. “For most of these products, they are sold with part numbers that show an equivalency to a manufacturer’s part number. Electric vehicles are a very high-tech market and they would not accept low-quality or cells and batteries of questionable origin.”

The trouble with battling the counterfeit battery scourge, Pecht says, is that new rules tend to focus on consumer behavior, such as trying to prevent people from improperly storing or charging e-bike batteries in their apartments. Safe handling and charging are indeed crucial, but what’s even more important is trying to keep counterfeits out of the supply chain . “They want to blame the user, like you overcharged it or you did this wrong,” he says. “But in my view, it’s the cells themselves” that are the problem.

Windows ARM64 Internals: Deconstructing Pointer Authentication

Lobsters
www.preludesecurity.com
2025-11-22 22:00:49
Comments...
Original Article

Pointer Authentication Code , or PAC, is an anti-exploit/memory-corruption feature that signs pointers so their use (as code or data) can be validated at runtime. PAC is available on Armv8.3-A and Armv9.0-A (and later) ARM architectures and leverages virtual addressing in order to store a small cryptographic signature alongside the pointer value.

On a typical 64-bit processor a pointer is considered a "user-mode" pointer if bit 47 of a 64-bit address is set to 0 (meaning, then, bits 48-63 are also 0). This is known as a canonical user-mode address. If bit 47 is set to 1, bits 48-63 are also set to 1, with this being considered a canonical kernel-mode address. Additionally, LA57, ARM 52 or 56 bit, or similar processors extend the most significant bit out even further (and PAC can also be enabled in the ARM-specific scenarios). For our purposes, however, we will be looking at a typical 64-bit processor with the most significant bit being bit 47.

It has always been an "accepted" standard that the setting of the most significant bit denotes a user-mode or kernel-mode address – with even some hardware vendors, like Intel, formalizing this architecture in actual hardware with CPU features like Linear Address Space Separation (LASS). This means that bits 48-63 are unused on a current, standard 64-bit processor, as the OS typically ignores them. Because they are unused, this allows PAC to store the aforementioned signature in these unused bits alongside the pointer itself.

As mentioned, these “unused” bits are now used to store signing information about a particular pointer in order to validate and verify execution and/or data access to the target memory address. Special CPU instructions are used to both generate and validate cryptographic signatures associated with a particular pointer value. This blog post will examine the Windows implementation of PAC on ARM64 installations of Windows, which, as we will see, supports a very specific implementation of PAC in both user-mode and kernel-mode.

PAC Enablement on Windows

PAC enablement on Windows begins at the entry point of ntoskrnl.exe, KiSystemStartup. KiSystemStartup is responsible for determining if PAC is supported on Windows and also for initializing basic PAC support. KiSystemStartup receives the loader parameter block (LOADER_PARAMETER_BLOCK) from winload.efi, the Windows boot loader. The loader block denotes if PAC is supported. Specifically, the loader parameter block extension (LOADER_PARAMETER_EXTENSION) portion of the loader block defines a bitmask of various features which are present/supported, so say the boot loader. The PointerAuthKernelIpEnabled bit of this bitmask denotes if PAC is supported. If PAC is supported, the loader parameter block extension is also responsible for providing the initial PAC signing key (PointerAuthKernelIpKey) used to sign and authenticate all kernel-mode pointers (we will see later that the "current" signing key is updated many times). When execution is occurring in kernel-mode, this is the key used to sign kernel-mode pointers. The bootloader generates the key in OslPrepareTarget by calling the function SymCryptRngAesGenerate to generate the initial kernel pointer signing key passed via the loader parameter block.

The ARM architecture supports having multiple signing keys for different scenarios, like signing instruction pointers or data pointers with different keys. Typically, "key A" and "key B" (as they are referred to), which are stored in specific system registers, are used for signing pointers used in instruction executions (like return addresses). Windows currently only uses PAC for "instruction pointers" (more on this later) and it also it only uses "key B" for cryptographic signatures and, therefore, loads the target pointer signing value into the APIBKeyLo_EL1 and APIBKeyHi_EL AArch64 system registers . These "key registers" are specific system registers, which are special registers on ARM systems which control various behaviors/controls/statuses for the system, and are responsible for maintaining the current keys used for signing and authenticating pointers. These two registers ("lo" and hi") each hold a single 64-bit value, which results in a concatenated 128-bit key. EL1, in this case, refers to exception level “1” - which denotes the ARM-equivalent of “privilege level” the CPU is running in (as ARM-based CPUs are “exception-oritented”, meaning system calls, interrupts, etc. are all treated as “exceptions”). Typically EL1 is associated with kernel-mode. User-mode and kernel-mode, for Windows, share EL1’s signing key register (although the "current" signing key in the register changes depending on if a processor is executing in kernel-mode or user-mode). It should be noted that although the signing key for user-mode is stored in an EL1 register, the register itself (e.g., reading/writing) is inaccessible from user-mode (EL 0).

It is possible to examine the current signing key values using WinDbg. Although WinDbg, on ARM systems, has no extension to read from these system registers, it was discovered through trial-and-error that it is possible to leverage the rdmsr command in WinDbg to read from ARM system registers using the encoding values provided by the ARM documentation. The two PAC key system registers used by Windows have the following encodings:

1. APIBKeyLo_EL1
- op0: 0b11 (3)
- op1: 0b000 (0)
- CRn: 0b0010 (2)
- CRm: 0b0001 (1)
- op2: 0b010 (2)

2. APIBKeyHigh_EL1
- op0: 0b11 (3)
- op1: 0b000 (0)
- CRn: 0b0010 (2)
- CRm: 0b0001 (1)
- op2: 0b011 (3)

Concatenating these binary values into their hexadecimal values, it is then possible to leverage the rdmsr command to view the current signing key values:

After the initial signing key value has been configured, the kernel continues executing its entry point in order to continue to fill out some of the basic functionality of the kernel (although the kernel is not done yet being fully initialized). Almost immediately after performing basic PAC initialization, the function KiInitializeBootStructures is called from the kernel entry point, which also receives the loader parameter block and initializes various items such as the feature settings bitmask, setting the proper stack sizes (especially for "special" stacks like ISR stacks and DPC stacks), etc. One of those crucial things that this function does is call into KiDetectPointerAuthSupport, which is responsible for the bulk of the PAC initialization. This function is responsible for reading from the appropriate PAC-related ARM system registers in order to determine what specific PAC features the current CPU is capable of supporting.

After the current CPU's supported options are configured, "phase 0" of the system initialization process (achieved via KeInitsystem) will fully enable PAC. Currently, on Windows 11 24H2 and 25H2 preview builds, enablement is gated through a feature flag called Feature_Pointer_Auth_User__private_featureState. If the feature flag is enabled, a secondary check is performed to determine if a registry override option to disable PAC was present. Additionally, if the PAC feature flag is disabled, a check is performed to see if a registry override to enable PAC is present. The applicable registry paths are:

- HKLM\System\CurrentControlSet\Control\Session Manager\Kernel\PointerAuthUserIpEnabled

- HKLM\System\CurrentControlSet\Control\Session Manager\Kernel\PointerAuthUserIpForceDisabled

Note that the "enablement" flags are not directly tied one-to-one to the "supported flags". As previously seen, KePointerAuthEnabled is masked with the value 4 in KiSystemStartup before the "supported" options are even evaluated. Additionally, note that the KePointerAuthEnabled variable is marked as read-only and is present in the CFGRO section, which is also read-only in the VTL 0 guest page tables (known in ARM as the “Stage 2 tables” with “Stage 2” tables being the final level of translation from guest memory to system memory) thanks to the services of Hypervisor-Protected Code Integrity (HVCI) , along with KePointerAuthKernelIpKey and KePointerAuthMask. As seen below, even using WinDbg, it is impossible to overwrite these global variables as they are read-only in the “Stage 2” page tables.

As an aside, the supported and enabled PAC features can be queried via NtQuerySystemInformation through the SystemPointerAuthInformation class.

C:\>C:\WindowsPAC.exe
[+] System Pointer Authentication Control (PAC) settings:
[>] SupportedFlags: 0x1F
[>] EnabledFlags: 0x101
[*] AddressAuthFaulting: TRUE
[*] AddressAuthQarma: TRUE
[*] AddressAuthSupported: TRUE
[*] GenericAuthQarma: TRUE
[*] GenericAuthSupported: TRUE
[*] KernelIpAuthEnabled: TRUE
[*] UserGlobalIpAuthEnabled: FALSE
[*] UserPerProcessIpAuthEnabled: TRUE

Once the appropriate PAC-related initialization flags have been set, PAC is then enabled on a per-process basis (if per-process PAC is supported, which currently on Windows it is). For user-mode PAC, the enablement process begins at process creation, specifically during the allocation of the new process object. If PAC is enabled, each user-mode process (meaning EPROCESS->Flags3.SystemProcess is not set) is unconditionally opted-in to PAC (as all kernel-mode code shares a global signing key).

Additionally, likely as a side effect of Intel CET enablement on x86-based installations of Windows, the mitigation value CetDynamicApisOutOfProcOnly is also set unconditionally for every process except for the Idle process on Windows.

For the sake of completeness, the CET dynamic address range feature is not actually supported as the PROCESSINFOCLASS enum value ProcessDynamicEnforcedCetCompatibleRanges, for the NtSetInformationProcess system service, always returns STATUS_NOT_SUPPORTED on Windows ARM systems.

Returning to user-mode PAC, Windows SDK contains two documented ways to enable/disable PAC for user-mode processes. For extended process creation parameters, the following parameters are available in the SDK:

//
// Define the ARM64 user-mode per-process instruction pointer authentication
// mitigation policy options.
//

#define PROCESS_CREATION_MITIGATION_POLICY2_POINTER_AUTH_USER_IP_MASK      (0x00000003ui64 << 44)

#define PROCESS_CREATION_MITIGATION_POLICY2_POINTER_AUTH_USER_IP_DEFER     (0x00000000ui64 << 44)

#define PROCESS_CREATION_MITIGATION_POLICY2_POINTER_AUTH_USER_IP_ALWAYS_ON (0x00000001ui64 << 44)

#define PROCESS_CREATION_MITIGATION_POLICY2_POINTER_AUTH_USER_IP_ALWAYS_OFF
(0x00000002ui64 << 44)

#define PROCESS_CREATION_MITIGATION_POLICY2_POINTER_AUTH_USER_IP_RESERVED  (0x00000003ui64 << 44)

Additionally, for runtime enablement/disablement, the following structure can be supplied with the ProcessUserPointerAuthPolicy:

typedef struct _PROCESS_MITIGATION_USER_POINTER_AUTH_POLICY {
union {
DWORD Flags
struct {
DWORD EnablePointerAuthUserIp : 1;
DWORD ReservedFlags : 31;
} DUMMYSTRUCTNAME;
} DUMMYUNIONNAME;
} PROCESS_MITIGATION_USER_POINTER_AUTH_POLICY,
*PPROCESS_MITIGATION_USER_POINTER_AUTH_POLICY;

However, testing and reverse engineering revealed that PAC is unconditionally enabled on user-mode processes (as shown above) with no way to disable the mitigation either at process creation (e.g., creating a child process with extended parameters) or by calling SetProcessMitigationPolicy at runtime. The only other supported way to enable a process mitigation at process creation is to use the ImageFileExecutionOptions (IFEO) registry key. This functionality is wrapped by the "Exploit Protection" UI on Windows systems, but the registry value can be set manually. Unfortunately, there is no PAC Exploit Protection setting in the UI.

Outside of the exploit mitigation policy for PAC, there is also an audit-mode exploit mitigation policy option in the ImageFileExecutionOptions policy map. This can be confirmed through the presence of the mitigation flag values of AuditPointerAuthUserIp and AuditPointerAuthUserIpLogged in the MitigationFlags2Values field of a process object on Windows.

The IFEO registry key contains a list of processes that have IFEO values. One of the items encapsulated in the IFEO key, as mentioned, is both the mitigation policy settings and audit-mode mitigation policy settings (meaning that an ETW event is logged but the target operation is not blocked/process is not terminated by a mitigation violation) for a target process. These per-process mitigation values are used in making considerations about what mitigation policies will be applied to a particular target process at process creation time. On a default installation of Windows 11 24H2 running an ARM build of Windows, no processes have the audit-mode PAC flags set.

Further investigation reveals that this is because there is no way to set the PAC audit-mode exploit policy value on a per-process basis, even through the IFEO key. This is because if pointer authentication is enabled, for example, the slot in the map (represented by the 0x000000000000X000 nibble) in which audit-mode PAC may be enabled is explicitly overridden by PspAllocateProcess (and no ETW event exists in the manifest of the Microsoft-Windows-Security-Mitigations ETW provider for PAC violations).

Once PAC support has been instantiated for the process, the per-process signing key is configured. Yes, this means that each process has its own key it can use to sign pointers. This occurs in PspAllocateProcess and, if a process has not opted in to inheriting the signing key, a random key is generated with BCryptGenRandom.

The "per-process" signing key differs from the initial (kernel) signing key that was configured in KiSystemStartup. This is because, obviously, execution is in kernel mode when the initial signing key is instantiated. However, the implementation of PAC on Windows (as we can see above) instruments a per-process signing key (along with a single kernel key). When execution transitions into user mode, the signing key system register(s) are updated to the current process signing key (which is maintained through a process object). The example below outlines the current PAC signing key being updated to that of a user-mode process, specifically when a return into user-mode happens after a system call is handled by the kernel (KiSystemServiceExit).

This is how the necessary PAC infrastructure is updated for user-to-kernel and kernel-to-user transitions and how kernel-mode and user-mode PAC on Windows is set up. Let's now examine what Windows does when the proper infrastructure is in place.

Windows PAC As An Exploit Mitigation

Windows currently offers an implementation of PAC (with the ability to expand in the future). Windows currently supports PAC for signing and authenticating "instruction pointers". The way that this manifests itself, however, really results in the signing of return addresses. On Windows, for both user-mode and kernel-mode ARM64 code, one can specify the /guard:signret(-) compiler flag to either explicitly enable or disable the signing of return addresses. Enabling this flag instruments the pacibsp and autibsp instructions into the prologue and epilogue of each function, which are "PAC" instructions used to both sign and subsequently validate return addresses.

In the ARM64 architecture, the semantics of preserving return addresses across call boundaries slightly differ from Intel x86. On x86-based systems, a call instruction will also push the target return address onto the stack. Then, right before a return, the aforementioned return address is "popped" off of the stack and loaded into the instruction pointer. On ARM64 the bl (Branch with Link, similar to a call) instruction will instead place the current, in-scope return address an architectural register (lr, or "link register") with a typical operating system, like Windows, also storing this value on the stack to preserve the return address so the lr register can be used for the next call's return address (meaning the return addresses are still stored on the stack on ARM, even with the presence of lr).

The pacibsp instruction will use "key b" (APIBKeyLo_EL1 and APIBKeyHi_EL1) and the value of the in-scope stack pointer to sign the return address. The target return address will remain in this state, with the upper bits (non-canonical) being transformed through the signing.

This assumes, however, that there is already a return address to process. What if a user-mode thread, for example, is just entering its initial execution, and there is no return address? Windows has two functions (for user-mode and kernel-mode) that will generate the necessary "first" signed return address via KiGenerateSignedReturnAddressForStartUserThread. These functions accept the initial stack value as the value to use in the signing of the return address, using instead the pacib instruction, which is capable of using a general-purpose architectural register in the signing process instead of just defaulting to "the current stack pointer".

At this point, the return address (stored in lr, but also present on the stack) has been signed. The in-scope function performs its work and eventually the epilogue of a function is reached (which is responsible for returning to the caller for the current function). When the epilogue is reached, but before the ret has been executed, the autibsp instruction is used to authenticate the return address (in lr) before performing the return control-flow transfer. This will result in transforming the value in lr back to the "original" return address so that the return occurs back into a valid memory address.

The effectiveness of PAC, however, relies on what happens if a return address has been corrupted with a malicious return address, like a ROP gadget or the corruption of a return address through a stack-based buffer overflow. In the example below, this is outlined by corrupting a return address on the stack with another return address on the stack. Both of these addresses used in this memory corruption example are signed, but, as we can recall from earlier, return addresses are signed with the considerations of the current in-scope stack pointer (meaning they are tied to a stack frame). Because the corrupted return address does not correspond to an "in-scope" stack frame, the authentication of the in-scope return address (which has been corrupted) results in a fastfail with the code FAST_FAIL_POINTER_AUTH_INVALID_RETURN_ADDRESS - and the application crashes. One interesting note, as you can see, is that WinDbg can convert a signed return address on the stack to its actual unsigned value (and appropriate symbol name).

Shifting focus slightly, when a kernel-mode PAC violation, identical to the previous scenario, occurs, a KERNEL_SECURITY_CHECK_FAILURE ensues, with the type of memory safety violation being FAST_FAIL_POINTER_AUTH_INVALID_RETURN_ADDRESS.

Secure Kernel And PAC

The curious reader may notice that the kernel itself is responsible for managing the key values for PAC. Additionally, we already covered the fact that the in-memory variable which tracks the kernel’s PAC signing key (used to sign kernel pointers) is read-only in VTL 0 memory thanks to the services of HVCI. However, the in-memory representation is simply a reflection of the system register value(s) we have talked about before - the APIBKeyLo_EL1 and APIBKeyHi_EL AArch64 registers (specifically when execution is in kernel-mode, loading the per-boot kernel-mode PAC key). What is preventing an attacker, in kernel-mode, from modifying the contents of this system register at any given time? After all, the register is writable from kernel-mode because the configuration is not delegated to a higher security boundary? To help alleviate this problem, Secure Kernel Patch Guard , more commonly referred to as “HyperGuard” - a security feature promulgated by the Secure Kernel - is used! HyperGuard achieves much of what PatchGuard attempts to defend against (modification of kernel data structures, MSRs on x86 systems, control registers, etc.) but it does so deterministically , as opposed to PatchGuard, because HyperGuard runs at a higher security boundary than the code it is attempting to defend (VTL 0’s kernel).

HyperGuard uses what is known as extents , which are definitions of what components/code/data/etc. should be protected by HyperGuard. On ARM64 installations of Windows, an ARM64-specific HyperGuard extent exists - the PAC system register extent. This extent is used by HyperGuard to ask the hypervisor to intercept certain items of interest - such as modifications to an MSR (or ARM64 system register), certain memory access operations, etc. Specifically for the ARM64 version of the Secure Kernel, an extent is registered for monitoring modifications to the PAC key system registers. This is done in securekernel!SkpgxInitializeInterceptMasks.

Although ARM-based hypervisors do not have “Virtual Machine Control Structure”, or VMCS (in the “canonical” sense that x86-based systems do, such as having dedicated instructions in the ISA for reading/writing to the VMCS), ARM hypervisors still must maintain the “state” of a guest. This, obviously, is used in situations like when a processor starts executing in context of the hypervisor software (due to a hypervisor call (HVC call), or other exceptions into the hypervisor), or when a guest starts resuming its execution. Part of this state - as is the case with x86-based systems - is the set of virtual registers (e.g., registers which are preserved across exception level changes into/out of the hypervisor and are specific to a guest). Among the virtual registers which are configurable by the hypervisor are, as you may have guessed, the “lo” and “hi” PAC signing key registers! This is what the function from the screenshot above intends to achieve - securekernel!SkpgArm64ReadRegister64.

Microsoft documents many of the 64-bit virtualized-registers. Among the undocumented registers, however, are the ARM-based virtualized registers. However, we can see above that values 0x4002E and 0x4002F correspond to the virtual/private PAC signing registers. For completeness sake, 0x40002 corresponds to SCTLR_EL1. This was determined by examining the bit being processed (bit 30, via the 0x40000000 mask). This was previously seen, in the beginning of our analysis, by the toggling of SCTLR_EL1.EnIB bit (bit 30).

This entire configuration allows the Secure Kernel to intercept, via HyperGuard, any unauthorized modification of the PAC signing key register.

Conclusion

ARM-based processors, without the presence of backwards-edge control flow integrity (CFI) mitigations like CET , are able to effectively leverage PAC to defend against return address corruption. Windows, as we have seen, currently leverages PAC only in limited circumstances (like the protection of return addresses), which is standard on many mainstream implementations of PAC (with the ability in the future, if feasible, to expand into protection of data accesses). PAC provides a viable solution to protect non-x86-based processors from certain classes of memory corruption exploits. In addition, current-generation ARM64 Microsoft devices, like the Surface Pro, are not shipped with chips that can support the Memory Tagging Extension (MTE) feature. Although not implemented today on Windows systems, the implementation of both PAC and MTE in the future would serve to greatly increase the cost of memory corruption exploits. Given the protections afforded by the hypervisor, plus the current implementation of PAC, ARM-based Windows provides both user-mode and kernel-mode code with additional security against memory corruption exploits.

WorldGen by Meta – Text to Immersive 3D Worlds

Hacker News
www.meta.com
2025-11-22 21:20:24
Comments...

A Reverse Engineer’s Anatomy of the macOS Boot Chain & Security Architecture

Lobsters
stack.int.mov
2025-11-22 21:16:25
Comments...
Original Article

1.0 The Silicon Root of Trust: Pre-Boot & Hardware Primitives

The security of the macOS platform on Apple Silicon is not defined by the kernel; it is defined by the physics of the die. Before the first instruction of kernelcache is fetched, a complex, cryptographic ballet has already concluded within the Application Processor (AP). This section dissects the immutable hardware logic that establishes the initial link in the Chain of Trust.

1.1 The Reset Vector & Boot ROM (SecureROM)

The Apple Silicon boot process begins in a state of absolute trust, anchored by the Boot ROM (often colloquially referred to as SecureROM). This code is mask-programmed into the silicon during fabrication. It is immutable, unpatchable, and serves as the hardware root of trust for the entire platform.

1.1.1 Execution at Exception Level 3 (EL3): Analyzing RVBAR_EL3

Upon Power-On Reset (POR), the cores of the M-series SoC (and A-series) initialize in the highest privilege state defined by the ARMv8/v9 architecture: Exception Level 3 (EL3) .

While Apple does not utilize TrustZone in the traditional sense (preferring their proprietary Secure Enclave for secure world operations), the AP Boot ROM executes at EL3 to perform the necessary low-level hardware initialization that requires access to secure configuration registers that are subsequently locked or hidden from EL2/EL1.

The execution flow begins at the address defined in the Reset Vector Base Address Register (RVBAR_EL3) . On recent Apple Silicon (M1/M2/M3), the memory map places the Boot ROM at a high base address, typically 0x100000000 .

The Initial Instruction Stream:
The very first instructions executed by the silicon are responsible for establishing a sane C execution environment from a raw hardware state. A disassembly of the entry point typically reveals the following sequence:

  1. Interrupt Masking: DAIF bits are set to mask all interrupts (IRQ, FIQ, SError, Debug). The Boot ROM operates in a strictly polled mode; interrupts are nondeterministic and introduce attack surface.
  2. Cache Invalidation: The instruction and data caches are invalidated to prevent cold-boot attacks or stale data usage.
  3. Stack Setup: The Stack Pointer ( SP_EL3 ) is initialized to point to a dedicated region of on-chip SRAM (Static RAM). Note that DRAM is not initialized at this stage. The Boot ROM runs entirely within the constraints of the SoC's internal SRAM (often referred to as L2 cache-as-RAM in older exploits, though modern SoCs have dedicated boot SRAM).
  4. MMU Configuration: The System Control Register ( SCTLR_EL3 ) is written to enable the MMU, mapping the Boot ROM text as Read-Only/Executable and the SRAM stack/heap as Read-Write/No-Execute.

RE Note: Apple’s implementation of EL3 is ephemeral. Unlike Android/Qualcomm devices where EL3 hosts a persistent Secure Monitor (QSEE), Apple Silicon demotes the exception level to EL2 (Hypervisor) or EL1 (Kernel) before handing off control to the next stage. Once the Boot ROM exits, EL3 is effectively locked out, and the RVBAR_EL3 is often locked to prevent re-entry.

1.1.2 The GID Key (Group ID): Hardware-entangled Decryption

The Boot ROM's primary objective is to load the Low-Level Bootloader (LLB). However, the LLB stored on the NAND flash (or NOR SPI flash in some configurations) is encrypted. To decrypt it, the Boot ROM utilizes the GID Key (Group ID Key) .

The GID Key is a 256-bit AES key fused into the silicon during manufacturing. It is shared across all processors of the same class (e.g., all M3 Pro chips share a GID, but it differs from the M3 Max).

The "Black Box" AES Engine:
Crucially, the GID Key is not accessible via software. There is no Memory-Mapped I/O (MMIO) register that an attacker can read to dump the key. Instead, the AES engine operates as a black box:

  1. Input: The Boot ROM writes the encrypted ciphertext (the LLB payload) into the AES engine's input FIFO.
  2. Control: The Boot ROM sets a specific control bit in the AES configuration register (e.g., AES_CMD_USE_GID ).
  3. Operation: The hardware AES engine loads the GID key from the fuses directly into the crypto core, performs the decryption, and flushes the key from the internal registers.
  4. Output: The plaintext is written to the output FIFO, which the Boot ROM then reads into SRAM.

This architecture ensures that even if an attacker gains arbitrary code execution within the Boot ROM (as seen in the checkm8 exploit on older A-series chips), they cannot extract the GID key to decrypt firmware images offline. Decryption must happen on-device .

1.1.3 The Public Key Accelerator (PKA): Hardware-Enforced Verification

Decryption provides confidentiality, but not integrity. To prevent the execution of malicious firmware, the Boot ROM enforces strict code signing using the Public Key Accelerator (PKA) .

The PKA is a dedicated hardware block optimized for asymmetric cryptography (RSA and ECC). The verification flow is as follows:

  1. Root of Trust Hash: The SHA-384 hash of the Apple Root CA public key is burned into the device's eFuses. This is the immutable anchor.
  2. Manifest Parsing: The Boot ROM parses the Image4 (img4) container of the LLB. It extracts the Image4 Manifest (IM4M), which contains the payload's signature and the public key used to sign it.
  3. Key Verification: The Boot ROM hashes the public key found in the manifest and compares it against the hash in the eFuses. If they do not match, the boot halts (DRAM is never initialized, and the device enters DFU mode).
  4. Signature Verification: The Boot ROM offloads the signature verification to the PKA. It passes the SHA-384 hash of the payload and the RSA/ECC signature. The PKA performs the mathematical verification and returns a boolean result to a status register.

Fault Injection Hardening:
Modern Apple Boot ROMs employ glitch-resistant logic around the PKA check. Rather than a simple B.EQ (Branch if Equal) instruction following the PKA result—which could be bypassed via voltage glitching—the code often employs redundant checks, loop invariants, or specific register values that must be populated by the PKA hardware itself to allow the boot flow to proceed.

1.1.4 RE Focus: Dev vs. Prod Fused Silicon

For the reverse engineer, distinguishing between Development (Dev) and Production (Prod) fused silicon is vital. The behavior of the Boot ROM changes fundamentally based on the "Security Domain" fuse.

  • Production (CPFM 01): The standard consumer state. JTAG is physically disabled. The GID key is locked to the production value. The Boot ROM enforces the full Apple Root CA chain.
  • Development (CPFM 00/03): Used internally by Apple engineers.
    • JTAG Enablement: The DBGEN signal is asserted, allowing hardware debuggers (like Lauterbach or Astris) to halt the core immediately after reset.
    • Demotion: Dev-fused chips often allow "demotion," enabling the device to boot unsigned or custom-signed firmware images.
    • GID Key Variance: Dev chips often use a different GID key, meaning firmware encrypted for Prod devices cannot be decrypted on Dev hardware, and vice versa.

Identifying Silicon State:
You can identify the fuse status by querying the CHIP_ID or MOJO registers via USB when the device is in DFU mode.

  • 0x8000... usually indicates a Production fuse.
  • 0x0000... or specific bitmasks in the ECID response indicate Development/Debug fusing.

The "Un-dumpable" Region:
Once the Boot ROM prepares to jump to the next stage (LLB), it performs a lockdown sequence. It writes to the memory controller to unmap its own address range ( 0x100000000 ). Any subsequent attempt by the LLB or Kernel to read the Boot ROM address space will result in a bus error or read back zeros. This "hide-and-seek" mechanism is why dumping the Boot ROM requires a vulnerability present during the Boot ROM execution window (like checkm8 or a voltage glitch at reset).

1.2 Proprietary ISA Extensions (arm64e+)

While the M-series chips ostensibly implement the ARMv8.6-A (and increasingly ARMv9) specification, Apple has aggressively extended the Instruction Set Architecture (ISA) with proprietary logic. For the reverse engineer, standard ARM documentation is insufficient. Understanding the security posture of macOS Tahoe requires mastering these custom extensions, as they form the hardware enforcement layer for the new kernel isolation model.

1.2.1 Pointer Authentication (PAC): The Cryptographic Control Flow

Apple’s implementation of ARMv8.3-PAuth is the most pervasive security mitigation in the XNU kernel. It repurposes the unused high-order bits of 64-bit virtual addresses (typically bits 63–47, depending on Translation Control Register TCR_EL1 settings) to store a cryptographic signature, or Pointer Authentication Code (PAC).

The Key Hierarchy:
The hardware maintains five distinct 128-bit keys in system registers, accessible only at EL1 or higher. These keys segregate pointer types to limit the utility of a signing gadget:

  • APIAKey / APIBKey (Instruction): Signs code pointers (function pointers, return addresses).
  • APDAKey / APDBKey (Data): Signs data pointers. Crucial for protecting C++ vtables in IOKit ( OSObject ).
  • APGAKey (Generic): Signs arbitrary data blobs, effectively a hardware-accelerated MAC.

The AUT Failure Mechanism (Canonical Non-Valid):
For the reverse engineer analyzing crash dumps, understanding the failure mode is critical. When an AUT* instruction (e.g., AUTIA ) is executed on a corrupted or forged pointer, the CPU does not immediately raise an exception.

Instead, the hardware corrupts the pointer in a deterministic way to ensure it causes a translation fault upon dereference.

  1. Validation: The CPU recalculates the PAC.
  2. Mismatch: If the calculated PAC does not match the bits in the pointer, the CPU flips specific high-order bits (typically bit 62 for data pointers, or bit 61 for instruction pointers, depending on Top Byte Ignore settings).
  3. Result: The pointer becomes "canonical non-valid." It looks like a kernel pointer (high address) but falls into a reserved, unmapped range.
  4. Crash: The subsequent LDR or BLR triggers a Data Abort or Prefetch Abort.

RE Tip: If you see a crash where x0 is 0x007f... or 0x00ff... (a pointer with high bits set but not fully canonical), you are looking at a PAC authentication failure, not a standard NULL dereference or heap corruption.

1.2.2 Branch Target Identification (BTI): The Landing Pads

Often deployed in tandem with PAC ( -mbranch-protection=standard ), BTI mitigates Jump-Oriented Programming (JOP). It enforces a state machine on indirect branches.

  • Marking Pages: The Global Page Table (GPT) or PTE entries now include a Guarded Page ( GP ) bit.
  • The BTI Instruction: This is a "hint" instruction (NOP on older silicon). It acts as a valid landing pad.
  • Enforcement: When the CPU executes an indirect branch ( BR , BLR ) targeting a Guarded Page, the very next instruction must be a BTI instruction of the correct type ( c for call, j for jump, jc for both).

If the target is not a BTI instruction, the CPU raises a Branch Target Exception . In XNU, this manifests as a SIGILL with a specific subcode. For exploit development, this necessitates finding gadgets that not only perform the desired operation but are also preceded by a valid landing pad.

1.2.3 New in Tahoe: The Guarded Execution Feature (GXF)

This is the most significant architectural divergence in the Apple Silicon era. Standard ARM defines a vertical privilege stack (EL0 -> EL1 -> EL2). Apple has introduced Lateral Exception Levels , referred to as Guarded Levels (GL).

GXF allows the processor to switch between execution modes that share the same architectural Exception Level (EL1) but possess vastly different hardware permissions and system register views.

  • GL0: The standard XNU Kernel context.
  • GL1: The Trusted Execution Monitor (TXM) context.
  • GL2: The Secure Page Table Monitor (SPTM) context.

The Proprietary Opcodes:
Transitions are not handled by HVC or SMC . Apple added custom instructions to the ISA:

  • GENTER (Opcode 0x00201420 ): Synchronous entry into a Guarded Level. It behaves like a function call but atomically switches the hardware context (SPRR state, stack pointer, and system registers).
  • GEXIT (Opcode 0x00201400 ): Returns from GL to the caller.

Reverse Engineering the Transition:
In disassembly, GENTER takes a dispatch ID (selector) in x15 and arguments in x0-x7 . The entry point for the transition is hardcoded in the proprietary system register GXF_ENTRY_EL1 (S3_6_C15_C8_2). This register is locked early in the boot process (by iBoot), preventing the kernel from hijacking the monitor's entry vector.

1.2.4 New in Tahoe: Shadow Permission Remapping Registers (SPRR)

To enforce isolation between GL0, GL1, and GL2, Apple replaced the older APRR (Access Permission Remapping Registers) with the more robust SPRR (Shadow Permission Remapping Registers) .

In standard ARM MMUs, the Page Table Entry (PTE) bits AP[2:1] directly define Read/Write permissions. In Apple Silicon with SPRR enabled, these bits are repurposed as an index into a hardware permission table.

The Indirection Layer:

  1. PTE Index: The PTE specifies a permission index (e.g., Index 5).
  2. Context Lookup: The hardware checks the current execution mode (GL0, GL1, or GL2).
  3. Resolution: It looks up Index 5 in the SPRR_PERM_EL1 register specific to that mode.

The Security Implication:
This allows for "View-Based" memory protection.

  • Index 5 in GL2 (SPTM): Resolves to Read-Write (RW) .
  • Index 5 in GL0 (Kernel): Resolves to Read-Only (RO) .

This is how the SPTM protects page tables. The physical pages containing the translation tables are marked with a specific SPRR index. The hardware configuration for GL0 (Kernel) maps that index to Read-Only. Even if an attacker has a kernel-level arbitrary write primitive, the MMU will reject the write to the page table because the SPRR configuration for GL0 forbids it. The only way to write to that page is to execute GENTER to switch to GL2, where the SPRR configuration permits the write.

2.0 The Secure Enclave Processor (SEP): The Parallel Computer

If the Application Processor (AP) is the brain of the device, the Secure Enclave Processor (SEP) is its conscience. It is not merely a coprocessor; it is a fully independent computer-on-a-chip, sharing the same die but architecturally severed from the AP. It runs its own kernel (an Apple-customized L4 microkernel), manages its own peripherals, and holds the keys to the kingdom (UID/GID). In the Tahoe architecture, the SEP’s role has expanded from simple key management to becoming the root of authority for biometric intent and hardware attestation.

2.1 SEP Initialization & Boot

The SEP boot process is designed to be resilient against a fully compromised Application Processor. From the moment power is applied, the SEP assumes the AP is hostile.

2.1.1 The SEPROM: SRAM Execution and the Memory Protection Engine (MPE)

Like the AP, the SEP begins execution from an immutable on-die Boot ROM, the SEPROM .

The Hardware Environment:
The SEP core (typically a customized ARM core, historically evolving from Cortex-A7 derivatives) initializes in a highly constrained environment. It does not initially have access to the system's main DRAM. Instead, it executes strictly within a dedicated, on-die SRAM region. This isolation prevents early-boot DMA attacks from the AP or Thunderbolt peripherals.

The Memory Protection Engine (MPE):
As the sepOS is too large to fit entirely in SRAM, it must eventually reside in the device's main DRAM (Unified Memory on M-series). To do this securely, the SEP utilizes a hardware Memory Protection Engine (MPE) .

The MPE sits inline between the SEP core and the memory controller. It creates a cryptographic window into physical memory that is opaque to the rest of the SoC.

  1. Ephemeral Keys: Upon SEP reset, the MPE generates a random, ephemeral AES key. This key exists only in the MPE hardware registers and is never exposed to software (even sepOS ).
  2. AES-XEX Encryption: Data written by the SEP to DRAM is encrypted transparently using AES in XEX (XOR-Encrypt-XOR) mode.
  3. Authentication: The MPE calculates a CMAC tag for every cache line. This tag is stored alongside the encrypted data.

RE Implication: If you attempt to dump the physical memory range assigned to the SEP from the AP (kernel mode), you will see high-entropy noise. Furthermore, any attempt to modify a single bit of this memory via the AP will invalidate the CMAC tag. The next time the SEP reads that line, the MPE will detect the forgery and trigger a hardware panic, locking down the Enclave until a full system reset.

2.1.2 The Boot Monitor: Hardware Enforcement of OS-Bound Keys

On modern silicon (A13/M1 and later), Apple introduced the Secure Enclave Boot Monitor to mitigate the risk of Boot ROM exploits (like checkm8 ) compromising the chain of trust for key derivation.

In older architectures, the SEPROM would verify the sepOS signature and then jump to it. If the SEPROM was exploited, the attacker could jump to a malicious payload while retaining access to the hardware UID key. The Boot Monitor closes this gap by enforcing System Coprocessor Integrity Protection (SCIP) .

The Boot Flow:

  1. Payload Staging: The AP (iBoot) loads the sep-firmware.img4 payload into a region of physical memory.
  2. Mailbox Signal: The AP signals the SEP via a hardware mailbox register.
  3. Verification: The SEPROM parses the Image4 container. It verifies the signature against the Apple Root CA hash fused into the SEP's eFuses.
  4. The Handoff: Crucially, the SEPROM cannot simply jump to the loaded image. The SCIP hardware prevents execution of mutable memory.
  5. Monitor Intervention: The SEPROM invokes the Boot Monitor hardware block.
    • The Monitor resets the SEP core to a known clean state.
    • The Monitor calculates a cryptographic hash of the loaded sepOS memory range.
    • The Monitor updates the SCIP registers to permit execution of that specific range.
    • The Monitor locks the hash into a dedicated register in the Public Key Accelerator (PKA).

OS-Bound Key Derivation:
This finalized hash is the critical component. When the sepOS later requests keys (e.g., to decrypt user data), the hardware Key Derivation Function (KDF) mixes the hardware UID with this locked hash.

$$ K_{derived} = KDF(UID, Hash_{sepOS}) $$

If an attacker modifies a single byte of the sepOS (even with a Boot ROM exploit), the Boot Monitor calculates a different hash. Consequently, the KDF derives a different key, and the encrypted user data remains mathematically inaccessible. This is "Bound Security"—the data is bound not just to the device, but to a specific, signed software version.

2.1.3 Anti-Replay Mechanisms: The Integrity Tree

A classic attack vector against secure enclaves is the Replay Attack : capturing a snapshot of the encrypted RAM (e.g., when the passcode retry counter is 0) and restoring it later after the counter has incremented.

To prevent this, the SEP implements a hardware-enforced Integrity Tree (Merkle Tree).

  1. The Root of Trust: The root node of the integrity tree is stored in dedicated on-chip SRAM within the Secure Enclave complex. This memory is physically distinct from the main DRAM and cannot be addressed by the AP.
  2. Tree Structure: The protected memory region (where sepOS data and the Secure Storage Manager reside) is divided into blocks. Each block's hash is stored in a parent node, recursively up to the root.
  3. Atomic Updates: When the SEP writes to protected memory (e.g., incrementing a failed attempt counter), the MPE updates the data, recalculates the hashes up the tree, and atomically updates the root hash in the on-chip SRAM.
  4. Verification: On every read, the MPE verifies the path from the data block up to the SRAM root.

If an attacker replays an old DRAM state, the hash of the replayed block will not match the current root hash stored in the internal SRAM. The MPE detects the mismatch (Anti-Replay Violation) and halts the SEP. This mechanism ensures that the SEP has a strictly monotonic view of time and state, rendering snapshot fuzzing and counter rollbacks impossible.

2.2 SEP Runtime Architecture

Once the sepOS is bootstrapped and verified, the Secure Enclave transitions into its runtime state. At this point, it functions as a fully autonomous operating system running an Apple-customized variant of the L4 microkernel. For the reverse engineer, understanding the runtime architecture is crucial for analyzing how the SEP communicates with the hostile "Rich Execution Environment" (the AP running XNU) and how it persists sensitive state.

2.2.1 The Mailbox Interface: Analyzing the Shared Memory IPC

Communication between the Application Processor (AP) and the SEP is strictly asynchronous and interrupt-driven. Unlike the tight coupling of the SPTM (which uses synchronous instruction traps), the SEP interaction is mediated by a hardware mechanism known as the Mailbox , which relies on the proprietary Apple Interrupt Controller (AIC) to manage signaling.

The Physical Transport: AIC and Ring Buffers
There is no shared virtual memory space; the two processors exchange messages via physical memory ring buffers and hardware interrupts.

  1. The Ring Buffers (Inbox/Outbox):
    The system reserves two physical memory regions in DRAM, carved out by iBoot and described in the Device Tree (typically under the sep node as inbox-size and outbox-size ). These function as circular buffers.

    • Control Structures: Each buffer is managed by a pair of pointers (Write Head, Read Tail) stored in shared SRAM or at the start of the DRAM region.
    • Coherency: Because the SEP and AP share the same Unified Memory fabric, cache coherency is critical. The AppleSEPDriver must ensure appropriate barriers ( DMB , DSB ) are used when updating the Write Head to ensure the SEP sees the payload before the interrupt fires.
  2. The Doorbell (Apple Interrupt Controller):
    To signal a message, the sender cannot simply write to memory; it must trigger an exception on the receiver. On Apple Silicon, this is handled by the AIC , not a standard ARM GIC.

    • AP $\rightarrow$ SEP: The kernel writes to a specific AIC "Set" register (mapped in the AP's MMIO space). This asserts a hardware IRQ line wired to the SEP's core.
    • SEP $\rightarrow$ AP: When the SEP replies, it writes to its own AIC interface, which triggers a specific IRQ (often IRQ 0 or IRQ 1 relative to the SEP driver's view) on the AP. The kernel's interrupt handler ( AppleSEPDriver::interruptOccurred ) acknowledges this by writing to the AIC "Clear" register.

The L4 IPC Protocol (A7IOP):
The data payload within the ring buffers follows a serialized format, historically referred to as the Apple A7 I/O Protocol (A7IOP) , though it has evolved significantly.

  • Message Header: Every message begins with a 64-bit header containing:
    • Endpoint ID (8 bits): The destination service within the sepOS (e.g., 0x10 for Biometrics, 0x14 for Secure Storage).
    • Tag (8 bits): A transaction ID used to correlate asynchronous replies.
    • Length (16 bits): The size of the payload.
  • The AppleSEPDriver : On the XNU side, this kext manages the ring buffer logic. It serializes kernel requests (e.g., "Unlock Keybag") into the mailbox format, updates the Write Head, rings the AIC doorbell, and sleeps the calling thread on a condition variable until the SEP triggers the reply interrupt.

RE Focus: Fuzzing the Boundary
The mailbox is the primary attack surface for the SEP. Vulnerabilities here (parsing malformed messages) can lead to code execution within the Enclave.

  • Endpoint Fuzzing: The sepOS kernel dispatches messages to user-mode L4 tasks based on the Endpoint ID. Fuzzing specific endpoints (especially legacy or debug endpoints left enabled in production) is a standard methodology.
  • Shared Memory Hazards (DART): While the mailbox buffers are used for control messages (metadata), bulk data (like a firmware update or a large biometric template) is passed via DART-mapped shared memory . The AP maps a page, pins it, and passes the physical address to the SEP in a mailbox message. The SEP then maps this page into its own address space.
    • TOCTOU: Race conditions here are a classic target: modifying the data in the shared buffer after the SEP has validated the header/signature but before it processes the body. The SEP attempts to mitigate this by copying data to internal SRAM before processing, but large payloads may be processed in-place, exposing a window of opportunity.

2.2.2 The Secure Storage Component (xART): Encrypted Persistent Storage

The SEP has no internal non-volatile storage (NAND) of its own. It must rely on the AP's NAND flash to store persistent data (like the user's passcode hash, biometric templates, and credit card tokens). However, it cannot trust the AP to store this data securely.

To solve this, Apple introduced the Secure Storage Component , often referred to in firmware and kexts as xART (eXtended Anti-Replay Technology).

The Architecture:

  1. The Physical Chip: On modern devices, xART is backed by a dedicated, tamper-resistant EEPROM or NOR flash chip physically connected to the SEP via a private I2C or SPI bus. This chip is not accessible to the AP.
  2. The Logical Volume: The SEP treats the external NAND (managed by the AP) as a "dumb" block device. It encrypts its filesystem using keys derived from the hardware UID and the anti-replay counters stored in the dedicated xART chip.

The Anti-Replay Guarantee:
When the SEP writes a file (e.g., updating the failed passcode attempt counter):

  1. It encrypts the file data.
  2. It generates a new random nonce or increments a counter.
  3. It writes the encrypted file to the AP's filesystem (via the Mailbox).
  4. Crucially, it writes the hash of the file and the new counter to the dedicated xART storage chip.

When reading the file back:

  1. The SEP requests the file from the AP.
  2. It reads the expected hash/counter from the xART chip.
  3. It verifies the file against the xART record.

If the AP (or an attacker with physical access to the NAND) tries to revert the file to an older version (e.g., resetting the counter to 0), the hash will not match the record in the xART chip. The SEP will detect the rollback and reject the data. This mechanism ensures that the SEP's state is strictly monotonic and immune to external storage manipulation.

2.2.3 RE Focus: Reverse Engineering the sepOS L4 Syscall Table

For the advanced reverse engineer, the holy grail is understanding the sepOS kernel itself. Since it is based on L4, it relies heavily on synchronous IPC for system calls.

Identifying the Syscall Handler:
In the disassembled sepOS binary (which can be decrypted if you have a Boot ROM exploit for the specific device class, or by analyzing unencrypted development builds if available), the exception vector table is the starting point.

  • Look for the SVC (Supervisor Call) exception handler.
  • This handler typically branches to a dispatch table based on the immediate value in the SVC instruction or a register (often x0 or x8 ).

Mapping the Endpoints:
The sepOS is modular. It consists of the kernel and several user-mode "apps" or "tasks" running within the Enclave.

  • SEPOS Kernel: Handles memory management, scheduling, and IPC routing.
  • L4 Apps: Distinct binaries for distinct functions.
    • biometrickitd : Handles FaceID/TouchID processing.
    • securekeyvault : Manages the Keychain and Data Protection classes.
    • sigp : The Secure Enclave Signal Processor (for neural network operations).

By tracing the IPC messages dispatched from the Mailbox handler, you can map which L4 task handles which service. For example, a message with Endpoint ID 0x10 might route to the biometrickitd task. Analyzing the message parsing logic within that specific task reveals the proprietary protocol used for biometric authentication.

Tooling Note: Standard tools like IDA Pro or Ghidra require custom loaders for sepOS binaries. The memory layout is non-standard, and the binary format (Mach-O) often has stripped headers or non-standard segment protections that must be manually reconstructed based on the SCIP configuration found in the Boot Monitor logic.

3.0 The Chain of Trust: Firmware & Bootloaders

With the hardware root of trust established and the Secure Enclave operating as a parallel authority, the Application Processor begins the process of bootstrapping the mutable software stack. This phase is governed by the Image4 serialization format and a strict chain of cryptographic handover.

3.1 Low-Level Bootloader (LLB)

The Low-Level Bootloader (LLB) is the first piece of mutable code executed by the Application Processor. Loaded by the Boot ROM from the boot partition of the NAND (or NOR SPI on some development boards), it executes initially within the constraints of the SoC's SRAM. Its primary directive is architectural: it must bridge the gap between the raw silicon state and the feature-rich environment required by iBoot.

3.1.1 Parsing the Image4 ( img4 ) Container

To the reverse engineer, "firmware" on Apple Silicon is synonymous with Image4 . LLB is not a raw binary; it is encapsulated in an Image4 container, a format based on ASN.1 (Abstract Syntax Notation One) and DER (Distinguished Encoding Rules). Understanding this structure is prerequisite to any firmware analysis.

The Image4 container consists of three distinct sequences:

  1. IM4P (Payload): The actual executable code (the LLB binary).

    • Encryption: The payload is encrypted using AES-256. On production devices, this is wrapped with the GID Key . This means the payload is opaque to external analysis unless decrypted on-device (or via a GID oracle).
    • Compression: Once decrypted, the payload is typically compressed (LZSS or LZFSE).
    • Type Tag: A 4-character code (e.g., ibot , illb ) identifying the component.
  2. IM4M (Manifest): The signature and constraints, commonly known as the ApTicket .

    • The Signature: An RSA or ECDSA signature over the SHA-384 hash of the payload.
    • The Body: A set of entitlements and constraints (tags) that dictate where and how this payload can run.
    • Certificate Chain: The manifest includes the certificate chain leading back to the Apple Root CA (burned into the PKA fuses).
  3. IM4R (Restore Info): (Optional) Contains hardware-specific personalization data used during the restore process, such as the unique nonce generated by the SEP.

The Validation Logic:
When the Boot ROM loads LLB (and when LLB subsequently loads iBoot), it performs the following image4_validate routine:

  1. Parse the ASN.1 structure to separate IM4M and IM4P .
  2. Hash the IM4P (ciphertext).
  3. Locate the corresponding hash in the IM4M (under the specific tag, e.g., illb ).
  4. Verify the IM4M signature using the PKA.
  5. If valid, decrypt the IM4P using the AES engine (GID context).

3.1.2 DRAM Training and Memory Controller Configuration

The most critical hardware initialization task performed by LLB is DRAM Training .

When LLB starts, the system is running on internal SRAM (a few megabytes at most). The external LPDDR4X/LPDDR5 Unified Memory is inaccessible because the Memory Controller (MCU) is uninitialized. The physical characteristics of RAM—signal timing, voltage margins, and skew—vary slightly between every physical device due to manufacturing tolerances.

The Training Sequence:

  1. Reading SPD/Calibration Data: LLB reads calibration data from the device tree or dedicated EEPROM areas.
  2. PHY Configuration: It configures the Physical Layer (PHY) interface of the memory controller.
  3. Training Loop: LLB executes a complex algorithm that writes patterns to DRAM and reads them back, adjusting delay lines (DLLs) and drive strengths until the signal is stable.
  4. Remapping: Once training is complete, the MCU is brought online. LLB then reconfigures the Memory Management Unit (MMU) to map the vast expanse of DRAM into the address space.

RE Implication:
If you are attempting to exploit the Boot ROM or early LLB, you are constrained to SRAM. You cannot load large payloads or use heap spraying techniques that require gigabytes of memory until after LLB has successfully trained the DRAM. This creates a "choke point" for early-boot exploits.

3.1.3 Verifying the Exclusive Chip ID (ECID) and Board ID

Apple utilizes a mechanism called Personalization (or Taming) to prevent firmware replay attacks. You cannot simply take a valid, signed LLB from one iPhone and run it on another, nor can you downgrade to an older, vulnerable LLB version.

This enforcement happens inside the Image4 parser logic within LLB (checking the next stage) and the Boot ROM (checking LLB).

The Constraint Tags:
The IM4M manifest contains specific tags that bind the signature to the hardware:

  • ECID (Exclusive Chip ID): A 64-bit unique integer derived from the silicon die's coordinates on the wafer.
  • BORD (Board ID): Identifies the PCB model (e.g., 0x10 for a specific iPhone logic board).
  • CHIP (Chip ID): Identifies the SoC model (e.g., 0x8101 for M1).
  • SDOM (Security Domain): 0x1 for Production, 0x0 for Development.

The Check:
During boot, the executing code reads the actual values from the hardware fuses and compares them against the values present in the signed IM4M .

  • If Hardware.ECID != Manifest.ECID , the boot halts.
  • If Hardware.BORD != Manifest.BORD , the boot halts.

This mechanism, combined with the Nonce (a random value generated by the SEP during updates and baked into the IM4M ), ensures that the firmware is:

  1. Authentic: Signed by Apple.
  2. Targeted: Valid only for this specific device .
  3. Fresh: Valid only for this specific boot/update cycle (preventing downgrades).

Note: In the "Tahoe" architecture, this verification logic is hardened against fault injection. The comparison is often performed using redundant variables and bitwise checks that resist simple instruction skipping (e.g., glitching a B.NE instruction).

3.2 iBoot (Stage 2 Bootloader)

Once LLB has initialized the DRAM and verified the next stage, it hands off execution to iBoot . While LLB is a hardware-focused shim, iBoot is a sophisticated, single-threaded operating system in its own right. It contains a full USB stack, a display driver (for the Apple logo), a filesystem driver (APFS/HFS+), and the logic required to bootstrap the XNU kernel. In the Tahoe architecture, iBoot's role has expanded to become the orchestrator of the new security domains.

3.2.1 Device Tree (DT) Flattening

The hardware configuration of an Apple Silicon device is not discoverable via standard buses like PCI enumeration alone. Instead, iBoot relies on a Device Tree (DT) —a hierarchical data structure (similar to OpenFirmware or Linux FDT) that describes the SoC's topology.

The Source:
The raw Device Tree is embedded within the iBoot binary (or loaded as a separate devicetree.img4 payload). It contains nodes describing CPUs, memory maps, interrupt controllers (AIC), and peripherals.

The Flattening Process:
Before jumping to the kernel, iBoot "flattens" this tree into a binary format (FDT) and populates the /chosen node with runtime parameters.

  • kaslr-seed : A high-entropy random value generated by the TRNG. The kernel uses this to randomize its slide.
  • memory-map : A critical array of structures defining physical memory regions. iBoot marks regions used by the Boot ROM, LLB, and itself as reserved, ensuring the kernel does not overwrite them.
  • boot-args : The command-line arguments passed to the kernel (e.g., debug=0x14e , -v ). On production devices, these are strictly filtered; only specific flags are allowed unless the device is "demoted" or in a specific research state.

3.2.2 New in Tahoe: Loading the Security Monitors

In pre-Tahoe architectures (iOS 14 / macOS 11), iBoot would simply load the kernelcache and jump to it. In the Tahoe era (A15/M2+), iBoot must construct the Guarded Execution Environment before the kernel can exist.

Allocation and Reservation:
iBoot parses the device tree to identify physical memory ranges reserved for the new monitors. It carves these out of the available DRAM:

  • SPTM Region: Reserved for the Secure Page Table Monitor.
  • TXM Region: Reserved for the Trusted Execution Monitor.

Payload Loading:
iBoot locates the specific Image4 payloads within the kernelcache container:

  • Ap,SecurePageTableMonitor : The GL2 binary.
  • Ap,TrustedExecutionMonitor : The GL1 binary.

It decrypts and verifies these payloads just like any other firmware component. However, instead of loading them into standard memory, it loads them into the reserved physical regions identified above.

Locking SPRR Regions:
This is the critical security pivot. Before handing off control, iBoot configures the Shadow Permission Remapping Registers (SPRR) for the initial state.

  1. It configures the GL2 (SPTM) SPRR view to have Read/Write/Execute access to its own memory region.
  2. It configures the GL1 (TXM) SPRR view to have access to its region.
  3. Crucially, it configures the GL0 (Kernel) SPRR view to mark the SPTM and TXM regions as Inaccessible (No-Access) .

This ensures that when the processor eventually drops to EL1 (GL0) to run XNU, the kernel is physically incapable of reading or modifying the monitor code, even though it resides in the same physical DRAM.

3.2.3 LocalPolicy & BAA: The Shift to Local Signing

For macOS, Apple introduced a mechanism to allow users to boot older OS versions or custom kernels (Permissive Security) without breaking the hardware chain of trust. This is managed via LocalPolicy .

The Problem:
The Boot ROM and LLB enforce strict signature checks against Apple's global signing server (TSS). If you want to boot a custom kernel, you can't get a valid signature from Apple.

The Solution:

  1. LocalPolicy: A policy file stored on the Data Volume (in the iSCPreboot volume). It specifies the security mode (Full, Reduced, Permissive) and the hash of the custom kernel collection.
  2. BAA (Basic Attestation Authority): When a user authorizes a downgrade (via Recovery Mode authentication), the Secure Enclave generates a Local Signing Key .
  3. Re-Signing: The LocalPolicy is signed by this Local Key (inside the SEP).
  4. Boot Time: iBoot fetches the LocalPolicy. It asks the SEP to verify the signature. If the SEP confirms the policy is valid (and matches the user's intent), iBoot proceeds to load the custom kernel hash specified in the policy, effectively "blessing" it for this boot cycle.

This allows "Permissive Security" to exist while keeping the Boot ROM and LLB strictly locked down.

3.2.4 RE Focus: Decrypting iBoot Payloads via the AES MMIO Interface

To analyze iBoot, one must decrypt it. Since the GID key is fused into the silicon and physically disconnected from the CPU's register file, it cannot be extracted via software. Reverse engineers must instead turn the device into a Decryption Oracle by manipulating the dedicated AES hardware peripheral.

The kbag Mechanism:
The Image4 payload ( IM4P ) is encrypted with a random, per-file symmetric key (the target key). This target key is wrapped (encrypted) with the GID key and stored in the IM4P header as a Keybag ( kbag ) . To decrypt the firmware, one must unwrap this kbag.

The Hardware Distinction (ISA vs. MMIO):
It is critical to distinguish between the ARMv8 Crypto Extensions (instructions like AESE , AESD ) and the Apple AES Peripheral .

  • ARMv8 Crypto: Operates on keys loaded into standard NEON/SIMD registers ( v0 - v31 ). Useful for TLS or disk encryption where the key is known to the OS.
  • Apple AES Peripheral: A memory-mapped I/O (MMIO) block, typically located at a base offset like 0x23D2C0000 (on M1/T8103) or similar 0x2... ranges on newer SoCs. This peripheral has exclusive hardware access to the GID key fuses.

The Oracle Exploit:
Using a Boot ROM exploit (like checkm8 on A-series) or a specialized iBoot exploit, researchers execute a payload that drives this MMIO interface directly:

  1. Reset: Reset the AES peripheral via the AES_CTRL register to clear internal state.
  2. Key Selection: Write to the configuration register to select the GID Key as the decryption source. This sets an internal mux; the key itself is never exposed to the bus.
  3. FIFO Loading: Write the kbag (IV + Ciphertext) into the AES_DATA_IN FIFO registers.
  4. Execution: Trigger the engine. The hardware pulls the GID key from the fuses, performs the AES-256-CBC unwrap, and pushes the result to the output buffer.
  5. Extraction: Read the unwrapped target key (typically formatted as iv:key ) from the AES_DATA_OUT register.

Bit-Flipping and Timing Countermeasures:
Modern Apple Silicon (A12+/M1+) implements countermeasures against this oracle usage. The AES engine may enforce a state machine that requires the output of a GID decryption to be immediately DMA'd to executable memory and jumped to, rather than read back into a general-purpose register. Bypassing this often requires Fault Injection (voltage glitching) to corrupt the state machine or precise timing attacks to race the hardware's "sanitize on read" logic, allowing the extraction of the plaintext key before the hardware scrubs it.

4.0 The Security Monitor Layer (GL1/GL2): The New Hypervisor

In the "Tahoe" architecture, the XNU kernel has been demoted. It no longer possesses the ultimate authority to define the virtual memory layout of the system. That power has been migrated to a hardware-enforced monitor running in a proprietary execution state. This section dissects the mechanics of this new layer, which effectively functions as a lightweight, silicon-enforced hypervisor for the kernel itself.

4.1 The Secure Page Table Monitor (SPTM) - GL2

The Secure Page Table Monitor (SPTM) operates at Guarded Level 2 (GL2) . While architecturally sharing the EL1 exception level with the kernel, the Guarded Execution Feature (GXF) ensures that GL2 has a strict superset of permissions compared to the kernel's GL0. The SPTM is the sole entity permitted to write to the physical pages that constitute the translation tables (TTBR0/TTBR1).

4.1.1 The GENTER and GEXIT Instructions: Context Switching

Transitions into the SPTM are not handled by standard ARM exception vectors ( VBAR_EL1 ). Instead, they utilize the proprietary GENTER instruction, which performs a synchronous, atomic context switch.

The GENTER ABI:
To invoke the SPTM, the kernel populates specific registers and executes the opcode.

  • Opcode: 0x00201420 (Little Endian).
  • x15 (Selector): The Dispatch ID. This integer identifies which SPTM primitive to execute.
  • x0 - x7 (Arguments): The parameters for the call (e.g., physical addresses, permission flags).
  • x16 / x17 : Often used as scratch or secondary dispatch modifiers in newer revisions.

The Hardware Transition:
Upon execution of GENTER :

  1. SPRR Switch: The hardware swaps the active Shadow Permission Remapping Register configuration. The memory regions containing the SPTM code and data—previously invisible to the kernel—become Read/Write/Execute. Conversely, the kernel's own text might become Read-Only or Non-Executable depending on the monitor's logic.
  2. Stack Switch: The Stack Pointer ( SP ) is switched to the SP_GL2 register, pointing to a dedicated secure stack within the SPTM's private memory.
  3. PC Jump: Execution jumps to the vector defined in GXF_ENTRY_EL1 .

The Return:
The SPTM returns control to the kernel using GEXIT ( 0x00201400 ). This restores the GL0 SPRR configuration and the kernel's stack pointer. Crucially, GEXIT clears sensitive registers to prevent data leakage from the secure context.

4.1.2 The Frame Table (FTE): Tracking Physical Reality

To enforce security, the SPTM cannot rely on the kernel's data structures (like vm_page_t ), as they are mutable by a compromised kernel. Instead, the SPTM maintains its own "God View" of physical memory called the Frame Table .

The Frame Table is a linear array of Frame Table Entries (FTE) , located in SPTM-private memory. There is one FTE for every 16KB page of physical RAM.

FTE Structure (Reconstructed):
While the exact struct evolves, it generally contains:

  • Type (Bitfield): The strict classification of the page.
    • XNU_DATA : Generic kernel heap/stack.
    • XNU_TEXT : Immutable kernel code.
    • PAGE_TABLE : A page containing translation entries (TTEs).
    • USER_DATA : Memory mapped to EL0 processes.
    • SPTM_PRIVATE : Internal monitor structures.
  • Refcount : Tracks how many virtual mappings point to this physical page.
  • OwnerID : Identifies the Trust Domain (e.g., Kernel vs. Exclave).

The Security Invariant:
The SPTM enforces that a physical page can only be mapped into a virtual address space if the mapping permissions are compatible with the page's Type . For example, a page marked XNU_DATA in the FTE cannot be mapped as Executable. A page marked PAGE_TABLE cannot be mapped as Writable by the kernel.

4.1.3 The Dispatch Table: Reverse Engineering the Selectors

The interface between XNU and the SPTM is a strict, register-based API. However, unlike the stable syscall numbers of the BSD layer, the SPTM Dispatch IDs (Selectors) are not guaranteed to remain static across macOS versions. Apple frequently rotates these IDs between major (and occasionally minor) releases to frustrate static analysis tools and exploit chains that rely on hardcoded offsets.

The ABI:

  • x15 : The Dispatch ID (Selector).
  • x0 - x7 : Arguments (Physical Addresses, Permission Bitmasks, ASIDs).
  • x16 / x17 : Scratch registers, occasionally used for secondary modifiers or sub-ops.

Heuristic Identification:
Since relying on static IDs (e.g., 0x00 ) is brittle, reverse engineers must fingerprint the logic of the handler functions within the Ap,SecurePageTableMonitor binary to identify the primitives.

  • sptm_retype(ppn, old_type, new_type) (Often ID 0x00 or 0x01 ):

    • Fingerprint: Look for a function that accepts a Physical Page Number (PPN), reads the corresponding Frame Table Entry (FTE), and performs a Sanitization Loop . The SPTM must zero-fill ( bzero ) or cache-invalidate the page before transitioning it from XNU_DATA to PAGE_TABLE to prevent the kernel from initializing a page table with pre-computed malicious entries.
    • Logic: assert(refcount == 0); memset(pa, 0, PAGE_SIZE); fte->type = new_type;
  • sptm_map(asid, va, ppn, perms) :

    • Fingerprint: Look for a function that walks the translation tables (reading physical memory) and performs a Permission Check against the FTE. It will contain logic that explicitly compares the requested perms (e.g., Write) against the fte->type (e.g., XNU_TEXT ).
    • Logic: if (fte->type == XNU_TEXT && (perms & WRITE)) panic(); write_tte(...);
  • sptm_unmap(asid, va) :

    • Fingerprint: Look for the TLB Invalidation sequence. After clearing a TTE, the SPTM must execute TLBI instructions (e.g., TLBI ASIDE1IS ) to ensure the hardware translation lookaside buffer is coherent. The kernel is forbidden from executing TLBI instructions that affect the secure context; it must delegate this to the SPTM.
  • sptm_map_iommu(dart_id, context_id, dva, ppn, perms) :

    • Fingerprint: Look for writes to MMIO regions associated with DART controllers, rather than standard RAM. This function validates that the ppn is not a protected kernel page before mapping it into a device's IOVA space.

RE Implication:
Automated analysis scripts should not rely on CMP x15, #0x1 . Instead, they should symbolically execute the GENTER handler in the SPTM binary, identifying the dispatch table jump, and then classify the target functions based on the presence of DC ZVA (cache zero), TLBI , or FTE array access patterns.

4.1.4 RE Focus: Analyzing Panic Strings and the State Machine

The SPTM is designed to be Fail-Secure . Unlike standard kernel APIs that return KERN_FAILURE , the SPTM treats invalid requests as evidence of kernel compromise.

The Panic Mechanism:
If XNU sends a malformed request (e.g., trying to retype a page that is still mapped), the SPTM does not return. It triggers a system-wide panic.

  • Panic String: "received fatal error for a selector from TXM" or "invalid state transition" .
  • Analysis: These strings are gold for reverse engineers. They confirm that the SPTM enforces a strict Finite State Machine (FSM) for memory pages.

Mapping the State Machine:
By analyzing the panic logic, we can deduce the allowed transitions:

  1. FREE $\rightarrow$ XNU_DATA (Allocation)
  2. XNU_DATA $\rightarrow$ PAGE_TABLE (Retype for MMU use - requires sanitization)
  3. PAGE_TABLE $\rightarrow$ XNU_DATA (Teardown - requires unmapping all entries)
  4. XNU_DATA $\rightarrow$ XNU_TEXT (KEXT loading - One-way transition!)

Any attempt to deviate from this graph (e.g., trying to turn PAGE_TABLE directly into XNU_TEXT ) results in an immediate halt. This prevents "Page Table Spraying" and other heap manipulation techniques used to gain kernel execution.

4.2 The Trusted Execution Monitor (TXM) - GL1

If the SPTM is the brawn—enforcing the physics of memory mapping—the Trusted Execution Monitor (TXM) is the brains. Operating at Guarded Level 1 (GL1) , the TXM is the supreme arbiter of system policy. It represents the architectural decoupling of "mechanism" from "policy." While the SPTM handles how a page is mapped, the TXM decides if it is allowed to be mapped executable.

4.2.1 Decoupling AMFI: Moving Code Signature Verification

Historically, the Apple Mobile File Integrity (AMFI) kernel extension was the primary enforcement point for code signing. However, as a KEXT running in EL1, AMFI was susceptible to kernel-level memory corruption. Exploits like tfp0 could patch amfi_get_out_of_my_way or swizzle the MAC (Mandatory Access Control) hooks to bypass signature checks.

In the Tahoe architecture, the core verification logic has been lifted out of the kernel and placed into the TXM. Crucially, the TXM does not necessarily parse the full CMS blob on every request; its primary currency of trust is the Code Directory Hash (CDHash) .

The Verification Flow:

  1. Load & Hash: The kernel (XNU) loads a binary into memory (typed as XNU_DATA ). It parses the LC_CODE_SIGNATURE load command and calculates the CDHash (SHA-256) of the Code Directory.
  2. The Query: XNU issues a GENTER call to the TXM. It passes the CDHash and the physical address of the memory range.
  3. Trust Cache Lookup: The TXM first checks its internal Trust Caches (located in GL1 memory).
    • Static Trust Cache: Contains CDHashes for all immutable OS binaries (from the Cryptex).
    • Dynamic Trust Cache: Contains CDHashes for recently verified third-party apps.
  4. Cold Validation: If the CDHash is not found (a cache miss), the system enters a "Cold Start" validation path. The kernel (often aided by amfid in userland for complex policy checks) must provide the CMS signature blob to the TXM. The TXM performs the cryptographic verification of the blob against the Apple Root CA (or Developer ID Root) within the secure world. If valid, the CDHash is promoted to the Dynamic Trust Cache.
  5. Bless: Once the CDHash is validated (either via Cache Hit or Cold Validation), the TXM updates its internal state to "bless" the specific physical pages associated with that CDHash.
  6. Enforcement (GL2): When XNU subsequently asks the SPTM to map those pages as Executable ( RX ), the SPTM queries the TXM: "Is this physical page verified?" If the TXM has not blessed the CDHash associated with those pages, the SPTM denies the Execute permission.

RE Implication: Patching the kernel to ignore signature errors is now futile. Even if XNU "thinks" a binary is signed and attempts to set the executable bit in the PTE, it lacks the hardware authority to do so. The SPTM will simply refuse the PTE update because the TXM never validated the CDHash.

4.2.2 The Trust Cache: Static vs. Dynamic

To avoid the performance penalty of cryptographic verification on every page fault, the TXM manages the Trust Cache —a database of known-good CDHashes.

  • The Static Trust Cache:
    This is loaded by iBoot and passed to the TXM during the GL1 initialization. It contains the hashes of every binary in the OS (now encapsulated in the immutable Cryptexes ). This cache resides in GL1 memory and is strictly Read-Only.

  • The Dynamic Trust Cache:
    This handles third-party applications and JIT regions. When a user launches an app, the TXM verifies the signature once and adds the CDHash to the Dynamic Trust Cache.

    • Query Interface: The kernel queries the Trust Cache via a specific GENTER selector.
    • Attack Surface: The Dynamic Trust Cache is a mutable structure in GL1. A logic bug in the TXM's management of this cache (e.g., a race condition during entry removal or a hash collision attack) is a high-value target for persistence.

4.2.3 Developer Mode Enforcement and Downgrade Protection

The TXM is also the guardian of the device's security posture, specifically Developer Mode .

In previous iterations, enabling debugging capabilities was often a matter of setting nvram variables or boot-args (like cs_enforcement_disable=1 ). In Tahoe, these states are managed by the TXM.

The State Transition:
Enabling Developer Mode requires a reboot and explicit user authorization (Secure Intent via physical buttons). The TXM persists this state (likely via the Secure Enclave's secure storage).

Downgrade Protection:
The TXM enforces that the system cannot transition from a "Production" state to a "Developer" state without a full reboot and authentication ceremony. This prevents a kernel-level attacker from dynamically relaxing security policies to load unsigned modules.

Furthermore, the TXM validates the LocalPolicy (signed by the SEP) to determine if the system is booting in "Permissive Security" mode. If the LocalPolicy signature is invalid or missing, the TXM defaults to "Full Security," rejecting any code not signed by the Apple Root CA, regardless of what the kernel requests.

5.0 XNU Kernel Initialization: Entering EL1

The handoff from iBoot to the XNU kernel marks the transition from a single-threaded bootloader to a symmetric multiprocessing (SMP) operating system. However, in the Tahoe architecture, this is no longer a handover of absolute power. The kernel enters Exception Level 1 (EL1) not as a master, but as a client of the Guarded Level 2 (GL2) monitor.

The entry point is defined in osfmk/arm64/start.s . At this precise moment, the system state is fragile: the MMU is likely disabled (or running on an identity map provided by iBoot), interrupts are masked ( DAIF bits set), and the stack pointer is essentially arbitrary. The kernel's first objective is to orient itself within physical memory, calculate the KASLR slide, and establish the virtual memory structures required to turn on the lights.

5.1 The start routine and KASLR

The _start symbol is the architectural entry point. Unlike x86_64, where the kernel might handle its own decompression and relocation, the Apple Silicon kernel is loaded as a raw Mach-O executable (within the kernelcache container) directly into physical memory by iBoot.

The Register State at Entry:

  • x0 : Physical address of the boot_args structure (version 2).
  • x1 : Physical address of the Device Tree base (if not inside boot_args ).
  • x2 : 0 (Reserved).
  • x3 : 0 (Reserved).
  • sp : Invalid/Temporary.

5.1.1 Deriving the Kernel Slide: The Dual-Slide Paradigm

Kernel Address Space Layout Randomization (KASLR) on Apple Silicon is a cooperative effort between iBoot and XNU. Unlike x86_64, where the kernel might handle its own relocation, the Apple Silicon kernel is loaded as a raw Mach-O executable directly into physical memory by iBoot. However, in the Tahoe architecture, KASLR has evolved from a simple obfuscation technique into a compartmentalized security boundary.

The boot_args Structure:
Upon entry at _start , the kernel immediately parses the boot_args structure (version 2) pointed to by x0 . This structure acts as the handover manifest, containing the virtBase (the static link address, typically 0xFFFFFFF007004000 ) and the physBase (the actual physical load address in DRAM).

The Slide Calculation:
The slide is not generated by the kernel at runtime; it is consumed. iBoot generates a high-entropy value from the TRNG, populates the /chosen/kaslr-seed property in the Device Tree, and physically relocates the kernel text in DRAM to match this slide.

The kernel calculates its own slide using the delta between the compile-time base and the runtime physical base (adjusted for the static virtual-to-physical offset):

$$ \text{vm_kernel_slide} = \text{boot_args.virtBase} - \text{CompileTimeBase} $$

The Tahoe Constraint: Entropy Decorrelation:
In the Tahoe architecture, the system operates under a Dual-Slide Paradigm . The SPTM (GL2) and the Kernel (GL0/EL1) reside in the same physical DRAM but operate in distinct translation regimes.

  1. Kernel Slide: Randomized by iBoot based on kaslr-seed .
  2. SPTM Slide: Randomized by iBoot based on a separate, decorrelated entropy source (or a cryptographic derivation of the master seed that is not exposed to EL1).

Security Implication:
This decorrelation is critical. A kernel-level memory leak (e.g., an infoleak revealing a kernel pointer) allows an attacker to calculate vm_kernel_slide . In previous architectures, if the monitor (PPL) was mapped at a fixed offset relative to the kernel, a kernel leak would instantly reveal the monitor's location.

In Tahoe, knowing vm_kernel_slide yields zero information about the virtual address of the SPTM. The SPTM's virtual mapping is established by iBoot in the GL2 translation tables ( TTBR1_EL2 context) before the kernel executes. The kernel is aware of the SPTM's physical pages (marked as "Reserved" in the memory map to prevent the VM subsystem from overwriting them), but it is architecturally blind to the SPTM's virtual location.

RE Focus: Finding the Slide:
For a reverse engineer with a kernel panic log or a JTAG connection, identifying these slides requires inspecting distinct registers:

  • Kernel Slide: Inspect TTBR1_EL1 . The translation table base points to the physical location of the kernel's L1 table. The high bits of the PC (Program Counter) at the exception vector reveal the virtual slide.
  • SPTM Slide: This is invisible from EL1. To find it, one must inspect TTBR1_EL2 (or the proprietary system register aliasing the GL2 translation base) via JTAG while the core is halted in the GL2 context.
  • Static Analysis: The vm_kernel_slide global variable in XNU is one of the first initialized. In a raw memory dump, locating the boot_args struct (often at the start of a physical page aligned to 16KB) will reveal the virtBase directly.

5.1.2 Initializing the MMU: TCR_EL1 and the SPTM Handshake

Before the kernel can execute C code safely, it must enable the Memory Management Unit (MMU). On standard ARMv8, this involves populating translation tables and writing to TTBR0_EL1 (User) and TTBR1_EL1 (Kernel), then setting SCTLR_EL1.M .

On Tahoe, this process is fundamentally altered because the kernel cannot write to its own page tables.

The Bootstrap Problem:
How does the kernel build its initial page tables if it requires the SPTM to map pages, but the SPTM requires the kernel to make hypercalls?

The Solution: The Bootstrap Tables:
iBoot constructs a set of initial "Bootstrap Page Tables" before handing off control. These tables are identity-mapped (Physical == Virtual) for the PC and stack, but also contain the kernel's high-virtual mappings.

  1. TCR_EL1 Setup: The kernel configures the Translation Control Register ( TCR_EL1 ) .
    • T1SZ / T0SZ : Defines the size of the virtual address space (typically 36-bit or 39-bit on iOS, 48-bit on macOS).
    • TG1 : Granule size (16KB is standard for Apple Silicon, unlike the 4KB standard of Android/Linux).
    • IPS : Intermediate Physical Address Size (matches the SoC capability, e.g., 40 bits).
    • TBI1 (Top Byte Ignore): Critical for PAC. This bit must be set to 1. It tells the MMU to ignore the top byte (bits 63-56) during address translation, allowing PAC signatures to exist in valid pointers without causing translation faults.

The SPTM Handshake (The First GENTER ):
Once TCR_EL1 is configured, the kernel must transition from the iBoot-provided bootstrap tables to its own managed tables.

  1. Allocation: The kernel allocates physical pages for the new L1/L2/L3 translation tables from the XNU_DATA pool.
  2. Sanitization: The kernel zeroes these pages.
  3. Retype: The kernel executes GENTER (Selector 0x00 - sptm_retype ) to convert these pages from XNU_DATA to PAGE_TABLE .
  4. Mapping: The kernel executes GENTER (Selector 0x01 - sptm_map ) to populate the entries, replicating the kernel text and static data mappings.
  5. Activation: Finally, the kernel writes the physical address of the new L1 table to TTBR1_EL1 .

The SCTLR_EL1 Lockdown:
The final step of start is writing to the System Control Register ( SCTLR_EL1 ) .

  • M (MMU Enable): Set to 1.
  • C (Cache Enable): Set to 1.
  • WXN (Write-XOR-Execute): Set to 1.

In Tahoe, the SPTM monitors writes to SCTLR_EL1 . If the kernel attempts to disable WXN (a common technique in older jailbreaks to patch kernel text), the SPTM intercepts the system register write and panics the device. The hardware configuration enforced by GL2 ensures that SCTLR_EL1 modifications are privileged operations that must comply with the system's security policy.

Once the MMU is active and the kernel is running on its own page tables (managed by SPTM), the start routine branches to arm_init , beginning the high-level initialization of the BSD subsystem and IOKit.

5.2 Hardware Security Enforcements (The "Kill Switch" Registers)

As the kernel initialization sequence progresses through start , it reaches a critical inflection point. The memory management structures are initialized, and the kernel is about to transition from a setup phase to a runtime phase. To prevent a compromised runtime kernel from modifying its own logic, the initialization routine must engage the hardware "Kill Switches."

These are proprietary system registers that, once written to, become immutable until a full system reset. In the pre-Tahoe era, these registers were the primary defense against persistent kernel compromises. In the Tahoe architecture, they serve as a hardware-enforced baseline that the SPTM relies upon to maintain the invariant of Kernel Text Immutability .

5.2.1 KTRR (Kernel Text Read-Only Region): The Physical Lock

Kernel Text Read-Only Region (KTRR) is Apple’s hardware solution to the "W^X" (Write XOR Execute) problem at the physical memory controller level. While the MMU (via page tables) controls virtual access permissions, page tables are mutable data structures. If an attacker gains arbitrary read/write (KRW) access to the kernel, they could theoretically modify the page tables to make the kernel text writable.

KTRR bypasses the MMU entirely. It enforces permissions based on Physical Addresses (PA) directly within the Memory Controller (MCU).

The Register Interface:
KTRR is controlled via a set of proprietary system registers, typically accessible via MSR instructions at EL1 (or GL2 in Tahoe).

  • KTRR_LOWER_EL1 ( S3_4_c15_c2_3 ): Defines the physical start address of the protected range.
  • KTRR_UPPER_EL1 ( S3_4_c15_c2_4 ): Defines the physical end address.
  • KTRR_LOCK_EL1 ( S3_4_c15_c2_2 ): The kill switch. Writing 1 to the lock bit enables the protection.

The "RoR" (Read-only Region) Mechanism:
Once the lock bit is set:

  1. Write Protection: Any write transaction (store instruction or DMA) targeting a physical address within the [LOWER, UPPER] range is dropped by the memory controller. Depending on the SoC generation, this either fails silently or triggers a synchronous external abort (SError).
  2. Execute Protection: The memory controller ensures that instruction fetches are permitted from this region.
  3. Immutability: The KTRR_LOCK_EL1 register itself becomes Read-Only. It cannot be unlocked by software.

The Tahoe Evolution (Virtualization of KTRR):
On M3/M4 chips running the SPTM, the kernel's interaction with KTRR changes. Since the kernel (EL1) is deprivileged, it cannot be trusted to set up KTRR correctly (it might set the range to zero, leaving itself unprotected).

  • SPTM Enforcement: The SPTM configures the physical KTRR registers during its own initialization (in Ap,SecurePageTableMonitor ).
  • Virtualization: When the XNU kernel executes the legacy instructions to write to KTRR_LOWER/UPPER in start , the hardware traps these accesses to GL2. The SPTM validates that the kernel is attempting to cover the correct physical range (matching the XNU_TEXT entries in the Frame Table) and effectively "mocks" the success of the operation to the kernel, while ensuring the hardware is actually locked down according to the SPTM's policy.

RE Focus: The KTRR Slide Alignment
Because KTRR operates on physical ranges, it lacks the granularity of 4KB/16KB pages. It typically operates on larger blocks (e.g., 1MB or L2 cache line boundaries). This forces the KASLR slide to be aligned to the KTRR granularity. If you are brute-forcing the KASLR slide, knowing the KTRR alignment constraint significantly reduces the entropy search space.

5.2.2 Kernel Integrity Protection (KIP): Extending the Shield

KTRR protects the static kernel binary ( kernelcache ). However, modern macOS relies heavily on the Boot Kernel Collection (BKC) and Auxiliary Kernel Collection (AKC) —large caches of drivers and extensions loaded during boot. These reside in memory adjacent to the kernel but are technically distinct payloads.

Kernel Integrity Protection (KIP) is the architectural evolution designed to protect these dynamic-but-immutable regions.

The Mechanism:
Unlike KTRR, which typically defines a single contiguous range, KIP (often implemented via the GXF or APRR logic on newer chips) allows for a more flexible definition of immutable regions.

  1. Registration: During the start routine, the kernel iterates over the loaded extensions (IOKit drivers).
  2. Sealing: Once the extensions are linked and relocated, the kernel issues a call to "seal" the region. In Tahoe, this is a GENTER call to the SPTM (Selector sptm_retype or sptm_protect ).
  3. Retyping: The SPTM updates the Frame Table Entries (FTE) for the physical pages backing the drivers. It transitions them from XNU_DATA (Writable) to XNU_TEXT (Executable/Read-Only).

The "One-Way" Door:
The security invariant enforced here is that memory typed as XNU_TEXT is never writable by EL1.

  • If the kernel attempts to write to a KIP-protected page, the SPRR configuration for GL0 (Kernel) triggers a permission fault.
  • If the kernel attempts to ask the SPTM to map it as writable, the SPTM checks the FTE, sees XNU_TEXT , and panics.

This effectively turns the kernel extensions into ROM. This mitigates a massive class of rootkits that historically operated by patching IOKit vtables or function pointers in memory.

5.2.3 The System Control Register ( SCTLR_EL1 ) Lockdown

The final "Kill Switch" is the configuration of the ARM processor itself. The SCTLR_EL1 register controls the MMU, caches, and alignment checks.

Critical Bits:

  • WXN (Write-XOR-Execute): Bit 19. When set, any memory region mapped as Writable is implicitly treated as Non-Executable ( XN ). This prevents the execution of shellcode on the heap or stack.
  • M (MMU Enable): Bit 0.
  • A (Alignment Check): Bit 1.

The Trap-and-Emulate Trap:
In a standard ARM system, EL1 can modify SCTLR_EL1 at will. An attacker with code execution could simply write to SCTLR_EL1 to disable WXN , map their shellcode, and execute it.

In the Tahoe architecture, SCTLR_EL1 is a trapped resource.

  1. The Trap: The hardware is configured (via HCR_EL2 or proprietary GXF controls) to trap writes to SCTLR_EL1 to the monitor layer (GL2/SPTM).
  2. The Policy: When the kernel executes MSR SCTLR_EL1, x0 , the CPU switches to the SPTM.
  3. The Check: The SPTM inspects the value in x0 . It enforces a strict policy: WXN must always be 1.
  4. The Verdict: If the kernel attempts to clear WXN , the SPTM denies the write and panics the system with a "Security Violation" code.

This ensures that the fundamental security properties of the execution environment (W^X) cannot be disabled, even by the kernel itself. The kernel is permanently locked into a secure configuration from the moment start completes.

5.3 Exclaves: The Microkernel within the Monolith

The introduction of Exclaves in the Tahoe architecture represents the most profound structural change to the Apple OS ecosystem since the transition from Mac OS 9 to OS X. It is an admission that the monolithic kernel architecture (XNU) has become too large, too complex, and too mutable to serve as the ultimate Trusted Computing Base (TCB) for high-value assets.

Exclaves introduce a Microkernel architecture running side-by-side with the monolithic XNU kernel on the same Application Processor cores. Unlike the Secure Enclave (which is a separate coprocessor with limited compute power), Exclaves harness the full performance of the M-series P-cores and E-cores while maintaining cryptographic isolation enforced by the SPTM.

5.3.1 The L4 Influence: Domains, Conclaves, and IPC

The architecture of the Exclave system is heavily indebted to the L4 microkernel family (specifically seL4). It prioritizes minimalism, capability-based security, and strict isolation.

The Hierarchy of Isolation:

  1. The Secure Kernel ( ExclaveOS ): A tiny, formally verifiable kernel that manages scheduling and IPC within the secure world. It runs at a privilege level guarded by the SPTM (likely GL1 or a restricted GL2 domain).
  2. Domains: The highest level of separation. The "Insecure Domain" hosts XNU and userland. The "Secure Domain" hosts Exclave workloads.
  3. Conclaves: Within the Secure Domain, workloads are siloed into Conclaves . A Conclave is a lightweight container consisting of an address space, a set of capabilities (handles to resources), and threads.
    • Example: The FaceID Conclave cannot access the memory of the Wallet Conclave, even though both run within the Exclave environment.

Memory Management via SPTM:
The isolation is enforced by the SPTM's Frame Table. Physical pages assigned to an Exclave are typed in the FTE (likely as EXCLAVE_DATA or SK_DATA ).

  • XNU View: The kernel sees these physical pages as "reserved" or "stolen" in the device tree. Any attempt by XNU to map these pages via sptm_map will result in a panic, as the SPTM forbids mapping Exclave-owned pages into the XNU_DOMAIN .
  • Exclave View: The ExclaveOS requests mappings from the SPTM. The SPTM updates the translation tables specific to the Exclave context (a distinct TTBR or context ID).

5.3.2 RE Focus: The RingGate Mechanism and XNUProxy

For the reverse engineer, the critical question is: How does the Kernel talk to an Exclave? They share no virtual memory, run in different hardware contexts, and the SPTM actively prevents XNU from mapping Exclave physical pages. The bridge is a mechanism internally referred to as RingGate , facilitated by a kernel extension named XNUProxy ( com.apple.driver.XNUProxy ).

The Communication Stack:

  1. Tightbeam (The IDL):
    Apple has introduced a new Interface Definition Language (IDL) called Tightbeam . It replaces the legacy Mach Interface Generator (MIG) for secure world communication. Unlike MIG, which is loosely typed and message-centric, Tightbeam is strongly typed and buffer-centric, designed to minimize parsing ambiguity.

    • Userland Analysis: The serialization logic is visible in /usr/lib/libTightbeam.dylib . Analyzing this library reveals the wire format: a header containing a protocol version and message ID, followed by a packed struct of arguments. Symbols like tb_message_encode , tb_client_call , and tb_endpoint_create are the primary indicators of this traffic.
    • Kernel Analysis: In XNUProxy , look for the TBTransport C++ classes that wrap the raw ring buffer management.
  2. The Downcall (XNU $\rightarrow$ Exclave):
    When XNU needs a service (e.g., "Authenticate this Passkey"), it cannot call the function directly.

    • Marshaling: XNUProxy serializes the request using Tightbeam into a shared memory ring buffer.
    • The Gate: The kernel executes a specific instruction to trigger the world switch. This is a GENTER instruction targeting a specific Dispatch ID reserved for the Secure Kernel (distinct from the SPTM/TXM selectors).
    • Context Switch: The hardware (mediated by SPTM) saves the EL1 state, switches the SPRR configuration to the Exclave view, and jumps to the ExclaveOS entry point.
    • Dispatch: ExclaveOS inspects the ring buffer and routes the message to the appropriate Conclave's IPC port.
  3. The Upcall (Exclave $\rightarrow$ XNU):
    Exclaves are not completely autonomous; they often rely on XNU for file system I/O or networking (since writing a full network stack in a microkernel is impractical).

    • The Exclave writes a request to the outbound ring buffer.
    • It triggers an interrupt or executes a GEXIT yield.
    • XNUProxy receives the notification, reads the request (e.g., "Write this blob to disk"), performs the operation via standard VFS calls, and returns the result via a Downcall.

Analyzing the XNUProxy Kext:
This kext is the Rosetta Stone for Exclave interaction. It maintains the mapping between Mach Ports (in XNU) and Tightbeam Endpoints (in Exclaves).

  • Endpoint Mapping: Reverse engineering the XNUProxy binary reveals structures that map 64-bit integer IDs to specific Exclave services. It effectively acts as a NAT (Network Address Translation) layer between Mach Ports and Exclave Endpoints.
  • Memory Loaning (The "DART" Window): While control messages go through ring buffers, large data transfers (like camera frames or biometric buffers) occur via Memory Loaning .
    • XNUProxy pins a userland page and retrieves its physical address.
    • It passes this Physical Page Number (PPN) to the Exclave via Tightbeam.
    • The Exclave requests the SPTM to map this specific PPN into its address space.
    • Vulnerability Class: This "Loaned Memory" is a prime target for TOCTOU (Time-of-Check to Time-of-Use) attacks. If the kernel (or a malicious user thread) modifies the content of the physical page after the Exclave has validated the header but before it processes the body, the Exclave's parser could be compromised. The SPTM likely enforces "Shared" vs "Exclusive" ownership states in the Frame Table to mitigate this, but the logic is complex and fragile.

5.3.3 Use Case: Secure Control of Privacy Indicators and Passkeys

The "Killer App" for Exclaves in macOS Tahoe is the hardware-enforced privacy indicator (the green/orange dots).

The Pre-Tahoe Vulnerability:
In previous macOS versions, the "Green Dot" (camera active) was drawn by WindowServer or the kernel. If an attacker achieved kernel-level code execution (Ring 0), they could hook the display driver and suppress the dot while recording the user.

The Tahoe Solution:

  1. Hardware Ownership: The physical frame buffer region corresponding to the status bar indicators is not mapped in the XNU domain. It is owned exclusively by a specific Privacy Conclave .
  2. The DART Lock: The Display Coprocessor's IOMMU (DART) is configured by the SPTM such that the main display pipe cannot write to the indicator pixels. Only the secure overlay pipe, controlled by the Exclave, can write there.
  3. Sensor Interrupts: When the camera sensor is energized, the GPIO interrupt is routed directly to the Exclave (via the AIC - Apple Interrupt Controller).
  4. The Drawing Path: The Exclave receives the "Camera On" interrupt. It writes directly to the secure frame buffer overlay to render the green dot.
  5. Immutability: Because XNU cannot map the physical memory of the secure overlay, and cannot reconfigure the DART (locked by SPTM), a compromised kernel is physically incapable of erasing the indicator.

Passkeys and FIDO:
Similarly, the cryptographic operations for Passkeys (WebAuthn) have moved to an Exclave.

  • Private Keys: The private keys for Passkeys are generated and stored within the Exclave's secure storage (mediated by SEP).
  • Biometric Binding: The Exclave communicates directly with the SEP to verify FaceID/TouchID.
  • Isolation: Even if malware injects code into LocalAuthentication daemon or the kernel, it cannot extract the private key material, as it resides in a memory domain that simply does not exist in the attacker's address space.

6.0 The Mach Subsystem: The Nervous System

While the SPTM and Exclaves represent the new fortress walls of the Apple Silicon architecture, the Mach subsystem remains the internal nervous system that coordinates activity within the XNU kernel. Originating from the NeXTSTEP era, Mach provides the fundamental primitives for Inter-Process Communication (IPC), thread scheduling, and virtual memory management.

For the reverse engineer, Mach is the primary vector for local privilege escalation (LPE). Despite decades of hardening, the complexity of state management in Mach messaging remains a fertile ground for logic bugs, race conditions, and reference counting errors. In the Tahoe era, Mach has been retrofitted with heavy PAC enforcement to protect its object graph.

6.1 Mach Ports & IPC Primitives

At the conceptual level, Mach is an object-oriented kernel. The fundamental unit of addressing is the Mach Port . To a userland process, a port is merely a 32-bit integer handle ( mach_port_name_t ). To the kernel, it is a complex, reference-counted data structure ( ipc_port ) that acts as a unidirectional communication channel.

6.1.1 Port Rights: Receive, Send, Send-Once, and Dead Names

The security model of Mach is capability-based. Possessing a port name is meaningless without the associated Port Right . The kernel tracks these rights in the process's IPC space.

  • MACH_PORT_RIGHT_RECEIVE : The ownership right. Only one task can hold the Receive right for a specific port at any given time. This task is the destination for messages sent to the port.
    • Kernel Structure: The ipc_port struct contains a pointer ( ip_receiver ) to the ipc_space of the task holding this right.
  • MACH_PORT_RIGHT_SEND : The ability to queue messages into the port. Multiple tasks can hold send rights to the same port. This is the standard "client" handle.
  • MACH_PORT_RIGHT_SEND_ONCE : A "fire-and-forget" right that vanishes after a single message is sent. This is critical for the Request/Reply pattern (RPC). When a client sends a message, it typically includes a MAKE_SEND_ONCE right to its own reply port. The server uses this to send exactly one reply, preventing the server from spamming the client later.
  • MACH_PORT_RIGHT_DEAD_NAME : If the task holding the Receive right dies or destroys the port, all outstanding Send rights in other tasks are instantly transmuted into Dead Names. Any attempt to send a message to a dead name returns MACH_SEND_INVALID_DEST .

RE Focus: The ipc_port Structure and PAC:
In previous generations, a common exploit technique involved "Fake Ports"—spraying the heap with crafted data that looked like an ipc_port struct and then tricking the kernel into using it.

In the arm64e/Tahoe architecture, the ipc_port structure is heavily fortified:

  1. ip_object : The base header of the port.
  2. ip_kobject : A pointer to the underlying kernel object (e.g., a task, a thread, or a user-client). This pointer is PAC-signed.
  3. ip_context : A 64-bit context value, also PAC-signed.

If an attacker attempts to forge a port, they must generate a valid signature for the ip_kobject pointer. Without the APDAKey (Data Key A), the kernel will panic upon AUT execution during message delivery.

6.1.2 The IPC Space ( ipc_space ) and the Global Name Server

Every task (process) in macOS has an associated IPC Space ( ipc_space ). This structure acts as the translation layer between the userland integer handles ( mach_port_name_t ) and the kernel pointers ( ipc_port * ).

The Translation Table ( is_table ):
The IPC space is implemented as a dynamic table (or radix tree for large spaces) of IPC Entries ( ipc_entry ).

  • Index: The userland handle (e.g., 0x103 ) is essentially an index into this table.
  • Entry: The ipc_entry struct contains:
    • ie_object : A pointer to the actual ipc_port (or ipc_pset ). Crucially, in Tahoe, this pointer is signed with PAC.
    • ie_bits : A bitfield storing the rights (Receive, Send, etc.) and the generation number (to detect stale handles).

The Lookup Process ( ipc_right_lookup_write ):
When a user executes mach_msg() , the kernel:

  1. Retrieves the current_task()->itk_space .
  2. Uses the handle passed in msgh_remote_port to index into the is_table .
  3. Validates the ie_bits to ensure the task actually possesses the required right (e.g., MACH_PORT_RIGHT_SEND ).
  4. Authenticates the ie_object pointer using AUT .
  5. Retrieves the ipc_port .

The Global Name Server:
Mach does not have a string-based global namespace in the kernel. "Service Discovery" is implemented in userland by launchd (the Bootstrap Server).

  • Special Ports: The kernel does maintain a small array of "Special Ports" attached to the Host and Task objects.
    • HOST_PORT : Represents the kernel itself.
    • HOST_PRIV_PORT : The "God Mode" port (see Section 6.2).
    • TASK_BOOTSTRAP_PORT : The handle to launchd .

When a process calls bootstrap_look_up("com.apple.foo") , it is actually sending a Mach message to the port stored in its TASK_BOOTSTRAP_PORT slot.

6.1.3 Copy-on-Write (CoW) optimizations in Out-of-Line (OOL) message passing

Mach messages are not limited to small scalars. They can transfer massive amounts of data using Out-of-Line (OOL) descriptors. This mechanism relies on Virtual Memory (VM) tricks rather than data copying, making it highly efficient but historically dangerous.

The Mechanism:

  1. Sender: Includes a mach_msg_ool_descriptor_t in the message, pointing to a buffer in its address space (e.g., 100MB of data).
  2. Kernel Processing: The kernel does not copy the 100MB. Instead, it walks the sender's VM map.
  3. Copy-on-Write (CoW): The kernel marks the physical pages backing that buffer as Read-Only in the sender's map.
  4. Receiver: The kernel maps those same physical pages into the receiver's address space, also as Read-Only .
  5. Faulting: If either the sender or receiver tries to write to the buffer, the MMU triggers a fault. The kernel catches this, allocates a new physical page, copies the data, and updates the mapping for the writer. This preserves the illusion of a copy.

The Tahoe/SPTM Intersection:
In the Tahoe architecture, this VM manipulation is complicated by the SPTM.

  • Permission Downgrade: When the kernel marks the pages as CoW (Read-Only), it cannot simply update the PTEs. It must issue a GENTER call ( sptm_protect or sptm_map ) to the SPTM to downgrade the permissions of the physical pages in the sender's address space.
  • The Vulnerability Class: This complexity introduces a race window. If the kernel logic fails to correctly lock the VM map object before requesting the SPTM update, or if the SPTM state machine has a logic flaw regarding shared pages ( refcount > 1 ), it might be possible to modify the physical page after the message has been "sent" but before the receiver reads it. This is known as a Double Fetch or Physically-Backed-Virtually-Disjoint attack.

RE Focus:
Analyze vm_map_copyin and vm_map_copyout in the XNU source (or binary). Look for how vm_map_entry structures are flagged with needs_copy and how these flags translate into SPTM calls. The interaction between Mach IPC (which assumes it controls VM) and the SPTM (which actually controls VM) is the friction point where new bugs will likely emerge.

6.2 The host_priv Port

In the lexicon of XNU exploitation, the Host Privilege Port ( host_priv ) has historically been synonymous with "Game Over." It is the Mach port representing the kernel task itself. Possessing a send right to this port allows a userland process to invoke the host_priv_server subsystem, granting the ability to manipulate physical memory, load kernel extensions (legacy), and control processor sets.

However, in the Tahoe architecture, the omnipotence of host_priv has been severely curtailed. The Secure Page Table Monitor (SPTM) and the Trusted Execution Monitor (TXM) have effectively neutered the "God Mode" capabilities traditionally associated with this handle.

6.2.1 The "God Mode" Handle: Generation and Restriction

The host_priv port is backed by the realhost kernel global structure. Unlike standard task ports, which map to a task_t , the host_priv port maps to the host object, but with a specific bit set in its ip_kobject pointer or context indicating privileged access.

Generation at Boot:
During the kernel bootstrap ( osfmk/kern/host.c ), the system initializes the host subsystem.

  1. host_init() : Allocates the realhost structure.
  2. Port Allocation: Calls ipc_port_alloc_kernel() to create the port.
  3. Kobject Association: The port is associated with the realhost structure.
    • PAC Enforcement: On arm64e, the ip_kobject pointer stored in the port structure is signed using the APDAKey (Data Key A) combined with the address of the port structure as the context. This prevents an attacker with a kernel read/write primitive from simply overwriting the ip_kobject of a user port to point to realhost .

Distribution:
The kernel is extremely stingy with this port. It is vended primarily to:

  • kernel_task : The kernel itself holds the receive right.
  • launchd (PID 1): Receives a send right during the userland bootstrap handoff.
  • kextd / kernelmanagerd : Required for managing kernel extensions (though this role is diminishing with DriverKit).

The "TFP0" Fallacy in Tahoe:
Historically, obtaining host_priv allowed an attacker to call task_for_pid(host_priv, 0, &kernel_task) . This returned the kernel_task port, granting full vm_read / vm_write access to the kernel's entire address space (TFP0).

In the Tahoe architecture, this chain is broken by the SPTM :

  1. task_conversion_eval : Even if you possess host_priv , the kernel checks the System Integrity Protection (SIP) policy managed by the TXM. If SIP is active, task_for_pid(0) returns KERN_FAILURE .
  2. The Immutable Kernel: Even if you bypass the SIP check (e.g., via a logic bug), and obtain the kernel_task port, the vm_write operations are intercepted.
    • If you attempt to write to kernel text ( XNU_TEXT ), the SPTM panics the system (Permission Violation).
    • If you attempt to write to page tables ( PAGE_TABLE ), the SPTM panics the system.
    • If you attempt to write to Data Const ( XNU_DATA_CONST ), the SPTM panics.

Thus, in Tahoe, host_priv degrades from a "Write-Anywhere" primitive to a "Logic Control" primitive. It allows you to invoke privileged kernel APIs, but it does not grant direct memory corruption capabilities against protected regions.

6.2.2 Task Special Ports: The Privileged Directory

While host_priv itself is restricted, it acts as the directory service for a set of Special Ports that control specific subsystems. These are accessed via host_get_special_port (MIG ID 205).

The realhost structure maintains a static array of these ports: ipc_port_t special[HOST_MAX_SPECIAL_PORT + 1] .

Critical Special Ports:

  • HOST_KEXTD_PORT (ID 1): The handle used to communicate with the kernel extension manager. Historically used to force the kernel to load a malicious kext. In Tahoe, kext loading is gated by the TXM, which verifies the signature and the LocalPolicy .
  • HOST_SEATBELT_PORT (ID 7): The control port for the Sandbox ( sandbox.kext ). Possession of this port allows a process to query and potentially manipulate sandbox policies (though policy application is usually one-way).
  • HOST_AMFID_PORT (ID 18): The communication channel for amfid . This port is critical for the Code Signing validation loop.
  • HOST_AUTOMOUNTD_PORT (ID 11): Used to trigger filesystem mounts.

RE Focus: The host_set_special_port Trap:
A common post-exploitation technique involves overwriting one of these special ports with a port controlled by the attacker (e.g., replacing the KEXTD port). This allows the attacker to intercept kernel upcalls intended for system daemons.

  • Mitigation: The host_set_special_port routine checks the caller's entitlements. Only a process with the com.apple.private.set-special-port entitlement (usually only launchd ) can overwrite these entries.
  • Bypass: Attackers look for logic bugs where a race condition allows replacing a port before the entitlement check completes, or finding a daemon that holds this entitlement and coercing it to perform the set operation (Confused Deputy).

6.2.3 RE Focus: Fuzzing Mach Message Parsing (MIG)

Since host_priv exposes a wide attack surface via the Mach IPC interface, it is a primary target for fuzzing. The kernel handles these messages using the Mach Interface Generator (MIG) .

The host_priv_server Routine:
When a message is sent to host_priv , the kernel's IPC dispatcher calls host_priv_server . This is an auto-generated function that deserializes the Mach message and dispatches it to the implementation (e.g., kern_host.c ).

Vulnerability Classes in MIG:

  1. Type Confusion: MIG relies on the message header to define the size and type of arguments. If the userland client sends a malformed message (e.g., claiming a descriptor is OOL memory when it is actually inline data), the kernel's unmarshaling logic might misinterpret the data, leading to heap corruption.
  2. Reference Counting Leaks: If a MIG routine returns an error (e.g., KERN_INVALID_ARGUMENT ) after it has incremented the reference count on a port or VM object but before it consumes it, the object leaks. In the kernel, this can lead to a refcount overflow (though 64-bit refcounts make this hard) or a Use-After-Free if the cleanup logic is flawed.
  3. TOCTOU on OOL Memory: As discussed in Section 6.1.3, if the message includes Out-of-Line memory, the kernel maps it Copy-on-Write. If the MIG handler verifies the content of the memory and then uses it later, the userland process might be able to race the kernel and modify the physical page (via a side-channel or SPTM state confusion) between the check and the use.

The Tahoe Hardening:
In the Tahoe kernel, MIG-generated code has been hardened with PAC .

  • Function Pointers: The dispatch tables used by host_priv_server are signed.
  • Context: The ipc_kmsg structure (representing the message in flight) is heavily protected to prevent modification of the message contents after validation but before processing.

However, logic bugs in the implementation of the host calls (the C functions called by MIG) remain reachable. For example, host_processor_set_priv allows manipulating CPU sets. If the logic fails to account for a processor being in a low-power state or being managed by an Exclave, it could trigger an inconsistent state in the scheduler.

7.0 IOKit & Driver Architecture

While the Mach subsystem provides the primitives for IPC and scheduling, IOKit provides the object-oriented framework for device drivers. Historically, IOKit has been the "soft underbelly" of the XNU kernel. Written in a restricted subset of C++, it relies heavily on virtual function dispatch, complex inheritance hierarchies, and manual reference counting ( OSObject::retain / release ).

In the Tahoe architecture, IOKit has undergone a radical hardening process. The transition to Apple Silicon has allowed Apple to enforce strict Control Flow Integrity (CFI) on C++ objects using PAC, while the SPTM enforces the immutability of the driver code itself.

7.1 IOKit Initialization

The initialization of IOKit is the bridge between the static hardware description provided by iBoot (the Device Tree) and the dynamic, runtime object graph that constitutes the macOS driver environment.

7.1.1 The IORegistry: Populating the Device Tree into C++ Objects

When the kernel boots, the hardware topology is described by the Flattened Device Tree (FDT) located at the physical address passed in x1 to _start . IOKit's first major task is to hydrate this binary blob into a live graph of IORegistryEntry objects.

The IOPlatformExpert :
The bootstrap process is driven by the IOPlatformExpert class (specifically IOPlatformExpertDevice on Apple Silicon).

  1. Unflattening: The kernel parses the FDT. For every node in the tree (e.g., arm-io , uart0 , aic ), it instantiates an IORegistryEntry .
  2. The IODeviceTree Plane: These objects are attached to the IODeviceTree plane of the Registry. This plane represents the physical topology as reported by iBoot.
  3. Property Mapping: Properties from the FDT (like reg , interrupts , compatible ) are converted into OSData , OSString , or OSNumber objects and attached to the registry entries.

Matching and Driver Loading:
Once the Registry is populated, IOKit begins the Matching phase ( IOService::startMatching ).

  1. The compatible String: IOKit iterates over the registry entries. It compares the compatible property (e.g., apple,s5l8960x-uart ) against the IOKitPersonalities dictionary defined in the Info.plist of every loaded driver.
  2. The Probe/Start Lifecycle: When a match is found, the driver's C++ class is instantiated.
    • init() : Constructor.
    • probe() : The driver verifies the hardware is actually present (rarely used on SoCs where hardware is static).
    • start() : The driver initializes the hardware, maps MMIO regions, and registers interrupts.

RE Focus: The "Missing" Hardware:
On Tahoe systems, you will notice gaps in the IORegistry compared to the raw Device Tree.

  • Reserved Regions: The SPTM and TXM reserve specific hardware blocks (e.g., the Secure Storage controller or specific GPIO banks for privacy LEDs).
  • Filtering: During the unflattening process, the kernel checks the memory map. If a device node's reg property overlaps with a physical range reserved by the SPTM, the kernel skips creating an IORegistryEntry for it. This prevents the kernel from even attempting to map the MMIO registers for secure hardware, effectively making that hardware invisible to the OS.

7.1.2 Boot Kernel Collection (BKC) vs. Auxiliary Kernel Collection (AKC)

Gone are the days of loading individual .kext bundles from /System/Library/Extensions . To optimize boot speed and enforce immutability, macOS now uses Kernel Collections .

The Boot Kernel Collection (BKC):
This is a single, massive Mach-O binary contained within the kernelcache Image4 payload ( Ap,BootKernelCollection ).

  • Content: It contains the XNU kernel and all "Essential" drivers required to mount the root filesystem, start launchd , and initialize the display.
  • Linkage: All internal symbols are pre-linked. There are no external relocations required at boot time for these drivers.
  • Protection (KIP): This is the critical security distinction. Because the BKC is loaded by iBoot, its physical pages are known before the kernel starts. The SPTM marks these pages as XNU_TEXT (Immutable) in the Frame Table. The kernel cannot modify the BKC code or read-only data, even with a write primitive. It is effectively ROM.

The Auxiliary Kernel Collection (AKC):
Drivers not required for early boot (e.g., Bluetooth, Wi-Fi, Audio) are packaged into the AKC.

  • Loading: The AKC is loaded later in the boot process by kernelmanagerd (userland).
  • Verification: When kernelmanagerd passes the AKC to the kernel, the kernel must verify its signature. In Tahoe, this verification is offloaded to the TXM . The TXM checks the signature against the LocalPolicy .
  • Sealing: Once verified and linked, the kernel issues a GENTER call to the SPTM to "seal" the AKC memory region. The SPTM transitions the pages from XNU_DATA to XNU_TEXT . Once sealed, the AKC becomes just as immutable as the BKC.

RE Implication:
If you are reverse engineering a driver, you must determine if it lives in the BKC or AKC.

  • BKC Drivers: Their offsets are static relative to the kernel slide.
  • AKC Drivers: They are loaded at a randomized offset after the kernel boot. You must traverse the kmod_info linked list in memory to find their load addresses.

7.1.3 RE Focus: PAC-signing of C++ Vtables ( OSObject ) and BLRAA

The OSObject class is the root of the IOKit inheritance hierarchy. In C++, dynamic dispatch is handled via Virtual Tables (vtables) —arrays of function pointers. Historically, attackers would overwrite the vtable pointer in an object to point to a fake vtable controlled by the attacker (vtable hijacking).

In the arm64e architecture, Apple has fundamentally altered the C++ ABI for kernel code to mitigate this.

The Signed Vtable Pointer:
In a standard C++ object, the first 8 bytes are the pointer to the vtable. In XNU on arm64e, this pointer is signed .

  • Key: APDAKey (Data Key A).
  • Context: The context is derived from the address of the object itself (specifically, the storage location of the vtable pointer) combined with a salt (often the type hash).

$$ \text{SignedPtr} = \text{PAC}(\text{VtableAddr}, \text{Key=APDA}, \text{Context}=&Object) $$

Address Diversity:
Because the object's address is part of the signature context, an attacker cannot simply copy a valid vtable pointer from Object A to Object B . The signature for Object A is valid only at Object A 's address. Moving it invalidates the PAC. This kills "vtable reuse" attacks.

The BLRAA Dispatch:
When the kernel calls a virtual function (e.g., object->release() ), the compiler emits a specialized instruction sequence:

LDR     x0, [x20]       ; Load the object pointer
LDR     x16, [x0]       ; Load the signed vtable pointer
AUTDA   x16, x0         ; Authenticate Data Key A, Context = Object Address (x0)
LDR     x10, [x16, #0x18] ; Load the target function pointer from the vtable
BLRAA   x10, x16        ; Branch with Link, Authenticating Key A, Context = Vtable Address

Note the two-stage authentication:

  1. AUTDA : Authenticates that the vtable pointer belongs to this object. If the pointer was overwritten, x16 becomes a canonical non-valid pointer.
  2. BLRAA : The function pointers inside the vtable are also signed (using APIAKey ). The BLRAA instruction authenticates the function pointer (using the vtable address as context) and branches.

The "Recursive" PAC:
This creates a chain of trust:

  • The Object trusts the Vtable Pointer (via APDAKey ).
  • The Vtable trusts the Function Pointers (via APIAKey ).
  • The SPTM trusts the Vtable Memory (via XNU_TEXT immutability).

For a reverse engineer, this means that patching a vtable in memory is impossible (SPTM), and forging an object requires the ability to sign pointers with the APDAKey for an arbitrary address—a capability that requires a "Signing Oracle" gadget, which BTI aims to eliminate.

7.2 DriverKit (dexts)

The introduction of DriverKit represents a strategic retreat for the XNU kernel. For decades, the kernel’s attack surface was effectively the sum of the core kernel plus every third-party driver loaded into the address space. A vulnerability in a Wacom tablet driver or a USB-to-Serial adapter was functionally identical to a vulnerability in the scheduler: both yielded EL1 code execution.

DriverKit bifurcates this model by moving hardware drivers into userland, executing as System Extensions ( .dext ) . While they look and feel like drivers to the developer (using a C++ subset similar to Embedded C++), architecturally they are unprivileged processes. In the Tahoe architecture, this isolation is not merely a sandbox; it is a hardware-enforced chasm guarded by the TXM and SPTM.

7.2.1 Moving drivers to userland: IOUserClient and Entitlement Checks

A dext does not have direct access to the kernel's task port, nor does it run with root privileges by default. It operates within a tightly constrained sandbox, communicating with the kernel via a specialized IPC bridge.

The IOUserServer Proxy:
When a dext is matched and loaded (managed by kernelmanagerd ), the kernel instantiates a shadow object known as IOUserServer . This kernel-side object acts as the proxy for the userland driver.

  • The Shim: When the kernel needs to call a function in the driver (e.g., Start() ), it calls a method on IOUserServer .
  • Serialization: IOUserServer serializes the arguments into a specialized Mach message format (distinct from standard MIG).
  • The Upcall: The message is sent to the dext process. The DriverKit runtime (linked into the dext ) deserializes the message and invokes the implementation of the IOService subclass in userland.

The IOUserClient Interface:
Conversely, when the dext needs to talk to the kernel (e.g., to register an interrupt handler or map memory), it cannot call kernel APIs directly. It uses IOUserClient .

  • Restricted API Surface: The dext can only invoke a specific subset of kernel functionality exposed via IOUserClient traps. These traps are heavily scrutinized.
  • OSAction : Interrupts are no longer handled via ISRs (Interrupt Service Routines) in the driver. Instead, the kernel handles the physical IRQ, masks it, and dispatches an OSAction event to the dext via a Mach notification. This eliminates the entire class of vulnerabilities related to interrupt context safety and spinlock deadlocks in third-party code.

Entitlements as the Gatekeeper (TXM Enforcement):
In Tahoe, the ability of a dext to bind to specific hardware is governed by Entitlements .

  • Hardware Binding: A dext cannot simply mmap any MMIO region. It must possess specific entitlements (e.g., com.apple.developer.driverkit.transport.usb ) to access specific device families.
  • TXM Verification: When kernelmanagerd launches the dext , the signature and entitlements are validated by the Trusted Execution Monitor (TXM) .
    • The TXM verifies the CDHash against the Trust Cache.
    • The TXM validates that the provisioning profile allows the specific HID or USB entitlements requested.
    • If the TXM returns a failure, the kernel refuses to establish the IOUserServer link, and the driver fails to start. This prevents a compromised kernelmanagerd from launching a rogue driver with elevated hardware access.

RE Focus: The IOUserClass Metaclass:
Reverse engineering a dext requires understanding the OSMetaClass infrastructure in userland. The dext binary contains OSMetaClass information that describes the RPC interface. By parsing the __DATA,__const sections, one can reconstruct the vtables and the mapping between the kernel-side dispatch IDs and the userland C++ methods.

7.2.2 Memory Mapping Constraints and IOMMU (DART) Protection

The most dangerous capability of a driver is Direct Memory Access (DMA) . A malicious or buggy driver could program a peripheral (like a GPU or Network Card) to write data to physical address 0x0 (or wherever the kernel text resides), bypassing CPU-enforced protections like KTRR.

To mitigate this, Apple Silicon employs a pervasive IOMMU architecture known as DART (Device Address Resolution Table) .

The DART Architecture:
Every DMA-capable peripheral on the SoC sits behind a DART. The device does not see Physical Addresses (PA); it sees I/O Virtual Addresses (IOVA) . The DART translates IOVA $\rightarrow$ PA, enforcing permissions (Read/Write) at the page level.

DriverKit Memory Model:

  1. IOMemoryDescriptor : When a dext allocates a buffer for DMA, it creates an IOMemoryDescriptor .
  2. Mapping: The dext calls IOMemoryDescriptor::CreateMapping . This triggers a call into the kernel.
  3. The Kernel's Role: The kernel allocates physical pages ( XNU_DATA ) and pins them.
  4. DART Programming: The kernel programs the DART associated with the specific hardware device controlled by the dext . It maps the physical pages to an IOVA range visible to that device.

The Tahoe/SPTM Enforcement:
In the Tahoe architecture, the kernel is no longer trusted to program the DARTs directly. If the kernel could write to DART registers, it could map the kernel's own text segment as writable to the GPU, then tell the GPU to overwrite it (a DMA attack).

  • SPTM Ownership: The physical pages containing the DART translation tables (or the MMIO registers controlling the DART) are typed as SPTM_PRIVATE or a specific hardware-protected type in the Frame Table.
  • The sptm_map_iommu Selector: When the kernel needs to map a buffer for a dext , it issues a GENTER call to the SPTM.
    • Validation: The SPTM verifies that the physical pages being mapped are owned by the dext (or are valid shared memory). It strictly forbids mapping any page typed XNU_TEXT , PAGE_TABLE , or SPTM_PRIVATE into a DART.
    • Execution: The SPTM performs the write to the DART hardware.

MMIO Mapping Restrictions:
Similarly, when a dext needs to control hardware registers (MMIO), it requests a mapping.

  • The kernel cannot simply map physical device memory into the dext 's address space.
  • The SPTM validates the physical address against a Device Allow List (embedded in the Device Tree and signed/verified by iBoot).
  • This ensures that a USB driver can only map the USB controller's registers, and cannot map the registers for the Secure Enclave Mailbox or the KTRR controller.

RE Implication:
Exploiting a dext to gain kernel privileges is exponentially harder in Tahoe. Even if you gain code execution in the dext (Userland), you cannot issue arbitrary syscalls (sandbox), you cannot map kernel memory (VM isolation), and you cannot use the hardware device to perform DMA attacks against the kernel (SPTM-enforced DART). The attacker is contained within a hardware-enforced cage, limited to the specific capabilities of that one peripheral.

7.3 The Graphics Stack (AGX)

If the XNU kernel is the central nervous system, the Apple Graphics (AGX) stack is a secondary, alien brain grafted onto the SoC. On M-series silicon, the GPU is not merely a peripheral; it is a massive, autonomous compute cluster running its own proprietary operating system, managing its own memory translation, and executing a command stream that is almost entirely opaque to the main OS.

For the reverse engineer, AGX represents the largest and most complex attack surface in the kernel. The driver ( AGX.kext ) is enormous, the firmware is encrypted (until load), and the hardware interface is undocumented. In the Tahoe architecture, Apple has moved to aggressively sandbox this beast, wrapping the GPU's memory access in strict DART (Device Address Resolution Table) policies enforced by the SPTM to prevent DMA-based kernel compromises.

7.3.1 RTKit: The Proprietary RTOS running on the GPU Coprocessor (ASC)

The GPU does not execute driver commands directly. Instead, the M-series SoC includes a dedicated Apple Silicon Coprocessor (ASC) —typically a hardened ARMv8-R or Cortex-M class core—that manages the GPU hardware. This coprocessor runs RTKit , Apple’s proprietary Real-Time Operating System.

The Firmware Blob:
The kernel driver does not contain the logic to drive the GPU hardware registers directly. Instead, upon initialization ( AGX::start ), it loads a firmware payload from the filesystem (e.g., /usr/share/firmware/agx.bin or embedded in the kext).

  • Format: The firmware is a standard Mach-O binary, often multi-architecture.
  • Sections: It contains __TEXT and __DATA segments just like a userland program.
  • RTKit Structure: Reverse engineering the firmware reveals a microkernel architecture. It has a scheduler, an IPC mechanism, and a set of "Endpoints" (services).

The RTKit IPC Protocol:
Communication between the XNU kernel ( AGX.kext ) and the ASC ( RTKit ) occurs via a shared memory mailbox protocol.

  1. Mailbox Registers: The AP writes to a specific MMIO register to ring the doorbell of the ASC.
  2. Message Buffer: The message payload is placed in a shared memory ring buffer.
  3. Endpoints: The protocol is endpoint-based. The kernel sends messages to specific service IDs running on the ASC:
    • EP_PM : Power Management (Voltage/Clock gating).
    • EP_GR : Graphics Rendering (Command submission).
    • EP_COMP : Compute (GPGPU/Metal).

RE Focus: The RTKit State Machine:
The AGX.kext contains extensive logging strings and state tracking for RTKit. By analyzing the RTKit class in the kext, one can reconstruct the message structures.

  • Crash Logs: When the GPU hangs, RTKit writes a "Coredump" to a shared buffer. The kernel captures this. Analyzing these logs reveals the internal memory layout of the ASC and the state of the GPU pipeline at the time of the crash.
  • Filter Bypass: Historically, vulnerabilities existed where the kernel could send malformed IPC messages to the ASC, causing memory corruption inside the GPU firmware. While this doesn't directly yield Kernel R/W, compromising the ASC allows an attacker to use the GPU as a confused deputy for DMA attacks (see 7.3.3).

7.3.2 IOMobileFramebuffer (IOMFB): Secure Framebuffers and Exclave Compositing

While AGX.kext handles rendering, IOMobileFramebuffer.kext (IOMFB) handles the display controller (DCP). This driver is responsible for the "Swap Chain"—taking the rendered frames and scanning them out to the display panel.

The Unified Memory Architecture (UMA):
On Apple Silicon, the Framebuffer is just a region of system DRAM. WindowServer (userland) renders into an IOSurface . The physical pages backing this surface are passed to IOMFB, which programs the Display Coprocessor (DCP) to read from them.

The Security Criticality:
IOMFB is a high-value target because it handles complex shared memory structures ( IOMFBSharedMemory ) mapped into both the kernel and userland ( WindowServer ).

  • Fuzzing Surface: The Connect method and external methods of IOMobileFramebufferUserClient have historically been riddled with race conditions and bounds-checking errors.

Tahoe and the "Secure Overlay":
In the Tahoe architecture, IOMFB's control over the display is no longer absolute. To support the Hardware Privacy Indicators (Green/Orange dots), the display pipeline has been bifurcated.

  1. Standard Pipe: Managed by IOMFB/WindowServer. Draws the desktop/apps.
  2. Secure Pipe: Managed by an Exclave . Draws the privacy indicators.

Hardware Compositing:
The compositing of these two pipes happens in the display hardware, not in memory.

  • The Exclave owns a small, reserved framebuffer region.
  • The Display Controller overlays this region on top of the standard framebuffer during scanout .
  • Immutability: Because the Secure Pipe's framebuffer memory is owned by the Exclave (and protected by the SPTM), neither the kernel nor the GPU can write to it. This guarantees that if the camera is on, the green dot will be visible, even if the kernel is compromised.

7.3.3 DART: The IOMMU Wall and DMA Containment

The GPU is effectively a DMA engine with the capability to read and write vast swathes of system memory. Without restriction, a compromised GPU firmware (or a malicious shader exploiting a GPU hardware bug) could overwrite kernel text or page tables.

To prevent this, the AGX hardware—and indeed every DMA-capable peripheral on the Apple Silicon SoC—sits behind a strict IOMMU known as the DART (Device Address Resolution Table) .

DART Architecture and Stream IDs (SIDs):
The DART translates Device Virtual Addresses (DVA) used by the peripheral into Physical Addresses (PA) in DRAM. However, the translation is not monolithic; it is segmented by the source of the traffic.

  • Stream IDs (SIDs): Every transaction on the SoC's Network-on-Chip (NoC) carries a hardware-generated Stream ID identifying the initiator (e.g., GPU Firmware, Vertex Fetcher, Display Controller).
  • Context Banks: The DART maintains multiple translation contexts (similar to distinct TTBR roots).
  • SID Matching: The DART hardware is configured to map specific SIDs to specific Context Banks. This allows isolation between different workloads on the same peripheral (e.g., isolating WindowServer rendering commands from a background compute shader).

The Tahoe Enforcement (SPTM):
In pre-Tahoe systems, the kernel ( AGX.kext or IOMapper ) managed the DART page tables and the SID configuration registers directly. This meant a kernel attacker could disable DART, remap SIDs to privileged contexts, or map kernel memory into the GPU's address space to bypass KTRR.

In Tahoe, DART management is privileged to the SPTM.

  1. Ownership: The physical pages containing the DART translation tables (L1/L2 TTEs) and the MMIO registers controlling SID configuration are typed as SPTM_PRIVATE (or a specific IOMMU_TABLE type) in the Frame Table.
  2. Mapping Requests: When AGX.kext needs to map a user's IOSurface for GPU access:
    • It pins the user pages.
    • It issues a GENTER call (Selector sptm_map_iommu ).
    • It passes the DART ID, the Context ID, the DVA, and the PA.
  3. Validation: The SPTM verifies:
    • The PA is valid USER_DATA (not Kernel Text, not Page Tables).
    • The DART ID corresponds to the GPU.
    • SID Integrity: Crucially, the SPTM enforces the immutable binding between SIDs and Contexts. It ensures that the kernel cannot reconfigure the DART to allow an untrusted SID (e.g., the Neural Engine) to write to a Context Bank reserved for the Secure Enclave or Display Pipe.
  4. Execution: The SPTM writes the DART PTE.

RE Focus: The "GART" Attack Surface:
Despite SPTM protection, the logic requesting the mapping still resides in the kernel.

  • Aliasing: Can the kernel trick the SPTM into mapping the same physical page to two different DART contexts with different permissions?
  • Stale TLBs: Does the SPTM correctly flush the DART TLB ( tlb_flush ) immediately after unmapping? If not, the GPU might retain access to a page that has been freed and reallocated to the kernel, leading to a Use-After-Free via DMA.
  • Side Channels: The DART configuration registers (e.g., TCR , TTBR , and SID match registers) are trapped by the hardware to GL2. Attempting to write to the DART control base address from EL1 should trigger a synchronous exception. Reverse engineering the IOMapper class in IOKit will reveal the specific GENTER trampolines used to bridge these operations.

8.0 Userland Bootstrap: The Birth of PID 1

The initialization of the XNU kernel concludes with the mounting of the root filesystem (the Signed System Volume) and the spawning of the first userland process. In the UNIX tradition, this is init (PID 1). In macOS, this is launchd .

However, launchd is far more than a SysV init replacement. It is the Mach Port Broker , the central registry for the operating system's IPC namespace, and the gatekeeper for service lifecycle management. In the Tahoe architecture, launchd 's role is further hardened; it is the first process to run under the full scrutiny of the Trusted Execution Monitor (TXM), and its ability to bootstrap the rest of the system relies on a complex dance of entitlements and port rights.

8.1 launchd : The Mach Port Broker

The transition from kernel mode to user mode is a singularity. The kernel thread executing bsd_init constructs a user context and never returns. From this point forward, the system's security relies on the correct enforcement of Mach port rights and the immutability of the bootstrap namespace.

8.1.1 Transition from kernel to userland: The first execve

The kernel routine bsd_init (in bsd/kern/bsd_init.c ) is responsible for hand-crafting the first process.

The Loading Sequence:

  1. Path Resolution: The kernel looks for the init binary. Historically /sbin/launchd , in the Tahoe/Cryptex era, this path is a firmlink or symlink resolving into the OS Cryptex (e.g., /System/Cryptexes/OS/sbin/launchd ).
  2. load_init_program : This kernel function mimics an execve syscall from within the kernel. It allocates a proc_t structure and a task_t structure.
  3. TXM Validation (The First Check):
    Before the kernel can map launchd into memory, it must validate the code signature.
    • XNU parses the launchd binary to extract the CDHash.
    • XNU issues a GENTER to the TXM .
    • The TXM verifies that the launchd CDHash exists in the Static Trust Cache (loaded by iBoot). Since launchd is part of the immutable OS, it must be in the static cache. If it is not, the system panics immediately (integrity violation).
  4. SPTM Mapping:
    Once validated, XNU requests the SPTM to map the physical pages containing the launchd text segment.
    • The SPTM updates the Frame Table Entries (FTE) for these pages to USER_DATA (or a specific immutable user type).
    • The SPTM writes the translation table entries (TTEs) for the new process's address space, marking the text segment as RX (Read-Execute) and the data segment as RW (Read-Write).

The host_priv Handoff:
Crucially, launchd is the only process in the system that receives the Host Privilege Port ( host_priv ) directly from the kernel without asking for it.

  • During load_init_program , the kernel calls ipc_port_alloc_kernel to retrieve the host_priv port.
  • It inserts a send right to this port into launchd 's IPC space ( ipc_space ).
  • This right is placed at a fixed, known port name (often mach_task_self_ + n ), or retrieved by launchd immediately upon startup via task_get_special_port .

This handle gives launchd the authority to configure the rest of the system, including setting the system clock, loading kernel extensions (via kernelmanagerd ), and shutting down the system.

8.1.2 Initializing the Bootstrap Port (subset of the Name Service)

Mach does not have a global, string-based name service in the kernel. If Process A wants to talk to "com.apple.windowserver", the kernel has no idea what that string means. The mapping of String -> Mach Port is the responsibility of the Bootstrap Server . launchd is the Bootstrap Server.

The Bootstrap Port:
Every task in XNU has a special port slot called TASK_BOOTSTRAP_PORT .

  • Initialization: When launchd starts, it allocates a receive right for a new port. It calls task_set_special_port(mach_task_self(), TASK_BOOTSTRAP_PORT, new_port) to register this port as the bootstrap port for itself .
  • Inheritance: When launchd spawns a child process (e.g., syslogd ), the child inherits this bootstrap port handle. This connects every process in the system back to launchd .

The Namespace Hierarchy:
The bootstrap namespace is not flat; it is hierarchical to support security domains and user separation.

  1. Startup Domain: The root domain, populated by LaunchDaemons (system-wide services running as root or specific users).
  2. User Domain: Created when a user logs in. Populated by LaunchAgents .
  3. Session Domain: Specific to a GUI login session (Aqua).

RE Focus: The bootstrap_check_in Routine:
When a daemon starts (e.g., locationd ), it must retrieve the receive rights for the Mach ports it is supposed to service.

  1. The daemon calls bootstrap_check_in(bootstrap_port, "com.apple.locationd", &service_port) .
  2. launchd receives this message.
  3. Security Check: launchd inspects the Audit Token (appended to the Mach message trailer by the kernel). It verifies that the PID sending the check-in request matches the PID that launchd just spawned for that label.
  4. Port Transfer: If the check passes, launchd moves the Receive Right for the service port (which it created earlier) into the daemon's IPC space.

This prevents "Port Squatting," where a malicious process tries to register "com.apple.password-server" before the real daemon starts. Only the process spawned by launchd as that service can claim the port.

8.1.3 Parsing launchd.plist and the binary protocol for XPC service lookups

launchd configuration is driven by Property Lists ( .plist ) located in /System/Library/LaunchDaemons , /Library/LaunchDaemons , etc. However, launchd does not simply read these files at runtime; it compiles them into an internal graph.

The MachService Dictionary:
The key element for reverse engineers is the MachServices dictionary in the plist.

<key>MachServices</key>
<dict>
    <key>com.apple.securityd</key>
    <true/>
</dict>

This entry tells launchd : "Allocate a Mach port. Associate it with the string 'com.apple.securityd'. If anyone sends a message to this port, launch this binary."

Demand Launching (The Lazy Allocation):
launchd does not start all daemons at boot. It creates the ports and listens on them.

  1. The Lookup: A client (e.g., Safari) calls xpc_connection_create_mach_service("com.apple.securityd", ...) .
  2. The Resolution: Under the hood, libxpc sends a message to the bootstrap port asking for the send right associated with that string.
  3. The Trigger: launchd looks up the string. It sees the service is not running. It performs posix_spawn to start the daemon.
  4. The Handoff: While the daemon is launching, launchd holds the client's request. Once the daemon checks in (see 8.1.2), launchd hands the send right to the client.

The Binary Protocol (XPC vs. MIG):
Historically, launchd used the Mach Interface Generator (MIG) for these lookups ( bootstrap.defs ). In modern macOS, this has largely been superseded by the XPC protocol, which runs over raw Mach messages but uses a proprietary serialization format.

  • xpc_pipe : This is the low-level mechanism. Messages sent to launchd are often XPC dictionaries serialized into a Mach message body.
  • Magic Bytes: XPC messages start with a specific magic header (often 0x42774242 - 'BwBB' in ASCII for binary blobs).
  • Structure: The payload is a serialized tree of objects (dictionaries, arrays, strings, file descriptors, and Mach ports ).

RE Focus: launchd 's job_dispatch :
Reverse engineering the launchd binary (which is stripped, but symbols can often be correlated from open-source releases of liblaunch ) reveals the job_dispatch or demux routine.

  • This routine parses the incoming Mach message.
  • It identifies if it is a legacy MIG request (subsystem 400) or an XPC request.
  • It routes the request to the appropriate subsystem (Job Management, Service Lookup, or XPC Domain management).

The "Domain" Concept:
In Tahoe, launchd enforces strict Domain Isolation . A process in the "App Sandbox" domain cannot look up services in the "System" domain unless those services are explicitly allowlisted in the sandbox profile ( com.apple.security.exception.mach-lookup.global-name ). launchd enforces this by checking the sender's entitlements (via the kernel audit token) against the requested service name before returning the port right. If the check fails, launchd returns BOOTSTRAP_UNKNOWN_SERVICE (or effectively 0 ), and the client sees "Connection Invalid."

8.2 The Dynamic Linker ( dyld )

If launchd is the architect of the userland process hierarchy, dyld (the dynamic linker) is the mason that lays the bricks for every single process. In the macOS ecosystem, dyld is not merely a library loader; it is a privileged extension of the kernel’s execution logic, responsible for enforcing the final mile of the platform’s security policies, including Library Validation, symbol resolution, and the application of Address Space Layout Randomization (ASLR).

On Apple Silicon and the Tahoe architecture, dyld has evolved significantly. It no longer relies on legacy load commands for rebasing; it utilizes Chained Fixups to enable page-in linking, and its operations are tightly coupled with the kernel’s memory management subsystem, which is in turn subservient to the Secure Page Table Monitor (SPTM) .

8.2.1 Mapping the Dyld Shared Cache (DSC)

The Dyld Shared Cache (DSC) is the defining characteristic of the macOS memory layout. It is a massive, pre-linked artifact (often exceeding 4GB) containing the combined text and data segments of nearly all system frameworks ( CoreFoundation , libSystem , Foundation , UIKit , etc.).

The Shared Region:
To optimize memory usage, the kernel maintains a Shared Region . This is a reserved area of the virtual address space (typically starting at 0x180000000 on arm64) where the DSC is mapped.

  • Physical Sharing: The physical pages backing the DSC are shared across all processes.
  • Split Segments: The DSC is split into three primary mappings to satisfy W^X (Write XOR Execute) requirements:
    1. __TEXT : Read-Execute (RX). Contains immutable code.
    2. __DATA_CONST : Read-Only (RO). Contains pointers and data that are resolved at link time and never change.
    3. __DATA_DIRTY : Read-Write (RW). Contains data that must be modified per-process (e.g., Objective-C class realization).

The Tahoe/SPTM Constraint:
In the Tahoe architecture, the mapping of the DSC is a privileged operation guarded by the SPTM.

  1. Boot Load: The DSC is loaded from the OS Cryptex ( /System/Cryptexes/OS/System/Library/dyld/ ).
  2. Immutable Mapping: When the kernel initializes the Shared Region, it issues GENTER calls to the SPTM. The SPTM updates the Frame Table Entries (FTE) for the physical pages backing the DSC __TEXT segment to USER_TEXT (or a specific shared immutable type).
  3. Enforcement: Once mapped, the SPTM forbids any entity (including the kernel) from writing to these physical pages. This renders the system libraries immutable even in the face of a kernel compromise. Hooking a system function like open in the shared cache via kernel memory writing is physically impossible; the attacker must use __DATA_DIRTY indirection or userland hooks.

8.2.2 RE Focus: Code Signature Validation ( fcntl(F_ADDFILESIGS) ) and the Call to TXM

dyld is the primary enforcer of Library Validation . This security hardening ensures that a process can only load libraries that are signed by Apple or signed by the same Team ID as the main executable.

The Validation Flow:
When dyld loads a Mach-O image (via dlopen or load command parsing):

  1. mmap : It maps the file into memory. At this stage, the pages are not yet executable.
  2. Signature Registration: dyld calls fcntl(fd, F_ADDFILESIGS_RETURN, ...) passing a fsignatures_t struct. This tells the kernel: "I want to execute code from this file. Here is the offset to the code signature (CMS blob)."

The Kernel $\rightarrow$ TXM Handoff:
In previous architectures, the kernel (specifically AMFI.kext ) would parse the blob and verify the signature. In Tahoe, the kernel is untrusted for verification.

  1. Marshaling: The kernel identifies the physical pages containing the signature blob.
  2. The Gate: The kernel executes GENTER to invoke the Trusted Execution Monitor (TXM) at GL1.
  3. TXM Verification:
    • The TXM parses the CMS blob in the secure world.
    • It verifies the cryptographic chain of trust against the Trust Cache (for platform binaries) or the Apple Root CA (for third-party).
    • It checks for constraints (e.g., is this a restricted binary? Does it have the com.apple.private.security.no-sandbox entitlement?).
  4. The Verdict: The TXM returns a verdict to the kernel. Crucially, it also updates the SPTM's view of those physical pages.
    • If valid, the TXM "blesses" the pages associated with the code directory hash (CDHash).
  5. Page Fault Enforcement:
    • When dyld later jumps into the library, a page fault occurs.
    • The kernel attempts to map the page as Executable ( RX ).
    • The SPTM intercepts the PTE update. It checks if the physical page has been blessed by the TXM.
    • If yes, the Execute bit is set. If no (e.g., the kernel lied about the signature verification), the SPTM denies the mapping, and the process crashes with SIGKILL (Code Signature Invalid) .

RE Tip: If you are analyzing a crash where dyld terminates a process immediately after loading a dylib, check the system logs for amfid or kernel messages related to TXM rejection. The error code returned by fcntl is often sanitized; the real reason (e.g., "Revoked Certificate") lies in the secure monitor logs, which may be redacted in production builds.

8.2.3 ASLR in Userland: Chained Fixups and the Death of dyld_info

Modern dyld (dyld 4.0+) on Apple Silicon has deprecated the legacy LC_DYLD_INFO rebasing opcodes in favor of Chained Fixups ( LC_DYLD_CHAINED_FIXUPS ). This architectural shift is not merely a performance optimization to reduce dirty pages; it is a fundamental change in how the binary format represents memory addresses, tightly coupling ASLR with Pointer Authentication (PAC).

The Problem with Legacy Rebasing:
In the legacy model, dyld parsed a list of opcodes ( REBASE_OPCODE_DO_* ) to locate pointers in the __DATA segment and add the ASLR slide. This required dyld to touch every page containing a pointer at launch time, triggering massive I/O and dirtying pages that might never be used.

The Chained Fixup Solution:
In a Chained Fixup binary, the "pointers" stored in the __DATA segment on disk are not memory addresses . They are 64-bit metadata structures that form a linked list (a chain) within each 16KB page.

The On-Disk Structure:
The LC_DYLD_CHAINED_FIXUPS load command points to a header ( dyld_chained_fixups_header ) containing offsets to "Starts" arrays ( dyld_chained_starts_in_image ).

  • Page-Level Granularity: The metadata identifies the offset of the first fixup in every page.
  • The Chain: The data at that offset is a 64-bit struct (e.g., dyld_chained_ptr_64_rebase ).
    • next (Bits 51-62): The offset (in 4-byte strides) to the next fixup in the same page. A value of 0 terminates the chain for that page.
    • target (Bits 0-35): The offset in the __TEXT segment (for rebasing) or the ordinal in the import table (for binding).
    • high8 (Bits 56-63): Additional attributes (e.g., auth diversity).

Page-In Linking (Lazy Fixups):
When dyld loads a library, it does not iterate the whole file. It sets up the memory mapping.

  1. The Fault: When the process execution flow hits a pointer in __DATA (e.g., a vtable entry or a global string pointer), a page fault occurs.
  2. The Trap: The kernel (or dyld via a user-fault handler) detects that the page requires fixups.
  3. The Walk: The fixup logic looks up the "Start" offset for that specific page. It walks the linked list defined by the next bits.
  4. The Patch: For each entry in the chain:
    • It extracts the target .
    • It adds the ASLR slide (for rebase) or resolves the symbol (for bind).
    • Crucially for arm64e: It calculates the PAC signature.
  5. The Write: It overwrites the 64-bit metadata struct with the final, signed, slid pointer. The page is now "dirty" and valid.

PAC Integration ( DYLD_CHAINED_PTR_ARM64E ):
On Apple Silicon, the fixup format is often DYLD_CHAINED_PTR_ARM64E . This struct contains the diversity data (salt) for the pointer.

  • dyld does not just write Base + Offset .
  • It writes PAC(Base + Offset, Key, Context) .
  • This binds the ASLR slide to the process-specific PAC keys ( APIAKey , APDAKey ) at the exact moment of paging in.

RE Implication: The "Broken" Binary:
For the reverse engineer, this mechanism means that static analysis of a raw Mach-O binary from disk is misleading.

  • If you open a raw binary in an older disassembler (or hexdump ), the global pointers will look like garbage (e.g., 0x0040... ). These are the chain metadata structs, not addresses.
  • CFG Destruction: Without processing these chains, the Control Flow Graph (CFG) is broken. Virtual function calls and global callbacks point to nowhere.

Tooling Requirement:
To analyze these binaries statically, you must "unchain" them—simulating the dyld fixup process to convert metadata back into generic, unslid pointers.

  • ipsw : The ipsw tool (specifically ipsw dyld fixup ) can parse the chains and output a "fixed" Mach-O where the __DATA segment contains valid pointers relative to the file base.
  • dyld_shared_cache_util : Apple's own tool (if available) or open-source equivalents must be used to extract and fixup binaries from the DSC.
  • IDA Pro / Ghidra: Modern versions automatically detect LC_DYLD_CHAINED_FIXUPS and apply the fixups in the database view, but the underlying file on disk remains "chained."

8.3 Cryptexes (Cryptographic Extensions)

The introduction of the Signed System Volume (SSV) in macOS Big Sur solved the problem of persistence; by rooting the filesystem trust in a cryptographic hash verified by iBoot, Apple ensured that the OS partition was immutable. However, this immutability introduced a significant logistical problem: patching a single binary (e.g., WebKit or dyld ) required a full OS update, a re-hashing of the entire volume, and a new signature from Apple’s Taming Server (TSS).

To resolve this tension between security and agility, Apple introduced Cryptexes (Cryptographic Extensions). A Cryptex is a cryptographically sealed, versioned filesystem image that is grafted onto the system hierarchy at boot or runtime. In the Tahoe architecture, the Cryptex subsystem is the primary mechanism for the "Split OS" design, decoupling the core kernel/BSD environment from the rapidly evolving userland frameworks.

8.3.1 The "Split OS" Architecture: /System/Cryptexes/OS

In modern macOS, the root filesystem ( / ) is essentially a skeleton. It contains configuration files and the kernel, but the vast majority of executable code—including dyld , libSystem , and the frameworks—resides within the OS Cryptex .

The Image4 Container:
A Cryptex is distributed as an Image4 ( img4 ) container.

  • Payload ( IM4P ): A disk image (typically APFS or HFS+) containing the directory hierarchy.
  • Manifest ( IM4M ): The signature. Crucially, the OS Cryptex is often Personalized . It is signed by TSS with the device's ECID, binding the specific version of the userland libraries to the specific silicon, preventing "library replay" attacks where a user might try to load a vulnerable dyld from an older OS version on a newer kernel.

The Mounting Process ( apfs_graft ):
The kernel does not mount Cryptexes using standard mount syscalls. It uses a specialized mechanism within the APFS kext known as Grafting .

  1. iBoot Handoff: For the OS Cryptex, iBoot loads the image into memory (or identifies its location on NAND) and verifies the IM4M signature against the hardware root of trust.
  2. Trust Cache Extraction: The Cryptex contains a wrapper/trustcache file. This is a binary blob containing the CDHashes of every executable inside the Cryptex.
  3. TXM Ingestion: Before the kernel grafts the filesystem, it extracts this Trust Cache and passes it to the Trusted Execution Monitor (TXM) via GENTER . The TXM verifies the signature of the Trust Cache itself. If valid, these hashes are added to the Static Trust Cache in GL1 memory.
  4. Grafting: The kernel calls apfs_graft . This stitches the Cryptex filesystem into the Virtual File System (VFS) namespace.
    • The OS Cryptex is grafted at /System/Cryptexes/OS .
    • Firmlinks: To maintain POSIX compatibility, the system uses APFS Firmlinks. When a process accesses /usr/lib/libSystem.B.dylib , the VFS transparently redirects the lookup to /System/Cryptexes/OS/System/Library/Frameworks/... .

RE Focus: The "Real" Path:
For reverse engineers, this abstraction can be confusing.

  • dyld : The dynamic linker itself lives in the Cryptex. The binary at /usr/lib/dyld on the SSV is often a shim or a hardlink to the Cryptex mount.
  • Debugging: When attaching a debugger to launchd or early boot processes, path resolution may return the /System/Cryptexes/... path. Scripts that rely on hardcoded paths to /System/Library must be updated to handle this redirection.

8.3.2 Rapid Security Response (RSR): Patching via Overlay Mounts

The Cryptex architecture enables Rapid Security Response (RSR) , allowing Apple to ship security fixes (e.g., for a WebKit zero-day) without a full OS update.

The Patch Cryptex:
An RSR update is essentially a "Sparse Cryptex." It does not contain a full OS; it contains only the binaries that have changed.

The Overlay Mechanism:
When an RSR is applied:

  1. cryptexd : The system daemon cryptexd (running as root) manages the staging of the update.
  2. Verification: The RSR Cryptex is verified by the TXM. Since RSRs are often generic (not personalized to ECID to allow faster distribution), the TXM verifies them against a specific "RSR Root" or a sub-CA in the Apple PKI.
  3. Union Mounting: The kernel mounts the RSR Cryptex over the existing OS Cryptex mount point using a union-like filesystem strategy.
    • If a file exists in the RSR Cryptex (e.g., WebKit.framework ), the VFS layer serves that file.
    • If it does not, the VFS falls back to the base OS Cryptex.

The "Antipatch" and Reversibility:
A critical security requirement for RSR is reversibility. If a security patch causes a boot loop, the system must be able to recover.

  • The Antipatch: RSRs include logic to revert the state.
  • Pre-Boot Authentication: Because the RSR modifies the executable code of the system, its presence must be attested to during boot. The state of "RSR Applied" vs. "Base OS" is managed by LocalPolicy and verified by iBoot/LLB. If the boot fails repeatedly, iBoot can discard the RSR overlay and boot the "Known Good" base OS Cryptex, effectively removing the patch.

Security Implications for the Researcher:

  • Atomicity: The RSR update is atomic. You cannot have a "half-patched" system. The TXM ensures that either the full RSR Trust Cache is loaded, or none of it is.
  • Persistence: RSRs do not modify the SSV. They persist in the Preboot volume and are re-grafted at every boot.
  • Analysis: To analyze an RSR patch, one cannot simply diff the filesystem. One must extract the RSR Cryptex ( .dmg inside the update payload), mount it, and compare the binaries against the base OS Cryptex. The "delta" is the patch.

9.0 The Security Daemon Hierarchy

While the kernel and the hardware monitors (SPTM/TXM) enforce the immutable laws of the system physics (memory protections, page table integrity), the complex, mutable business logic of macOS security is delegated to a hierarchy of userland daemons. These daemons operate with high privileges, often holding special ports or entitlements that allow them to influence kernel policy. For the reverse engineer, these daemons represent the "Policy Engine" of the OS—and historically, the most fertile ground for logic bugs and sandbox escapes.

9.1 amfid (Apple Mobile File Integrity Daemon)

The Apple Mobile File Integrity Daemon ( amfid ) is the userland arbiter of code execution policy. While the TXM (GL1) handles the cryptographic verification of platform binaries and the enforcement of the Static Trust Cache, it lacks the context to evaluate the complex web of third-party provisioning profiles, developer certificates, and MDM constraints.

In the Tahoe architecture, amfid functions as the Policy Decision Point (PDP) for third-party code, while the kernel and TXM act as the Policy Enforcement Points (PEP) .

9.1.1 The Interaction between launchd , the Kernel (MACF), and amfid

amfid does not poll for binaries; it is interrupt-driven by the kernel via the Mandatory Access Control Framework (MACF) hooks.

The Bootstrap Race:
amfid is a critical system daemon launched by launchd early in the boot process. Because amfid is responsible for verifying signatures, it presents a bootstrap paradox: Who verifies amfid ?

  • The Solution: amfid is a platform binary located in the OS Cryptex . Its CDHash is present in the Static Trust Cache loaded by iBoot.
  • TXM Verification: When launchd spawns amfid , the kernel consults the TXM. The TXM finds the hash in the immutable static cache and blesses the pages immediately. No upcall is required.

The Verification Upcall (The "Slow Path"):
When a user launches a third-party application (e.g., /Applications/Calculator.app ), the flow traverses the boundary between Kernel and Userland multiple times.

  1. The Hook: The kernel executes execve . The MACF hook mpo_vnode_check_signature is triggered in AppleMobileFileIntegrity.kext .
  2. TXM Miss: The kernel queries the TXM via GENTER . The TXM checks the Static Trust Cache and the Dynamic Trust Cache. For a newly launched third-party app, this lookup fails.
  3. The Upcall: The kernel constructs a Mach message containing the file's path, the CDHash, and the detached signature blob (if applicable). It sends this message to the HOST_AMFID_PORT (Special Port 18).
  4. amfid Processing: amfid receives the message. It performs the heavy lifting of parsing the CMS blob, validating the certificate chain (via IPC to trustd ), and checking provisioning profiles.
  5. The Verdict: amfid returns a boolean verdict to the kernel via host_set_exception_ports or a dedicated MIG reply.
  6. TXM Update: If amfid approves the binary, the kernel issues a second GENTER call to the TXM, instructing it to add the validated CDHash to the Dynamic Trust Cache .
    • Security Note: In Tahoe, the TXM likely verifies the signature blob again independently to ensure the kernel/amfid didn't lie about the cryptographic validity, but it relies on amfid for the policy decision (e.g., "Is this developer ID allowed on this system?").

RE Focus: The MIG Interface:
The communication interface is defined in mfi_server.defs (reverse engineered). The key routine is verify_code_directory .

  • Input: audit_token_t , mach_port_t file_port , off_t offset , int64_t file_size .
  • Attack Surface: Malformed Mach messages sent to amfid 's service port can trigger type confusion in the daemon. However, amfid checks the sender's audit token to ensure requests originate only from the kernel ( kernel_task ).

9.1.2 Validating Code Directories (CDHash), Entitlements, and Provisioning Profiles

The core logic of amfid resides in its ability to link a binary's Code Directory (CD) to a valid Provisioning Profile . This is the mechanism that enforces the "Walled Garden" on iOS and the Notarization requirements on macOS.

The Validation Logic ( MISValidateSignatureAndCopyInfo ):
amfid links against libmis.dylib (Mobile Installation Service), which exports the symbol MISValidateSignatureAndCopyInfo . This function is the heart of the check.

  1. CDHash Extraction: amfid reads the LC_CODE_SIGNATURE load command from the binary. It hashes the Code Directory slots to compute the CDHash.
  2. Entitlement Extraction: It parses the embedded entitlements (XML/plist) from the signature blob.
  3. Profile Correlation: If the binary is signed by a developer certificate (not Apple), amfid looks for an embedded embedded.mobileprovision file.
    • PKCS#7 Verification: It verifies the signature of the profile itself (signed by Apple).
    • Allowlist Check: It compares the entitlements requested by the binary against the Entitlements dictionary in the profile.
    • Device Check: For development profiles, it verifies the device's UDID is present in the ProvisionedDevices array.
  4. Constraint Enforcement:
    • Restricted Entitlements: Certain entitlements (e.g., com.apple.private.security.no-sandbox ) are "Restricted." They can only be granted if the provisioning profile is signed by a specific Apple internal certificate chain. amfid enforces this hierarchy.

The "Union" of Trust:
On macOS, amfid also interfaces with the Ticket system (Notarization).

  • It checks if a valid Notarization Ticket is stapled to the binary or present in the system's ticket database ( /var/db/SystemPolicy ).
  • If a ticket is found, amfid treats the binary as "Apple-Notarized," relaxing certain constraints compared to a purely ad-hoc signed binary.

RE Focus: libmis.dylib Reversing:
This library is heavily obfuscated and stripped. However, analyzing the failure paths of MISValidateSignature... is crucial for understanding why a specific binary is being killed.

  • Return Codes: The function returns an integer status. 0 is success. Other values map to specific failures (e.g., MIS_PROFILE_EXPIRED , MIS_ENTITLEMENT_MISMATCH ).
  • Logging: amfid writes detailed failure reasons to the system log ( os_log ), but often only if a specific boot-arg or configuration profile is installed ( AmfiDeveloperMode ).

9.1.3 Exception Handling: How get-task-allow and Debugging Entitlements are Processed

One of amfid 's most critical roles is gating access to process debugging. The ability to attach a debugger ( task_for_pid ) is effectively a full compromise of the target process.

The get-task-allow Entitlement:
In a standard production environment, a process cannot be debugged unless it possesses the com.apple.security.get-task-allow entitlement.

  • Xcode Builds: When you build and run an app from Xcode, it signs the binary with a development certificate and injects this entitlement.
  • App Store Builds: The App Store distribution process strips this entitlement.

The amfid Decision Matrix:
When the kernel parses a binary, it extracts the entitlements and passes them to amfid . amfid validates that the get-task-allow entitlement is permitted by the provisioning profile.

  • Valid: amfid tells the kernel to set the CS_GET_TASK_ALLOW flag in the process's cs_flags (Code Signing Flags).
  • Invalid: amfid strips the entitlement or rejects the signature entirely.

The task_for_pid Check:
When debugserver (or lldb ) calls task_for_pid(target_pid) , the kernel checks the target's cs_flags .

  • If CS_GET_TASK_ALLOW is set: Access granted (subject to root/procmod checks).
  • If CS_GET_TASK_ALLOW is clear: Access denied ( KERN_FAILURE ).

Developer Mode (The Tahoe Shift):
In the Tahoe architecture, the mere presence of the entitlement is insufficient. The system must be in Developer Mode .

  1. State Check: amfid queries the TXM to determine if Developer Mode is active.
  2. Enforcement: If Developer Mode is disabled , amfid will reject the execution of any binary signed with a development certificate, even if the profile is valid. This prevents an attacker from side-loading a malicious "debuggable" app onto a user's device to inspect memory, unless the user has explicitly lowered the device's security posture via the TXM-mediated reboot ceremony.

Unrestricted Debugging (SIP Disable):
On macOS, disabling System Integrity Protection (SIP) historically allowed unrestricted debugging ( csrutil disable ).

  • Legacy: The kernel checked the CSR_ALLOW_UNRESTRICTED_DEBUGGING bit in the NVRAM boot-args.
  • Tahoe: The SIP state is managed by the LocalPolicy and enforced by the TXM . amfid retrieves this policy state. If SIP is disabled, amfid relaxes the signature validation logic, effectively telling the kernel "Everything is valid," allowing the execution of unsigned code and the attachment of debuggers to system processes (except those protected by Exclaves).

9.2 sandboxd & The Seatbelt Policy

If amfid is the bouncer checking IDs at the door, the Seatbelt subsystem (marketed as App Sandbox) is the straightjacket strapped to the guest once they are inside. Originating from the TrustedBSD project, the macOS Sandbox is a Mandatory Access Control (MAC) mechanism that restricts a process's access to resources—files, network sockets, Mach ports, and IOKit drivers—regardless of the user's UID.

In the Tahoe architecture, the Sandbox has evolved from a path-based filter into a semantic, metadata-driven enforcement engine that is tightly coupled with the kernel's VFS layer and the new Data Vault primitives.

9.2.1 Compiling SBPL (Sandbox Policy Language) to Byte Code

The definition of a sandbox profile is written in SBPL (Sandbox Policy Language) , a Scheme-like (LISP) dialect. However, the kernel does not contain a LISP interpreter. The translation from human-readable policy to kernel-enforceable logic is the responsibility of sandboxd and the libsandbox.dylib runtime.

The Compilation Pipeline:

  1. Profile Selection:

    • App Store Apps: The system applies the generic container profile.
    • System Daemons: Daemons specify their profile name in launchd.plist (e.g., com.apple.syslogd.sb ).
    • Platform Profile: In modern macOS, individual SBPL files for system services are increasingly deprecated in favor of a monolithic Platform Profile embedded directly into the Boot Kernel Collection . This reduces the attack surface by removing the need to parse text files at runtime.
  2. The Compiler ( sandboxd ):
    When a process initializes the sandbox (via sandbox_init_with_parameters ), an XPC call is made to sandboxd .

    • sandboxd parses the SBPL.
    • It resolves variable expansions (e.g., ${HOME} , ${TemporaryDirectory} ).
    • It compiles the rules into a proprietary Byte Code format (often referred to as "TinyScheme" or "Filter Machine" code).
  3. The Byte Code Structure:
    The compiled blob is a serialized state machine. Reverse engineering this format reveals a graph of operations:

    • Opcodes: OP_MATCH_PATH , OP_MATCH_PATTERN , OP_CHECK_ENTITLEMENT , OP_ALLOW , OP_DENY .
    • Filters: The logic is optimized into a decision tree. For example, file operations are often grouped by path prefix to minimize evaluation time.

RE Focus: Reversing the Binary Blob:
The kernel receives this blob via the __mac_syscall interface (specifically the sandbox_set_profile command).

  • Extraction: You can intercept these blobs by hooking sandbox_check_common in the kernel or sandbox_compile_entitlements in userland.
  • Decompilation: Tools like sbs (Sandbox Scrutinizer) or custom IDA scripts can lift the bytecode back into a pseudo-SBPL format.
  • The "Profile Validation" Check: In Tahoe, the kernel performs a sanity check on the bytecode. It verifies that the profile does not contain "impossible" instructions or infinite loops that could hang the kernel thread (a DoS vector).

9.2.2 The Sandbox Kernel Extension: Hooking Syscalls via the MAC Framework

The enforcement engine is Sandbox.kext . It hooks into the XNU kernel using the Mandatory Access Control Framework (MACF) .

The MACF Hooks:
XNU is instrumented with hundreds of mac_ hooks placed at critical security bottlenecks.

  • Filesystem: mpo_vnode_check_open , mpo_vnode_check_rename , mpo_vnode_check_unlink .
  • IPC: mpo_mach_port_check_send , mpo_mach_port_check_receive .
  • IOKit: mpo_iokit_check_open_service .

The Evaluation Flow:

  1. Trigger: A sandboxed process calls open("/etc/passwd", O_RDONLY) .
  2. Hook: The kernel executes mac_vnode_check_open .
  3. Dispatch: The MAC framework iterates through registered policies. Sandbox.kext claims the hook.
  4. Credential Lookup: The kext retrieves the kauth_cred_t of the calling process. Attached to this credential is the Sandbox Label , which contains the pointer to the compiled bytecode profile.
  5. Evaluation: The Sandbox engine executes the bytecode against the request arguments.
    • Input: Operation ( file-read ), Path ( /etc/passwd ).
    • Logic: The engine traverses the decision tree.
  6. Caching (The Performance Critical Path):
    Evaluating bytecode on every syscall is prohibitively expensive. Sandbox.kext maintains a Check Cache .
    • If the decision is made, the result is cached in the vnode 's label or a per-process cache.
    • Subsequent accesses to the same file bypass the bytecode engine and use the cached verdict.
    • RE Vulnerability: Cache invalidation bugs are critical. If a file is moved or permissions change, but the Sandbox cache remains stale, enforcement can be bypassed.

The Tahoe/SPTM Intersection:
While the Sandbox policy is software-defined, the integrity of the hooks is hardware-enforced.

  • Immutable Hooks: The function pointers in the mac_policy_conf structure (which point to Sandbox.kext functions) reside in XNU_DATA_CONST .
  • SPTM Enforcement: If a rootkit attempts to unhook the Sandbox by overwriting these pointers, the SPTM will panic the system, as XNU_DATA_CONST is read-only to EL1. This prevents the classic "unhooking" attacks used by older jailbreaks.

9.2.3 Containerization: Data Vaults and Group Containers

In the Tahoe era, Apple has moved beyond simple path-based rules (which are fragile and prone to symlink attacks) toward Semantic Containerization .

Data Vaults:
Data Vaults are the "Nuclear Option" for privacy. They protect sensitive user data (e.g., Messages, Photos, Health) from everyone , including the root user.

  • Implementation: A Data Vault is a directory flagged with a specific VFS attribute (ACL or extended attribute).
  • Enforcement: The kernel checks for this attribute during lookup. Access is denied unless the calling process possesses a specific, restricted entitlement (e.g., com.apple.private.security.storage.AppDataVault ).
  • The "Root" Fallacy: Even if you run as root (UID 0) with host_priv , you cannot ls or cat a Data Vault. The Sandbox check happens before the DAC (Discretionary Access Control) check.
  • RE Focus: Look for rootless_check_datavault_flag in the kernel. This function verifies the entitlement against the vnode's flags.

Group Containers:
To allow IPC and data sharing between apps and their extensions (e.g., a Widget and its parent App), the Sandbox enforces Group Containers .

  • The application-group Entitlement: Defines the shared identifier (e.g., group.com.example.app ).
  • Container Manager: The system daemon containermanagerd manages the lifecycle of these directories ( ~/Library/Group Containers/ ).
  • Sandbox Logic: The profile compiler automatically injects rules allowing read/write access to ${HOME}/Library/Group Containers/<group-id> . This injection is based on the entitlements present in the binary's code signature, binding the filesystem access directly to the cryptographic identity of the executable.

If amfid validates the code's identity and sandboxd restricts the code's reach, tccd is the daemon responsible for managing the most fragile component of the security model: the user.

The Transparency, Consent, and Control (TCC) subsystem is the "User Intent Oracle." It governs access to privacy-sensitive sensors (Camera, Microphone), personal data (Contacts, Calendars, Photos), and privileged system capabilities (Full Disk Access, Screen Recording). In the Tahoe architecture, tccd has evolved from a simple prompt generator into a complex attribution engine that must defend against "Consent Hojacking" and "Attribution Spoofing."

9.3.1 The TCC Database: Schema, Integrity, and SIP

The state of user consent is persisted in SQLite databases. There is a bifurcation of state:

  • System TCC: /Library/Application Support/com.apple.TCC/TCC.db (Root-owned, system-wide permissions like Full Disk Access).
  • User TCC: ~/Library/Application Support/com.apple.TCC/TCC.db (User-owned, per-session permissions like Camera/Microphone).

The Schema:
The core of the database is the access table. For the reverse engineer, the critical columns are:

  • service : The string identifier of the privilege (e.g., kTCCServiceSystemPolicyAllFiles , kTCCServiceMicrophone ).
  • client : The bundle identifier or path of the requesting binary.
  • client_type : 0 (Bundle ID) or 1 (Absolute Path).
  • auth_value : 0 (Denied), 1 (Unknown), 2 (Allowed), 3 (Limited).
  • csreq : The Code Signing Requirement .

The csreq Blob (The Cryptographic Anchor):
TCC does not trust paths. If it did, an attacker could simply overwrite /Applications/TrustedApp.app with malware and inherit its camera permissions.
Instead, TCC stores a compiled Code Signing Requirement (CSReq) blob in the database.

  • When a process requests access, tccd obtains the process's code signature.
  • It validates the running code against the csreq stored in the database row.
  • Logic: (Current_Code_Signature) satisfies (Stored_CSReq) .

Protection Mechanisms:
While the database is a standard SQLite file, access to the file handle is heavily guarded.

  • SIP (System Integrity Protection): The System TCC database is protected by SIP. Even root cannot write to it. Only tccd (which has the com.apple.private.tcc.manager entitlement) and specific Apple-signed updaters can modify it.
  • Data Vaults: In newer macOS versions, the directory containing the database is often flagged as a Data Vault, requiring kernel-enforced entitlements just to traverse the directory structure.

RE Focus: The tccutil Fallacy:
The command-line tool tccutil is often used to reset permissions. However, it communicates with tccd via XPC. It does not touch the DB directly. Reverse engineering tccd reveals that it holds an exclusive lock on the DB handle, and any attempt to modify the DB file directly (e.g., via a bypass) usually results in database corruption or the daemon crashing and reloading from a backup, due to WAL (Write-Ahead Log) integrity checks.

9.3.2 The Attribution Chain: Determining Who is Asking

The hardest problem tccd solves is Attribution . When a user clicks a button in an app, and that app asks a daemon to take a photo, who is the "Client"? The App? Or the Daemon?

The XPC Audit Token:
When a message arrives at tccd (via com.apple.tccd.system or com.apple.tccd ), the primary identifier is the Audit Token attached to the Mach message trailer.

  • tccd parses the token to retrieve the PID, UID, and GID.
  • It uses csops (or the kernel-backed CS_VALID flag) to ensure the PID maps to a valid, signed binary.

The "Responsible Process" Problem:
Consider Terminal.app running a shell script that calls curl . If curl asks for network access, the user should see "Terminal.app would like to access...", not "curl".
To handle this, TCC supports Attribution Chains .

  1. responsible_pid : A process can designate another process as responsible for its actions (using launchd attributes or specific XPC flags).
  2. Validation: tccd verifies the relationship. It checks if the requesting process is a child of the responsible process, or if the responsible process possesses the com.apple.private.tcc.allow entitlement which permits it to proxy requests.

The Access Object:
Internally, tccd creates a TCCAccessRequest object.

  • Subject: The process requesting access.
  • Accessor: The process that will actually touch the resource (often the same as Subject).
  • Attributor: The process to be displayed in the UI.

RE Focus: The tccd State Machine:
By analyzing the service_listener function in tccd , one can map the decision logic:

  1. Entitlement Check: Does the client have com.apple.private.tcc.allow for this service? If yes, Grant immediately (no prompt).
  2. Database Lookup: Does a row exist in TCC.db ?
    • If yes, validate csreq . If valid, return auth_value .
  3. User Prompt: If no record exists, construct a UserNotification .
    • Secure UI: The prompt is not drawn by tccd . It is handed off to UserNotificationCenter (and CoreServicesUIAgent ). This prevents the requesting app from drawing over the prompt (Clickjacking) because the prompt window resides in a higher window level (Shield Window) managed by WindowServer.

9.3.3 RE Focus: XPC Attacks against TCC Endpoint Validation

For the vulnerability researcher, tccd is a prime target. If you can trick tccd into believing you are Finder.app , you gain Full Disk Access.

Attack Vector 1: XPC Injection:
Many apps load bundles or plugins. If an app with TCC permissions (e.g., a Video Editor with Camera access) loads a malicious dylib (via dylib hijacking or unsafe dlopen ), that dylib runs inside the privileged process.

  • TCC's View: tccd sees the PID of the Video Editor. It grants access.
  • Mitigation: Hardened Runtime and Library Validation (enforced by dyld and TXM) prevent loading unsigned dylibs. However, if the host app has the com.apple.security.cs.disable-library-validation entitlement, this vector is wide open.

Attack Vector 2: The "Fake" Attributor:
Attackers attempt to spoof the XPC dictionary sent to tccd .

  • The XPC message often contains a TCCAccessRequest dictionary.
  • Historically, researchers found they could inject a fake target_token into this dictionary.
  • Current Hardening: tccd now ignores user-supplied tokens in the XPC payload for most checks. It relies almost exclusively on the kernel-supplied Audit Token from the IPC trailer, which cannot be spoofed from userland.

Attack Vector 3: Semantic Confusion (The "Open" Trap):
An attacker uses the open command or AppleEvents to coerce a privileged app to perform an action.

  • Example: Telling Terminal to execute a script.
  • Mitigation: macOS introduced "Automation" permissions ( kTCCServiceAppleEvents ). App A cannot send AppleEvents to App B unless explicitly authorized by the user. tccd enforces this by intercepting the AppleEvent manager's IPC.

The "Tahoe" Impact:
On Apple Silicon, the TXM plays a subtle but vital role here. tccd relies on the Code Signing flags ( CS_VALID , CS_HARD , CS_KILL ) to trust the client. In Tahoe, these flags are ultimately enforced by the TXM.

  • If an attacker manages to patch the kernel to set CS_VALID on a malicious binary, the SPTM will still refuse to map the pages as executable if the TXM hasn't blessed the CDHash.
  • Therefore, tccd 's reliance on code signing is anchored in the hardware root of trust. A kernel compromise does not trivially allow bypassing TCC, because the attacker cannot easily forge the immutable identity of a system binary that tccd trusts.

10.0 User Session, Authentication & Data Protection

The transition from the system bootstrap phase to the interactive user session represents a critical boundary crossing. Up to this point, the system has operated primarily in the Startup domain, managed by the root launchd context. The instantiation of a user session requires the creation of a new security context—the Audit Session —and the decryption of user-specific cryptographic material anchored in the Secure Enclave.

In the Tahoe architecture, this process is not merely a check of a password hash against a file. It is a hardware-mediated ceremony involving the unwrapping of the Secure Token , the derivation of the User Keybag , and the establishment of a Kerberized identity that binds the user's biological presence (Biometrics) to their digital authority.

10.1 loginwindow & opendirectoryd

The graphical login experience is orchestrated by two primary userland daemons: loginwindow , which manages the session lifecycle and UI, and opendirectoryd , which serves as the abstraction layer for authentication and identity services. While these components are legacy codebases dating back to NeXTSTEP, their internals have been aggressively refactored to support the hardware-backed security model of Apple Silicon.

10.1.1 The Audit Session ID (ASID) and Kernel Tracking

In XNU, the concept of a "User" is tracked via the Audit Session ID (ASID) . This is distinct from the UNIX UID/GID. The ASID identifies a specific login instance (e.g., the graphical console session vs. an SSH session).

The setaudit_addr Syscall:
When loginwindow successfully authenticates a user, it does not simply setuid . It invokes the setaudit_addr syscall (wrapped by libbsm ).

  • Kernel Structure: This syscall populates the audit_token_t structure associated with the process.
  • Immutability: Once an ASID is set for a process, it is inherited by all children. Crucially, XNU enforces that a process cannot change its ASID once set (with very specific exceptions for sshd and loginwindow holding the com.apple.private.audit.session.create entitlement).

The Security Implications of ASID:
The ASID is the primary signal used by tccd and WindowServer to determine "Console Ownership."

  • TCC: When a process requests Camera access, tccd checks if the requesting process's ASID matches the ASID of the active graphical session. If it matches, the user is prompted. If it is a background session (e.g., SSH), the request is auto-denied or routed differently.
  • WindowServer: Only processes belonging to the active Console ASID are permitted to connect to the WindowServer and draw on the screen.

RE Focus: The security_authtrampoline :
Historically, the transition from the loginwindow context (root) to the user context involved a setuid binary called security_authtrampoline . In modern macOS, this logic is largely internalized within loginwindow 's interaction with launchd .

  • loginwindow calls xpc_session_create .
  • This triggers launchd to spawn a new instance of itself (the User launchd ) running as the target user.
  • This User launchd becomes the root of the user's process tree.

10.1.2 loginwindow : The Shield Window and Session State

loginwindow (located at /System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow ) is the session leader. It is responsible for drawing the login GUI, validating credentials, and managing the "Shield Window."

The Shield Window (Anti-Overlay):
To prevent "Clickjacking" or "Fake Login Screen" attacks where a malicious application draws a window over the login prompt to capture keystrokes, loginwindow utilizes a privileged connection to SkyLight (the WindowServer framework).

  • Window Level: The login UI is drawn at the kCGShieldingWindowLevel (or kCGCursorWindowLevel - 1). This is a Z-order reserved for the system.
  • Event Isolation: When the Shield Window is active, the WindowServer routes all keyboard and mouse events exclusively to loginwindow , regardless of what other applications might be running in the background.

The State Machine:
Reverse engineering loginwindow reveals a complex state machine driven by AppleEvents and Notificiations .

  • LoginHook / LogoutHook : While officially deprecated, the code paths for these legacy customization scripts still exist, though they are heavily sandboxed.
  • TAL (Transparent App Lifecycle): loginwindow manages the "Resume" feature (re-opening windows after reboot). This state is persisted in ~/Library/Preferences/ByHost/com.apple.loginwindow.*.plist , protected by the user's Data Protection keys.

10.1.3 opendirectoryd : The Authentication Broker

opendirectoryd is the daemon responsible for answering the question: "Is this password correct?" It is a modular daemon that loads plugins (bundles) to handle different directory services (LDAP, Active Directory, Local).

The Local Node ( /Local/Default ):
On a standalone Mac, authentication is handled by the Local Node . The data store for this node is located at /var/db/dslocal/nodes/Default/ .

  • Users: Stored as individual Property List ( .plist ) files in users/ .
  • Shadow Hash: The user's password hash is not stored in the plist. It is stored in a separate, restricted file structure known as the Shadow Hash , often embedded as binary data within the plist but stripped of read permissions for non-root users.

The Shadow Hash Data:
The ShadowHashData key in the user's plist contains a binary blob. In the Tahoe era, this blob is not a simple SHA-512 hash. It is a complex structure containing:

  1. SALTED-SHA512-PBKDF2: A standard hash for legacy compatibility (often disabled or randomized on FileVault-enabled systems).
  2. Secure Token Wrapper: The primary authentication secret. This is a blob encrypted by a key derived from the Secure Enclave .

The Verification Flow ( ODRecordVerifyPassword ):
When loginwindow passes a password to opendirectoryd :

  1. od_verify_password : The request is routed to the Local Node module.
  2. AppleSEPKeyStore Interaction: opendirectoryd cannot verify the Secure Token wrapper itself. It passes the plaintext password and the wrapped blob to the kernel extension AppleSEPKeyStore.kext .
  3. SEP Round Trip: The kernel marshals this request to the Secure Enclave via the Mailbox.
  4. Unwrapping: The SEP mixes the hardware UID with the user's password (tangling). It attempts to unwrap the blob.
  5. Verdict: If the unwrap succeeds, the SEP returns a success signal (and potentially a derived key for FileVault). If it fails, it returns an authentication error.

RE Implication:
This architecture means that offline cracking of macOS password hashes (extracted from dslocal ) is effectively impossible for FileVault-enabled users on Apple Silicon. The hash is not mathematically complete without the hardware UID of the specific SEP that created it.

10.1.4 Kerberos and the Local KDC (Heimdal)

A little-known fact about modern macOS is that every machine runs a Local Key Distribution Center (LKDC) . This is a stripped-down Heimdal Kerberos KDC running within opendirectoryd .

Why Kerberos on a Laptop?
Apple uses Kerberos to handle "Single Sign-On" between system services without passing the plaintext password around.

  1. Initial Login: When the user logs in, opendirectoryd obtains a Ticket Granting Ticket (TGT) from the Local KDC using the user's password.
  2. Credential Cache: This TGT is stored in the kernel-managed credential cache (accessible via the CCACHE server).
  3. Service Auth: When the user tries to unlock a System Preference pane, use Screen Sharing, or access the Keychain, the system requests a Service Ticket from the Local KDC.
  4. Validation: The service verifies the ticket. If valid, access is granted.

The "Smart Card" Enforcement:
In high-security environments (and increasingly the default in Tahoe), the Local KDC is backed by PKINIT .

  • If the user logs in with a Smart Card (or the built-in PIV token backed by the Secure Enclave), the KDC validates the x.509 certificate chain instead of a password hash.
  • The TXM plays a role here by enforcing that the Smart Card driver (running as a dext ) is valid and has not been tampered with, ensuring the certificate presented to the KDC is authentic.

RE Focus: Heimdal Framework:
The Heimdal.framework in the PrivateFrameworks directory contains the logic for this. Hooking krb5_get_init_creds_password in loginwindow is a common technique for Red Teams to intercept plaintext credentials before they are hashed, though this requires bypassing SIP and the hardened runtime.

10.2 Biometric Unlock (Touch ID / Face ID / Optic ID)

Biometric authentication on Apple platforms is frequently misunderstood as a replacement for the passcode. Architecturally, it is a convenience mechanism that authorizes the Secure Enclave to perform a cryptographic operation (unwrapping a key) that would otherwise require the user's manual secret entry.

In the Tahoe architecture, the biometric stack is a high-stakes interplay between userland daemons, kernel drivers, the Secure Enclave, and—in the case of Face ID and Optic ID—the Secure Neural Engine (SNE) .

10.2.1 The Daemon Hierarchy: coreauthd $\rightarrow$ biometrickitd $\rightarrow$ SEP

The userland implementation is split into two primary daemons to enforce separation of concerns: coreauthd (Policy) and biometrickitd (Mechanism).

coreauthd (The Policy Engine):
Located at /System/Library/Frameworks/LocalAuthentication.framework/Support/coreauthd , this daemon manages the Local Authentication (LA) context.

  • Context Management: When an app calls LAContext.evaluatePolicy , coreauthd creates a unique context ID ( ACMContext ).
  • ACL Evaluation: It parses the Access Control Lists (ACLs) attached to Keychain items. It determines what is required (e.g., "Biometry OR Passcode").
  • The ACM (Access Control Module): Inside coreauthd , the ACM is a library that mirrors the logic of the Secure Enclave. It prepares the request structures that will eventually be sent to the SEP.

biometrickitd (The Hardware Manager):
Located at /usr/libexec/biometrickitd , this daemon is responsible for the physical sensors.

  • Sensor Abstraction: It loads the specific plugin for the device's hardware (Mesa for Touch ID, Pearl for Face ID, Jade for Optic ID).
  • Hardware State: It manages the power state of the sensor and the emitter (e.g., the Flood Illuminator or IR Projector).
  • Data Relay: Crucially, biometrickitd does not process biometric data . It acts as a dumb pipe. It sets up a Direct Memory Access (DMA) channel or a shared memory buffer between the sensor hardware and the Secure Enclave.

The Handshake:

  1. coreauthd receives a request and validates the PID/ASID.
  2. coreauthd sends an XPC request to biometrickitd to "Arm" the sensor.
  3. biometrickitd issues an IOConnectCall to the kernel driver ( AppleBiometricSensor.kext ).
  4. The kernel driver wakes the SEP via the Mailbox and prepares the hardware interface.

10.2.2 The Hardware Path: Sensor-to-SEP Pairing

A critical security property of the biometric stack is the Hardware Pairing between the sensor and the SEP. This prevents "Evil Maid" attacks where an attacker replaces the fingerprint sensor with a malicious FPGA that replays a stored "match" signal.

The Factory Pairing:
During manufacturing, the sensor (e.g., the Touch ID button or the TrueDepth camera module) and the SEP exchange public keys. A shared secret is established and burned into the sensor's controller.

The Encrypted Channel:
When the sensor captures an image (fingerprint ridge map or IR depth map):

  1. Encryption: The sensor hardware encrypts the raw image data using the shared pairing key before it leaves the sensor module.
  2. Transport: The encrypted data travels over the SPI or MIPI bus to the Application Processor.
  3. Passthrough: The AP (kernel) writes this encrypted blob into a physical memory buffer shared with the SEP. The AP cannot decrypt this data; it sees only high-entropy noise.
  4. Decryption: The SEP reads the buffer, decrypts it using the pairing key, and processes the raw biometric data.

RE Implication:
If you hook the AppleBiometricSensor kext and dump the data buffers, you will not get a fingerprint image. You will get ciphertext. Furthermore, if you replace the sensor with a component from another iPhone (without running the Apple System Configuration tool to re-pair), the SEP will fail to decrypt the data, and biometrics will be disabled.

10.2.3 The Secure Neural Engine (SNE) & Optic ID

With the advent of Face ID and Optic ID (Vision Pro), the computational cost of biometric matching exceeded the capabilities of the general-purpose SEP core. Apple introduced the Secure Neural Engine (SNE) .

Architecture:
The Neural Engine (ANE) on the SoC is partitioned.

  • Standard Mode: Accessible to userland (CoreML) for standard ML tasks.
  • Secure Mode: A dedicated slice of the ANE is reserved for the SEP.

The Optic ID Flow (Tahoe/Vision Pro):

  1. Capture: The iris cameras capture spatiotemporally modulated IR images.
  2. Transfer: The encrypted images are DMA'd to the SEP's protected memory.
  3. SNE Handoff: The SEP sanitizes the ANE (flushing previous weights/data) and loads the Optic ID neural network model.
  4. Processing: The SEP feeds the image data into the SNE. The SNE performs the feature extraction (generating the mathematical representation of the iris).
  5. Matching: The SNE returns the feature vector to the SEP. The SEP compares this vector against the enrolled templates stored in the Secure Storage Component (xART) .
  6. Liveness: Simultaneously, the SEP analyzes the "spatiotemporal modulation" (the reaction of the pupil to light pulses) to ensure the subject is a living eye and not a contact lens or high-res display.

Memory Protection:
On M3/M4 chips, the memory used by the SNE during this process is protected by a dedicated Hardware Security Controller . If the AP attempts to read SNE memory while it is in Secure Mode, the transaction is blocked, and a system panic is triggered.

10.2.4 Secure Intent: The GPIO Hardline

For high-value transactions (Apple Pay, Smart Card pairing), biometric matching is insufficient. Malware could theoretically trick the user into looking at the phone (Face ID match) to authorize a background payment.

To solve this, Apple implements Secure Intent via a physical side-channel.

The Physical Button:
The power button (or side button) is wired to the Always-On Processor (AOP) and the AP, but it also has a dedicated GPIO line connected directly to the Secure Enclave .

The Logic:
When a transaction requires Secure Intent (flagged in the LAContext ):

  1. The SEP performs the biometric match.
  2. If successful, the SEP holds the authorization. It does not release the token.
  3. The SEP monitors its private GPIO pin.
  4. The Window: The user must double-click the physical button within a specific time window (usually < 2 seconds) of the biometric match.
  5. Release: Only when the SEP detects the electrical signal of the double-click on its private line does it sign the payment token or release the key.

RE Focus:
This mechanism renders UI spoofing attacks ineffective for Apple Pay. Even if an attacker controls WindowServer and draws a fake "Double Click to Pay" overlay, and even if the user looks at the phone (satisfying Face ID), the attacker cannot generate the electrical signal on the SEP's GPIO pin.

10.2.5 The LAC (Local Authentication Context) and Token Binding

When authentication succeeds, the SEP does not simply return true . It returns a cryptographic token.

The ACMHandle :
coreauthd receives an opaque handle from the SEP. This handle refers to a session in the SEP's memory.

Token Binding:
When accessing a Keychain item protected by kSecAccessControlUserPresence :

  1. securityd (Keychain daemon) sends the encrypted keyblob (wrapped key) and the ACMHandle to the SEP.
  2. The SEP verifies the ACMHandle is valid and associated with a recent biometric match.
  3. The SEP uses its internal hardware key to unwrap the keyblob.
  4. The SEP returns the raw AES key to securityd (or performs the crypto operation internally if the key is non-extractable).

The "Backoff" Strategy:
The SEP enforces the retry policy in hardware.

  • Counter: Stored in xART .
  • Logic:
    • 1-2 failures: Retry allowed immediately.
    • 3+ failures: Force delay.
    • 5 failures: Biometrics disabled. Passcode required.
    • Global Reset: If the device is rebooted, or if 48 hours pass, the SEP invalidates the biometric keys in memory, forcing a passcode entry to re-derive the master keys. This logic is encoded in the sepOS and cannot be patched by the kernel.

10.3 Data Protection & FileVault

On Intel Macs, FileVault was a distinct full-disk encryption layer (CoreStorage) that operated below the filesystem. On Apple Silicon, this distinction has vanished. FileVault is simply Data Protection applied to the Volume Key.

The architecture of macOS Tahoe unifies the security model of iOS and macOS. Every file on the SSD is individually encrypted with a per-file key, wrapped by a class key, protected by the Secure Enclave. "Turning on FileVault" merely changes the protection policy of the Volume Encryption Key (VEK) from "Protected by Hardware UID" to "Protected by User Password."

This section dissects the cryptographic hierarchy, the hardware-accelerated AES path that keeps keys out of RAM, and the Sealed Key Protection (SKP) mechanism that binds data availability to the system's boot measurement.

10.3.1 Unwrapping the User Keybag: The Class Key Hierarchy

The central data structure for Data Protection is the Keybag . It is a binary property list stored on disk (typically in /private/var/db/UserKeyStash or similar locations depending on the boot volume layout), containing the wrapped class keys.

The Hierarchy:

  1. Hardware UID: The root of trust, fused into the SEP. Accessible only to the SEP's AES engine.
  2. User Password: The user's login secret.
  3. Passcode Derived Key (PDK): The SEP mixes the UID and the User Password using a proprietary KDF (Tangling).
    $$ \text{PDK} = \text{Tangle}(\text{UID}, \text{PBKDF2}(\text{Password}, \text{Salt})) $$
  4. Class Keys: The Keybag contains the keys for specific protection classes (A, B, C, D), wrapped by the PDK.

The Classes:

  • Class A (Complete Protection): Keys are evicted from SEP memory immediately upon device lock.
  • Class B (Protected Unless Open): Keys are evicted on lock, but open file handles retain an ephemeral unwrapped key in the AES engine context to allow background writes to finish.
  • Class C (First Unlock): Keys remain in SEP memory after the first successful authentication until reboot. This is the class used for the FileVault Volume Key.
  • Class D (No Protection): Keys are wrapped only by the UID. They are accessible as long as the device is booted, even without a password. Used for system daemons that must start before login.

The Unwrapping Ceremony:
When the user logs in:

  1. loginwindow passes the password to the kernel ( AppleSEPKeyStore ).
  2. The kernel passes the password and the Keybag blob to the SEP via Mailbox.
  3. The SEP performs the Tangle operation to derive the PDK.
  4. The SEP attempts to unwrap the Class Keys in the Keybag using the PDK.
  5. Crucially: The unwrapped Class Keys are never returned to the kernel. They remain resident in the SEP's protected SRAM. The SEP returns a "Handle" (Key ID) to the kernel.

10.3.2 Sealed Key Protection (SKP): Binding Data to Measurement

In the Tahoe architecture, Apple introduced Sealed Key Protection (SKP) to mitigate "Evil Maid" attacks where an attacker boots a malicious operating system (or a vulnerable older version) to brute-force the keybag.

SKP binds the availability of the keys to the Measurement of the boot chain.

The Measurement Flow:

  1. Boot Monitor: Measures the sepOS .
  2. sepOS : Measures the KernelCache and the LocalPolicy .
  3. PCR Extension: These measurements are extended into registers within the SEP (conceptually similar to TPM PCRs).

The SKP Key:
When the Volume Encryption Key (VEK) is generated (at install time), it is wrapped not just by the UID/PDK, but by a key derived from the Current System Measurement .

$$ \text{WrappedVEK} = \text{Encrypt}(\text{VEK}, \text{KDF}(\text{UID}, \text{PDK}, \text{Measurement})) $$

The Security Guarantee:
If an attacker attempts to boot a custom kernel (Permissive Security) or a downgraded kernel to attack the Data Protection implementation:

  1. The Boot Monitor/ sepOS calculates a different measurement hash.
  2. The KDF derives a different wrapping key.
  3. The SEP fails to unwrap the VEK.
  4. The Data Volume remains inaccessible.

This forces the attacker to possess the user's credentials and boot a signed, measured, and authorized operating system to access the data.

10.3.3 The Hardware AES Engine & The "Wrapped Key" Path

A common misconception is that the kernel decrypts file data. In reality, the XNU kernel never sees the raw AES keys for file contents. Apple Silicon utilizes a dedicated AES DMA Engine that sits between the Memory Controller and the Storage Controller (ANS2).

The Inline Encryption Path:

  1. File I/O: When a user process reads a file, the kernel looks up the file's metadata to find the Per-File Key (wrapped by a Class Key).
  2. The Request: The kernel sends the Wrapped Per-File Key and the Class Key Handle to the SEP.
  3. SEP Translation: The SEP unwraps the Per-File Key using the resident Class Key.
  4. The Side Channel: The SEP does not return the raw key to the kernel. Instead, it programs the AES Engine's key registers directly via a dedicated hardware bus (or re-wraps the key with an ephemeral key known only to the AES Engine).
  5. DMA: The kernel initiates the DMA transfer from NAND.
  6. Decryption: The AES Engine decrypts the data on-the-fly as it flows into DRAM. The data lands in the page cache as plaintext, but the key that decrypted it effectively vanishes.

RE Implication:
Memory forensics on a running macOS kernel cannot recover the file encryption keys. They simply do not exist in kernel memory. To recover the keys, one must compromise the SEP firmware and dump its SRAM.

10.3.4 RE Focus: Analyzing the AppleSEPKeyStore Kernel Extension

The interface between the kernel and the SEP's key management logic is the AppleSEPKeyStore kext. This binary is a prime target for understanding the proprietary IPC protocol.

Key Symbols:

  • aks_unwrap_key : The primary function for unwrapping a keybag element.
  • aks_get_lock_state : Queries the SEP for the current lock state (Locked/Unlocked).
  • sep_key_store_client_handle_message : The demuxer for messages received from the SEP.

The Attack Surface:
The communication protocol involves complex Type-Length-Value (TLV) structures.

  • Handle Confusion: The kernel refers to keys by 32-bit integer handles. A potential vulnerability class involves guessing or forging a handle to trick the SEP into using a key belonging to a different security domain (e.g., using a System Key to unwrap User Data).
  • Race Conditions: The AppleSEPKeyStore maintains a shadow state of the lock status. Discrepancies between the kernel's view of the lock state and the SEP's actual state can lead to logic bugs where the kernel allows an operation that the SEP should have blocked, or vice versa.

Tahoe Hardening:
In Tahoe, AppleSEPKeyStore operations are increasingly gated by the TXM . Critical key operations (like enabling FileVault or changing the recovery key) require a GENTER transition to GL1 to verify the policy before the request is forwarded to the SEP, adding an additional layer of authorization check that cannot be bypassed by kernel patching.

11.0 Conclusion: The Attack Surface Landscape

The architectural transformation introduced with macOS Tahoe and the M3/M4 silicon generation signifies the end of the "Kernel is King" era. We have moved from a monolithic trust model, where uid=0 and tfp0 were the ultimate objectives, to a federated security model where the kernel is merely a highly privileged, yet strictly supervised, tenant within a hardware-enforced hypervisor.

For the vulnerability researcher, this necessitates a shift in methodology. Fuzzing syscalls is no longer sufficient to compromise the system's root of trust. The new frontier lies in the Boundary Crossings —the specific, hardware-mediated bridges that allow data and execution flow to traverse the isolated domains.

11.1 Summary of Boundary Crossings

The following matrix details the architectural boundaries, the mechanisms used to traverse them, and the specific attack surface exposed at each junction.

11.1.1 Userland (EL0) $\leftrightarrow$ Kernel (EL1)

The Traditional Boundary, Hardened by Silicon.

  • Transition Mechanism:
    • Entry: SVC (Supervisor Call) instruction triggering a synchronous exception to VBAR_EL1 + 0x400 .
    • Exit: ERET (Exception Return) restoring PC and PSTATE from ELR_EL1 and SPSR_EL1 .
  • Hardware Enforcement:
    • PAC: Entry points are signed. The kernel verifies the thread state signature ( kauth_thread_state ) on return.
    • PPL/SPTM: The kernel cannot modify its own text or page tables to disable SMEP/SMAP equivalents ( PAN / PXN ).
  • The Tahoe Shift:
    • The kernel is no longer the final arbiter of virtual memory. When a user process requests mmap(RWX) , the kernel cannot simply write to the translation table. It must request the SPTM to map the page.
    • Attack Surface:
      • Logic Bugs: Standard memory corruption (UAF, Heap Overflow) in kernel extensions still yields EL1 execution.
      • PAC Bypasses: Forging pointers to survive the ERET or function pointer authentication.
      • Argument Sanitization: The kernel must sanitize user pointers before passing them to the SPTM. A "Confused Deputy" attack where the kernel is tricked into asking the SPTM to map a privileged page into user space is the new tfp0 .

11.1.2 Kernel (EL1) $\leftrightarrow$ Secure Page Table Monitor (GL2)

The "Mechanism" Boundary: The New Hypervisor.

  • Transition Mechanism:
    • Entry: GENTER (Opcode 0x00201420 ) with Selector ID in x15 .
    • Exit: GEXIT (Opcode 0x00201400 ).
  • Hardware Enforcement:
    • GXF: Hardware context switch of SP_EL1 $\rightarrow$ SP_GL2 .
    • SPRR: Atomic switch of permission views. Kernel text becomes RO/NX; SPTM text becomes RX.
  • Data Exchange:
    • Registers: x0 - x7 carry physical addresses (PPNs) and permission bitmaps.
    • Shared Memory: None. The SPTM reads physical memory directly via its own linear map.
  • Attack Surface:
    • State Machine Confusion: The SPTM enforces a Finite State Machine (FSM) on every physical page (Frame Table). The primary attack vector is finding a sequence of retype / map / unmap calls that desynchronizes the SPTM's view of a page from the hardware's actual state (e.g., aliasing a PAGE_TABLE as XNU_DATA ).
    • Input Validation: Passing invalid physical addresses or edge-case permission combinations to sptm_map .
    • Panic-as-Oracle: Since SPTM panics on failure, timing side-channels or fault injection during the GENTER window are potential vectors to infer the memory layout of GL2.

11.1.3 Kernel (EL1) $\leftrightarrow$ Trusted Execution Monitor (GL1)

The "Policy" Boundary: The Signature Oracle.

  • Transition Mechanism:
    • Entry: GENTER (Targeting the TXM Dispatch Table).
  • Hardware Enforcement:
    • SPRR: TXM memory is visible only in GL1 context.
  • Data Exchange:
    • Physical Pointers: The kernel passes the physical address of a Code Directory or CMS blob.
    • Trust Cache: The kernel requests updates to the Dynamic Trust Cache.
  • Attack Surface:
    • Parsing Complexity: The TXM must parse complex, legacy formats (ASN.1, CMS, Mach-O headers) to verify signatures. This parsing happens in the highest privilege level. A buffer overflow in the TXM's ASN.1 parser is a "God Mode" exploit.
    • TOCTOU: The kernel loads the signature into memory, passes the physical address to TXM, and then TXM verifies it. If the kernel (or a DMA device) can modify that memory during the TXM's verification window, the signature check can be bypassed.
    • Logic Downgrades: Tricking the TXM into believing the device is in "Developer Mode" or "Permissive Mode" via corrupted NVRAM or LocalPolicy parsing.

11.1.4 Kernel (EL1) $\leftrightarrow$ Secure Enclave (SEP)

The "Air Gap" Boundary: The Parallel Computer.

  • Transition Mechanism:
    • Asynchronous IPC: Mailbox Registers (Doorbell) + Shared Memory Buffers (DART-mapped).
  • Hardware Enforcement:
    • Physical Isolation: Distinct CPU core, distinct MMU.
    • Memory Protection Engine: SEP memory is encrypted/authenticated inline.
  • Data Exchange:
    • Serialized Messages: L4 IPC format (Endpoints, TLV payloads).
    • Wrapped Keys: Keys are passed as opaque blobs; raw key material never crosses this boundary.
  • Attack Surface:
    • Message Parsing: Fuzzing the sepOS endpoint handlers (e.g., biometrickitd , securekeyvault ).
    • Shared Memory Races: Modifying the contents of a DART-mapped buffer after the SEP has validated the header but before it processes the payload.
    • Anti-Replay Logic: Attempting to rollback the xART storage state to force the SEP to reuse old nonces or counters.

11.1.5 Kernel (EL1) $\leftrightarrow$ Exclaves (Secure Domain)

The "Microkernel" Boundary: The RingGate.

  • Transition Mechanism:
    • RingGate: XNUProxy kext marshals data $\rightarrow$ GENTER (to Secure Kernel) $\rightarrow$ IPC to Conclave.
  • Hardware Enforcement:
    • SPTM: Enforces physical memory isolation between XNU_DOMAIN and SK_DOMAIN .
  • Data Exchange:
    • Tightbeam: A strongly-typed IDL serialization format.
  • Attack Surface:
    • Proxy Confusion: Exploiting XNUProxy to route messages to the wrong Conclave.
    • IDL Deserialization: Bugs in the Tightbeam generated code within the Exclave.
    • Resource Exhaustion: Flooding the Secure Kernel with Downcalls to starve secure workloads (DoS).

11.2 The "Intel Gap": Security Disparities between x86 and Apple Silicon

While macOS Tahoe presents a unified user experience across architectures, the underlying security reality is a tale of two operating systems. On Apple Silicon, macOS is a hypervisor-managed, hardware-attested fortress. On Intel (x86_64), it remains a traditional monolithic kernel relying on legacy protection mechanisms. This divergence has created a massive "Intel Gap"—a disparity in exploit mitigation so severe that the same vulnerability often yields a trivial root shell on Intel while resulting in a harmless panic on Apple Silicon.

For the reverse engineer, understanding this gap is essential for targeting. The Intel architecture represents the "Soft Target," lacking the silicon-enforced boundaries of the SPTM, TXM, and PAC.

11.2.1 The Absence of Lateral Privilege (Ring 0 is King)

The most profound difference lies in the privilege hierarchy.

  • Apple Silicon: As detailed in Section 4.0, the kernel (EL1) is deprivileged. It is subservient to the SPTM (GL2) and TXM (GL1). Achieving arbitrary kernel read/write (KRW) is merely the first step in a long chain of exploitation. The attacker must still bypass the hardware-enforced page table protection and code signing logic.
  • Intel x86: The kernel runs in Ring 0 (Supervisor Mode). There is no "Ring -1" hypervisor enforcing integrity on the host macOS kernel. Apple never implemented a Windows VBS/HVCI-equivalent on Intel Macs.
    • Consequence: On Intel, KRW is Game Over. If an attacker can write to kernel memory, they can disable SIP, unhook the Sandbox, patch amfid , and overwrite page tables directly. There is no hardware entity above Ring 0 to say "No."

11.2.2 Static vs. Dynamic Kernel Integrity (KTRR vs. SPTM)

Both architectures attempt to enforce Kernel Text Read-Only Region (KTRR) , but the implementation differs fundamentally in flexibility and robustness.

  • Intel (Hardware KTRR): On recent Intel Macs, KTRR is implemented via proprietary memory controller registers (configured via MSR ).
    • Mechanism: The firmware locks a physical range of memory as Read-Only/Executable.
    • Limitation: This is Static . Once the range is locked at boot, it cannot change. This forces the kernel to fit all immutable code into a contiguous block. It cannot protect dynamically loaded drivers (KEXTs) with the same hardware rigor. KEXTs rely on software-managed page tables ( CR0.WP bit), which a compromised kernel can disable.
  • Apple Silicon (SPTM):
    • Mechanism: The SPTM manages the Frame Table.
    • Advantage: This is Dynamic . The kernel can load a new extension (AKC), link it, and then ask the SPTM to "Seal" it. The SPTM transitions those specific pages to XNU_TEXT . This allows the "Immutable Kernel" coverage to extend to late-loaded drivers, a feat impossible on the static Intel KTRR implementation.

11.2.3 The CFI Chasm: PAC vs. CET

Control Flow Integrity (CFI) is the primary defense against ROP/JOP.

  • Apple Silicon: Pointer Authentication (PAC) is ubiquitous. It protects return addresses (stack), function pointers (heap/data), and C++ vtables. It provides cryptographic diversity based on pointer context.
  • Intel x86: Intel Macs support Control-flow Enforcement Technology (CET) , specifically Shadow Stacks ( IBT support is limited).
    • The Gap: CET Shadow Stacks protect return addresses effectively, but they do not protect Forward-Edge transfers (function pointers) with the same granularity as PAC.
    • Data Pointers: Crucially, Intel has no equivalent to APDAKey (Data Key). An attacker on Intel can still perform Data-Oriented Programming (DOP) —swapping valid object pointers or corrupting decision-making flags—without triggering a hardware fault. On Apple Silicon, these pointers are signed; forging them requires a signing gadget.

11.2.4 The Root of Trust: T2 vs. On-Die Boot ROM

The boot chain trust anchor differs physically.

  • Intel: The Root of Trust is the Apple T2 Security Chip (on models 2018-2020).
    • The Weakness: The T2 is a discrete bridge. It verifies the boot.efi and kernelcache signature before the Intel CPU starts. However, once the Intel CPU is executing, the T2 is effectively a peripheral connected via USB/PCIe. It cannot introspect the Intel CPU's execution state. It cannot stop a runtime kernel exploit.
  • Apple Silicon: The Root of Trust is the AP Boot ROM .
    • The Strength: The security logic (SEP, PKA, Boot Monitor) is on the same die . The Secure Enclave can monitor the power and clock lines of the AP. The SPTM (running on the AP) enforces the boot measurements continuously. The trust chain is not "handed off"; it is maintained throughout the runtime lifecycle.

11.2.5 I/O Security: VT-d vs. DART

DMA attacks are a classic method to bypass CPU memory protections.

  • Intel: Uses VT-d (Intel Virtualization Technology for Directed I/O).
    • Configuration: The kernel configures the IOMMU tables.
    • Vulnerability: If the kernel is compromised, it can reconfigure VT-d to allow a malicious Thunderbolt device to overwrite kernel memory (unless strict "DMA Protection" is enabled and locked, which relies on the kernel's integrity).
  • Apple Silicon: Uses DART (Device Address Resolution Table).
    • Enforcement: As detailed in Section 7.2.2, the kernel cannot write to DART registers. Only the SPTM can map I/O memory.
    • Result: Even a compromised kernel cannot weaponize a peripheral to perform a DMA attack against the monitor or the kernel text, because the SPTM will reject the mapping request.

11.2.6 Summary Table: The Security Disparity

Feature Intel Mac (x86_64) Apple Silicon (arm64e)
Highest Privilege Ring 0 (Kernel) GL2 (SPTM)
Page Table Protection Software ( CR0.WP ) Hardware (SPRR + SPTM)
Kernel Integrity Static KTRR (Boot only) Dynamic KIP (Runtime Sealing)
CFI CET (Shadow Stack) PAC (Cryptographic Signatures)
Vtable Protection None (Plaintext Pointers) Signed Pointers ( APDAKey )
Code Signing Kernel-enforced ( AMFI.kext ) Monitor-enforced (TXM)
DMA Protection Kernel-managed VT-d Monitor-managed DART
Secure Enclave Discrete (T2 Chip) Integrated (On-Die)
Exploit Consequence Persistent Rootkit Possible System Panic / Non-Persistence

Conclusion for the Researcher:
The "Intel Gap" means that legacy Intel Macs are essentially running a different, far more vulnerable operating system, despite sharing the macOS version number. Exploits that require complex, multi-stage chains on M3 (e.g., bypassing PAC, confusing SPTM, racing TXM) can often be reduced to a single Use-After-Free and a ROP chain on Intel. As Apple phases out Intel support, the "easy mode" of macOS exploitation is rapidly vanishing.

The trajectory of macOS security architecture is not asymptotic; it is directional. Apple is not merely patching vulnerabilities in XNU; they are actively architecting its obsolescence as a security boundary. The "Tahoe" architecture provides the silicon primitives (SPTM, TXM, GL2) required to execute a long-term strategy of Architectural Attrition .

The future of macOS exploitation lies in understanding two concurrent trends: the ossification of the XNU kernel into a static, immutable appliance, and the migration of high-value logic into the opaque, hardware-isolated world of Exclaves.

11.3.1 The Deprecation of kmod_load : The Static Kernel

For decades, the ability to load Kernel Extensions (KEXTs) was a defining feature of macOS. It was also its Achilles' heel. KEXTs run at EL1, share the kernel's address space, and historically lacked the rigorous code review applied to the core kernel.

The mechanism for this—the kmod_load syscall (and the associated kmod_control traps)—represents a massive attack surface. It requires the kernel to possess a runtime linker ( kld ), capable of resolving symbols, applying relocations, and modifying executable memory.

The DriverKit End-Game:
Apple has systematically introduced userland replacements for kernel drivers: USBDriverKit , HIDDriverKit , AudioDriverKit , and NetworkingDriverKit .

  • Current State: In Tahoe, third-party KEXTs are deprecated. The userland tool kmutil manages the policy, but the actual loading still relies on the kernel's ability to link code. Loading a legacy KEXT now requires reducing system security (disabling SIP/Secure Boot) and interacting with the TXM via LocalPolicy to explicitly authorize the hash.

Future State: The Death of the Runtime Linker:
We are approaching a point where the kernel will effectively lose the ability to load dynamic code entirely in "Full Security" mode. The goal is to remove the kmod_load logic from the kernel entirely.

  • The "Sealed" Kernel: The Boot Kernel Collection (BKC) (loaded by iBoot) and the Auxiliary Kernel Collection (AKC) (loaded early by kernelmanagerd ) will be the only permitted executable kernel code.
  • Pre-Linked Immutability: By moving all linking to build-time (kernelcache generation) or boot-time (iBoot verification), Apple can strip the dynamic linker logic ( kld ) from the runtime kernel. If the kernel doesn't know how to link a Mach-O, it cannot load a rootkit.
  • SPTM Enforcement: The SPTM already enforces that XNU_TEXT is immutable. The logical next step is for the SPTM to reject any sptm_retype request that attempts to create new XNU_TEXT pages after the initial boot sealing phase is complete.

RE Implication:
The era of the "Rootkit" is ending. If you cannot introduce new code into EL1 via kmod_load , and you cannot modify existing code due to KTRR/SPTM, persistence in the kernel becomes impossible. Attackers will be forced to live entirely within data-only attacks (DOP) or move their persistence to userland (which is easier to detect) or firmware (which is harder to achieve).

11.3.2 Exclave Expansion: Eating the Monolith

If XNU is the "Insecure World," Exclaves are the "Secure World." Currently, Exclaves are used for high-sensitivity, low-complexity tasks (Privacy Indicators, Passkeys). However, the architecture is designed to scale. Apple is effectively strangling the monolithic kernel by slowly migrating critical subsystems out of EL1 and into Exclaves.

Candidates for Migration:

  1. The Network Stack ( skywalk ):
    Apple has already introduced skywalk , a userland networking subsystem. The logical evolution is to move the TCP/IP stack and packet filtering logic into an Exclave.
    • Benefit: A remote code execution vulnerability in the Wi-Fi firmware or the TCP stack would compromise an isolated Exclave, not the entire kernel. The SPTM would prevent the compromised network stack from touching system memory.
  2. Filesystem Encryption (APFS):
    Currently, AppleSEPKeyStore handles key wrapping, but the bulk encryption happens via the AES Engine managed by the kernel. Moving the filesystem driver's cryptographic logic to an Exclave would ensure that even a kernel compromise cannot exfiltrate file keys, as the keys would exist only within the Exclave's memory domain.
  3. Audio and Media Processing:
    To protect DRM content and prevent microphone eavesdropping, the entire CoreAudio engine could be moved to a "Media Conclave."

The "Dark Matter" OS:
As more logic moves to Exclaves, a significant portion of the OS execution flow becomes invisible to standard introspection tools.

  • No DTrace: You cannot DTrace an Exclave.
  • No kdebug: Kernel tracing will show a "black hole" where the request enters XNUProxy and vanishes until the result returns.
  • Opaque State: The memory of an Exclave is physically unmappable by the kernel. A kernel memory dump (coredump) will contain gaps where the Exclave memory resides.

11.3.3 The "Hollow Kernel" Hypothesis

Extrapolating these trends leads to the Hollow Kernel Hypothesis .

In this future architecture, XNU (EL1) is demoted to a Compatibility Shim . Its primary role is to:

  1. Provide POSIX system call semantics for legacy userland applications.
  2. Manage coarse-grained scheduling of CPU resources.
  3. Act as a message bus (via XNUProxy ) between userland applications and the real system logic running in Exclaves.

The Security Inversion:
In the traditional model, the Kernel protects the User. In the Hollow Kernel model, the Hardware (SPTM/TXM) protects the System from the Kernel.

  • The kernel is treated as untrusted code.
  • The TCB (Trusted Computing Base) shrinks from "The entire Kernel" to "The SPTM, TXM, and specific Exclaves."
  • A kernel compromise becomes a "Local DoS" or "Privacy Violation" rather than a "System Compromise."

11.3.4 The Visibility Gap: The End of Passive Analysis

For the reverse engineer, this shift is catastrophic for visibility.

  • Tightbeam IDL: The interface between XNU and Exclaves is defined by Tightbeam. Unlike MIG, which was relatively static, Tightbeam protocols can evolve rapidly. Reverse engineering the system will require constantly reconstructing these serialization formats.
  • The "Intel Gap" Closure: As Apple phases out Intel support completely, they will likely remove the legacy code paths in XNU that supported the "un-isolated" model. This will make the kernel source code (if still released) increasingly divergent from the binary reality running on M-series chips.
  • Hardware-Locked Debugging: Debugging an Exclave likely requires "Red" (Development) fused silicon. Researchers working on retail "Green" (Production) hardware will be effectively locked out of analyzing the internal logic of these secure subsystems, forced to treat them as black boxes and fuzz their inputs via XNUProxy .

Final Thought:
macOS is no longer just a Unix system. It is a distributed system running on a single die, governed by a hypervisor that doesn't exist in software. The kernel is dead; long live the Monitor.

Kids who own smartphones before age 13 have worse mental health outcomes: Study

Hacker News
abcnews.go.com
2025-11-22 21:11:15
Comments...
Original Article

Children, especially girls, who own smartphones before they are 13 years old may have worse mental health outcomes when they're older, a new study suggests.

The study , published Sunday in the Journal of Human Development and Capabilities, analyzed self-reported questionnaire results from more than 100,000 young adults between the ages of 18 and 24.

The questionnaire asked respondents about mental health symptoms, such as having aggression, feelings of detachment, hallucinations and suicidal thoughts. Those who were given smartphones at an earlier age were associated with worse mental health outcomes for every year of smartphone ownership before the age of 13.

PHOTO: A small child plays on a smartphone in an undated stock photo.

STOCK PHOTO/Getty Images

Early smartphone ownership was associated with feelings of lower self-image and lower self-worth in both girls and boys. Girls reported lower emotional resilience and lower confidence, while boys reported feeling less calm, less stable and less empathetic.

"The younger the child gets a smartphone, the more exposure to all this impacts them psychologically and shapes the way they think and view the world," Tara Thiagarajan, one of the study's authors, told ABC News in an emailed statement.

About 48% of young women who had smartphones by 5 or 6 years old reported having severe suicidal thoughts, compared to 28% of females who had smartphones by 13 or older.

In young men, 31% of those who had smartphones by 5 or 6 years old reported having severe suicidal thoughts and 20% of males who had smartphones by 13 or older reported having severe suicidal thoughts.

Study authors attributed the differences between young women's and young men's mental health symptoms to social media usage. Other factors that seemed to impact mental health outcomes were cyberbullying, poor sleep and poor family relationships.

The study's authors recommended restricting smartphone and social media access for kids under 13, promoting digital literacy education and corporate accountability.

"Ideally, children should not have a smartphone until age 14, and when they do get a smartphone, parents should take the time to discuss with their children how to interact on the Internet and explain the consequences of doing various things," Thiagarajan added.

ABC News' Dr. Tara Narula also said on "Good Morning America" Monday that limiting kids' access to social media appears to be a key step in protecting children and their mental health outcomes.

"The longer we can push off allowing our kids to be on social media, we are learning, the better," Narula said. "I think lots of families are getting creative … landlines …. flip phones for kids [are] maybe an option so that they can have access to communicating without all the other things that come with smartphones."

The study's findings come amid an effort led by social psychologist Jonathan Haidt , author of "Anxious Generation," to limit kids' smartphone use due to the impact on their mental health. Haidt has proposed setting nationwide "norms" or guidelines, including not giving children a smartphone before high school, no social media before age 16 and establishing schools as phone-free zones.

Pediatrician Dr. Natasha Burgert also recommended that parents demonstrate to children how to use smartphones responsibly.

"Children watch everything you do -- and that doesn't stop until they leave your house," Burgert told ABC News via email. "Connect authentically and meaningfully for a few minutes every day, and show your children that the humans we live with are more important and worthy of our attention than our phones."

Popular Reads

The American Academy of Pediatrics also recommends families follow the 5 C's of media use , including teaching kids and teens how to be safe online, since content and advertisements may be targeting an older audience.

  • Child - Consider your child and their personality. What media are they drawn to and how do they react to it?
  • Content - Consider the content of the media your child consumes. Encourage them to consider good media sources.
  • Calm - Help your child learn how to manage their emotions, including without the help of media.
  • Crowding out - Consider what your family would like to spend more quality time doing, besides consuming media.
  • Communication - Discuss media with children early and often and encourage learning digital literacy.

Show HN: I built a wizard to turn ideas into AI coding agent-ready specs

Hacker News
vibescaffold.dev
2025-11-22 21:02:34
Comments...
Original Article

AI-Powered Spec Generator

Turn your messy ideas into
crystal clear specs. _

Stop writing vague prompt iterations. Generate detailed technical specifications, architecture diagrams, and agent directives from a single structured conversation.

01
02
03
04
05

function generateSpec (input)

{
// Analysis in progress...
const architecture = new SystemDesign(input);
return architecture.compile();
}

ONE_PAGER.md

DEV_SPEC.md

PROMPT_PLAN.md

AGENTS.md

=CAPABILITIES

Reason over your requirements

Product Definition

Converts abstract ideas into concrete MVP requirements and user stories.

Tech Architecture

Drafts complete schema designs, API routes, and security protocols.

Development Plan

Breaks complexity into sequential, LLM-testable prompt chains.

Agent Directives

Generates system prompts (AGENTS.md) for autonomous coding agents.

Ready to draft?

Show HN: Build the habit of writing meaningful commit messages

Hacker News
github.com
2025-11-22 20:54:35
Comments...
Original Article

smartcommit

smartcommit is an intelligent, AI-powered CLI tool that helps you write semantic, Conventional Commits messages effortlessly. It analyzes your staged changes, asks clarifying questions to understand the "why" behind your code, and generates a structured commit message for you.

Future you will thank you for deciding to use smartcommit !

smartcommit Demo

Features

  • AI-Powered Analysis : Automatically analyzes your staged git diff to understand what changed.
  • Interactive Q&A : Asks you specific, relevant questions to gather context that isn't obvious from the code alone (the "why" and "intent").
  • Multi-Provider Support :
    • OpenAI (GPT-4o) : For top-tier accuracy and performance.
    • Ollama (Llama 3.1) : Run locally and privately for free.
  • Conventional Commits : strictly enforces the Conventional Commits specification ( feat , fix , chore , etc.).
  • Beautiful TUI : A responsive, easy-to-use Terminal User Interface built with Bubble Tea .

Installation

Prerequisites

  • Go (1.21 or later)
  • Git installed and available in your PATH.
  • (Optional) Ollama installed locally if you plan to use the local model.
go install https://github.com/arpxspace/smartcommit@latest

Build from Source

  1. Clone the repository:

    git clone https://github.com/arpxspace/smartcommit.git
    cd smartcommit
  2. Build the binary:

  3. Move to your PATH (optional):

    mv smartcommit /usr/local/bin/

Use instead of git commit

To use smartcommit as your default git commit command, run:

git config --global alias.ci '!smartcommit'

Or for this repository only:

git config alias.ci '!smartcommit'

Usage:

  • To commit with smartcommit: git ci

🚀 Usage

  1. Stage your changes :

  2. Run smartcommit :

  3. Follow the TUI :

    • First Run : You'll be asked to choose your AI provider (OpenAI or Ollama) and configure it.
    • Analysis : The AI will analyze your changes.
    • Questions : Answer a few questions to provide context.
    • Review : The AI generates a commit message. You can edit it or confirm it to commit immediately.

Manual Mode

If you already know what you want to write, you can select "I already know what to write" from the main menu to open your default git editor.

⚙️ Configuration

smartcommit stores its configuration in a local file (usually ~/.smartcommit/config.json ).

Environment Variables

  • OPENAI_API_KEY : If set, smartcommit can detect this during setup and ask if you want to use it, saving you from pasting it manually.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

  1. Fork the project
  2. Create your feature branch ( git checkout -b feature/AmazingFeature )
  3. Commit your changes ( git commit -m 'feat: Add some AmazingFeature' )
  4. Push to the branch ( git push origin feature/AmazingFeature )
  5. Open a Pull Request

📄 License

Distributed under the MIT License. See LICENSE for more information.

A Reverse Engineer's Anatomy of the macOS Boot Chain and Security Architecture

Hacker News
stack.int.mov
2025-11-22 20:54:02
Comments...
Original Article

1.0 The Silicon Root of Trust: Pre-Boot & Hardware Primitives

The security of the macOS platform on Apple Silicon is not defined by the kernel; it is defined by the physics of the die. Before the first instruction of kernelcache is fetched, a complex, cryptographic ballet has already concluded within the Application Processor (AP). This section dissects the immutable hardware logic that establishes the initial link in the Chain of Trust.

1.1 The Reset Vector & Boot ROM (SecureROM)

The Apple Silicon boot process begins in a state of absolute trust, anchored by the Boot ROM (often colloquially referred to as SecureROM). This code is mask-programmed into the silicon during fabrication. It is immutable, unpatchable, and serves as the hardware root of trust for the entire platform.

1.1.1 Execution at Exception Level 3 (EL3): Analyzing RVBAR_EL3

Upon Power-On Reset (POR), the cores of the M-series SoC (and A-series) initialize in the highest privilege state defined by the ARMv8/v9 architecture: Exception Level 3 (EL3) .

While Apple does not utilize TrustZone in the traditional sense (preferring their proprietary Secure Enclave for secure world operations), the AP Boot ROM executes at EL3 to perform the necessary low-level hardware initialization that requires access to secure configuration registers that are subsequently locked or hidden from EL2/EL1.

The execution flow begins at the address defined in the Reset Vector Base Address Register (RVBAR_EL3) . On recent Apple Silicon (M1/M2/M3), the memory map places the Boot ROM at a high base address, typically 0x100000000 .

The Initial Instruction Stream:
The very first instructions executed by the silicon are responsible for establishing a sane C execution environment from a raw hardware state. A disassembly of the entry point typically reveals the following sequence:

  1. Interrupt Masking: DAIF bits are set to mask all interrupts (IRQ, FIQ, SError, Debug). The Boot ROM operates in a strictly polled mode; interrupts are nondeterministic and introduce attack surface.
  2. Cache Invalidation: The instruction and data caches are invalidated to prevent cold-boot attacks or stale data usage.
  3. Stack Setup: The Stack Pointer ( SP_EL3 ) is initialized to point to a dedicated region of on-chip SRAM (Static RAM). Note that DRAM is not initialized at this stage. The Boot ROM runs entirely within the constraints of the SoC's internal SRAM (often referred to as L2 cache-as-RAM in older exploits, though modern SoCs have dedicated boot SRAM).
  4. MMU Configuration: The System Control Register ( SCTLR_EL3 ) is written to enable the MMU, mapping the Boot ROM text as Read-Only/Executable and the SRAM stack/heap as Read-Write/No-Execute.

RE Note: Apple’s implementation of EL3 is ephemeral. Unlike Android/Qualcomm devices where EL3 hosts a persistent Secure Monitor (QSEE), Apple Silicon demotes the exception level to EL2 (Hypervisor) or EL1 (Kernel) before handing off control to the next stage. Once the Boot ROM exits, EL3 is effectively locked out, and the RVBAR_EL3 is often locked to prevent re-entry.

1.1.2 The GID Key (Group ID): Hardware-entangled Decryption

The Boot ROM's primary objective is to load the Low-Level Bootloader (LLB). However, the LLB stored on the NAND flash (or NOR SPI flash in some configurations) is encrypted. To decrypt it, the Boot ROM utilizes the GID Key (Group ID Key) .

The GID Key is a 256-bit AES key fused into the silicon during manufacturing. It is shared across all processors of the same class (e.g., all M3 Pro chips share a GID, but it differs from the M3 Max).

The "Black Box" AES Engine:
Crucially, the GID Key is not accessible via software. There is no Memory-Mapped I/O (MMIO) register that an attacker can read to dump the key. Instead, the AES engine operates as a black box:

  1. Input: The Boot ROM writes the encrypted ciphertext (the LLB payload) into the AES engine's input FIFO.
  2. Control: The Boot ROM sets a specific control bit in the AES configuration register (e.g., AES_CMD_USE_GID ).
  3. Operation: The hardware AES engine loads the GID key from the fuses directly into the crypto core, performs the decryption, and flushes the key from the internal registers.
  4. Output: The plaintext is written to the output FIFO, which the Boot ROM then reads into SRAM.

This architecture ensures that even if an attacker gains arbitrary code execution within the Boot ROM (as seen in the checkm8 exploit on older A-series chips), they cannot extract the GID key to decrypt firmware images offline. Decryption must happen on-device .

1.1.3 The Public Key Accelerator (PKA): Hardware-Enforced Verification

Decryption provides confidentiality, but not integrity. To prevent the execution of malicious firmware, the Boot ROM enforces strict code signing using the Public Key Accelerator (PKA) .

The PKA is a dedicated hardware block optimized for asymmetric cryptography (RSA and ECC). The verification flow is as follows:

  1. Root of Trust Hash: The SHA-384 hash of the Apple Root CA public key is burned into the device's eFuses. This is the immutable anchor.
  2. Manifest Parsing: The Boot ROM parses the Image4 (img4) container of the LLB. It extracts the Image4 Manifest (IM4M), which contains the payload's signature and the public key used to sign it.
  3. Key Verification: The Boot ROM hashes the public key found in the manifest and compares it against the hash in the eFuses. If they do not match, the boot halts (DRAM is never initialized, and the device enters DFU mode).
  4. Signature Verification: The Boot ROM offloads the signature verification to the PKA. It passes the SHA-384 hash of the payload and the RSA/ECC signature. The PKA performs the mathematical verification and returns a boolean result to a status register.

Fault Injection Hardening:
Modern Apple Boot ROMs employ glitch-resistant logic around the PKA check. Rather than a simple B.EQ (Branch if Equal) instruction following the PKA result—which could be bypassed via voltage glitching—the code often employs redundant checks, loop invariants, or specific register values that must be populated by the PKA hardware itself to allow the boot flow to proceed.

1.1.4 RE Focus: Dev vs. Prod Fused Silicon

For the reverse engineer, distinguishing between Development (Dev) and Production (Prod) fused silicon is vital. The behavior of the Boot ROM changes fundamentally based on the "Security Domain" fuse.

  • Production (CPFM 01): The standard consumer state. JTAG is physically disabled. The GID key is locked to the production value. The Boot ROM enforces the full Apple Root CA chain.
  • Development (CPFM 00/03): Used internally by Apple engineers.
    • JTAG Enablement: The DBGEN signal is asserted, allowing hardware debuggers (like Lauterbach or Astris) to halt the core immediately after reset.
    • Demotion: Dev-fused chips often allow "demotion," enabling the device to boot unsigned or custom-signed firmware images.
    • GID Key Variance: Dev chips often use a different GID key, meaning firmware encrypted for Prod devices cannot be decrypted on Dev hardware, and vice versa.

Identifying Silicon State:
You can identify the fuse status by querying the CHIP_ID or MOJO registers via USB when the device is in DFU mode.

  • 0x8000... usually indicates a Production fuse.
  • 0x0000... or specific bitmasks in the ECID response indicate Development/Debug fusing.

The "Un-dumpable" Region:
Once the Boot ROM prepares to jump to the next stage (LLB), it performs a lockdown sequence. It writes to the memory controller to unmap its own address range ( 0x100000000 ). Any subsequent attempt by the LLB or Kernel to read the Boot ROM address space will result in a bus error or read back zeros. This "hide-and-seek" mechanism is why dumping the Boot ROM requires a vulnerability present during the Boot ROM execution window (like checkm8 or a voltage glitch at reset).

1.2 Proprietary ISA Extensions (arm64e+)

While the M-series chips ostensibly implement the ARMv8.6-A (and increasingly ARMv9) specification, Apple has aggressively extended the Instruction Set Architecture (ISA) with proprietary logic. For the reverse engineer, standard ARM documentation is insufficient. Understanding the security posture of macOS Tahoe requires mastering these custom extensions, as they form the hardware enforcement layer for the new kernel isolation model.

1.2.1 Pointer Authentication (PAC): The Cryptographic Control Flow

Apple’s implementation of ARMv8.3-PAuth is the most pervasive security mitigation in the XNU kernel. It repurposes the unused high-order bits of 64-bit virtual addresses (typically bits 63–47, depending on Translation Control Register TCR_EL1 settings) to store a cryptographic signature, or Pointer Authentication Code (PAC).

The Key Hierarchy:
The hardware maintains five distinct 128-bit keys in system registers, accessible only at EL1 or higher. These keys segregate pointer types to limit the utility of a signing gadget:

  • APIAKey / APIBKey (Instruction): Signs code pointers (function pointers, return addresses).
  • APDAKey / APDBKey (Data): Signs data pointers. Crucial for protecting C++ vtables in IOKit ( OSObject ).
  • APGAKey (Generic): Signs arbitrary data blobs, effectively a hardware-accelerated MAC.

The AUT Failure Mechanism (Canonical Non-Valid):
For the reverse engineer analyzing crash dumps, understanding the failure mode is critical. When an AUT* instruction (e.g., AUTIA ) is executed on a corrupted or forged pointer, the CPU does not immediately raise an exception.

Instead, the hardware corrupts the pointer in a deterministic way to ensure it causes a translation fault upon dereference.

  1. Validation: The CPU recalculates the PAC.
  2. Mismatch: If the calculated PAC does not match the bits in the pointer, the CPU flips specific high-order bits (typically bit 62 for data pointers, or bit 61 for instruction pointers, depending on Top Byte Ignore settings).
  3. Result: The pointer becomes "canonical non-valid." It looks like a kernel pointer (high address) but falls into a reserved, unmapped range.
  4. Crash: The subsequent LDR or BLR triggers a Data Abort or Prefetch Abort.

RE Tip: If you see a crash where x0 is 0x007f... or 0x00ff... (a pointer with high bits set but not fully canonical), you are looking at a PAC authentication failure, not a standard NULL dereference or heap corruption.

1.2.2 Branch Target Identification (BTI): The Landing Pads

Often deployed in tandem with PAC ( -mbranch-protection=standard ), BTI mitigates Jump-Oriented Programming (JOP). It enforces a state machine on indirect branches.

  • Marking Pages: The Global Page Table (GPT) or PTE entries now include a Guarded Page ( GP ) bit.
  • The BTI Instruction: This is a "hint" instruction (NOP on older silicon). It acts as a valid landing pad.
  • Enforcement: When the CPU executes an indirect branch ( BR , BLR ) targeting a Guarded Page, the very next instruction must be a BTI instruction of the correct type ( c for call, j for jump, jc for both).

If the target is not a BTI instruction, the CPU raises a Branch Target Exception . In XNU, this manifests as a SIGILL with a specific subcode. For exploit development, this necessitates finding gadgets that not only perform the desired operation but are also preceded by a valid landing pad.

1.2.3 New in Tahoe: The Guarded Execution Feature (GXF)

This is the most significant architectural divergence in the Apple Silicon era. Standard ARM defines a vertical privilege stack (EL0 -> EL1 -> EL2). Apple has introduced Lateral Exception Levels , referred to as Guarded Levels (GL).

GXF allows the processor to switch between execution modes that share the same architectural Exception Level (EL1) but possess vastly different hardware permissions and system register views.

  • GL0: The standard XNU Kernel context.
  • GL1: The Trusted Execution Monitor (TXM) context.
  • GL2: The Secure Page Table Monitor (SPTM) context.

The Proprietary Opcodes:
Transitions are not handled by HVC or SMC . Apple added custom instructions to the ISA:

  • GENTER (Opcode 0x00201420 ): Synchronous entry into a Guarded Level. It behaves like a function call but atomically switches the hardware context (SPRR state, stack pointer, and system registers).
  • GEXIT (Opcode 0x00201400 ): Returns from GL to the caller.

Reverse Engineering the Transition:
In disassembly, GENTER takes a dispatch ID (selector) in x15 and arguments in x0-x7 . The entry point for the transition is hardcoded in the proprietary system register GXF_ENTRY_EL1 (S3_6_C15_C8_2). This register is locked early in the boot process (by iBoot), preventing the kernel from hijacking the monitor's entry vector.

1.2.4 New in Tahoe: Shadow Permission Remapping Registers (SPRR)

To enforce isolation between GL0, GL1, and GL2, Apple replaced the older APRR (Access Permission Remapping Registers) with the more robust SPRR (Shadow Permission Remapping Registers) .

In standard ARM MMUs, the Page Table Entry (PTE) bits AP[2:1] directly define Read/Write permissions. In Apple Silicon with SPRR enabled, these bits are repurposed as an index into a hardware permission table.

The Indirection Layer:

  1. PTE Index: The PTE specifies a permission index (e.g., Index 5).
  2. Context Lookup: The hardware checks the current execution mode (GL0, GL1, or GL2).
  3. Resolution: It looks up Index 5 in the SPRR_PERM_EL1 register specific to that mode.

The Security Implication:
This allows for "View-Based" memory protection.

  • Index 5 in GL2 (SPTM): Resolves to Read-Write (RW) .
  • Index 5 in GL0 (Kernel): Resolves to Read-Only (RO) .

This is how the SPTM protects page tables. The physical pages containing the translation tables are marked with a specific SPRR index. The hardware configuration for GL0 (Kernel) maps that index to Read-Only. Even if an attacker has a kernel-level arbitrary write primitive, the MMU will reject the write to the page table because the SPRR configuration for GL0 forbids it. The only way to write to that page is to execute GENTER to switch to GL2, where the SPRR configuration permits the write.

2.0 The Secure Enclave Processor (SEP): The Parallel Computer

If the Application Processor (AP) is the brain of the device, the Secure Enclave Processor (SEP) is its conscience. It is not merely a coprocessor; it is a fully independent computer-on-a-chip, sharing the same die but architecturally severed from the AP. It runs its own kernel (an Apple-customized L4 microkernel), manages its own peripherals, and holds the keys to the kingdom (UID/GID). In the Tahoe architecture, the SEP’s role has expanded from simple key management to becoming the root of authority for biometric intent and hardware attestation.

2.1 SEP Initialization & Boot

The SEP boot process is designed to be resilient against a fully compromised Application Processor. From the moment power is applied, the SEP assumes the AP is hostile.

2.1.1 The SEPROM: SRAM Execution and the Memory Protection Engine (MPE)

Like the AP, the SEP begins execution from an immutable on-die Boot ROM, the SEPROM .

The Hardware Environment:
The SEP core (typically a customized ARM core, historically evolving from Cortex-A7 derivatives) initializes in a highly constrained environment. It does not initially have access to the system's main DRAM. Instead, it executes strictly within a dedicated, on-die SRAM region. This isolation prevents early-boot DMA attacks from the AP or Thunderbolt peripherals.

The Memory Protection Engine (MPE):
As the sepOS is too large to fit entirely in SRAM, it must eventually reside in the device's main DRAM (Unified Memory on M-series). To do this securely, the SEP utilizes a hardware Memory Protection Engine (MPE) .

The MPE sits inline between the SEP core and the memory controller. It creates a cryptographic window into physical memory that is opaque to the rest of the SoC.

  1. Ephemeral Keys: Upon SEP reset, the MPE generates a random, ephemeral AES key. This key exists only in the MPE hardware registers and is never exposed to software (even sepOS ).
  2. AES-XEX Encryption: Data written by the SEP to DRAM is encrypted transparently using AES in XEX (XOR-Encrypt-XOR) mode.
  3. Authentication: The MPE calculates a CMAC tag for every cache line. This tag is stored alongside the encrypted data.

RE Implication: If you attempt to dump the physical memory range assigned to the SEP from the AP (kernel mode), you will see high-entropy noise. Furthermore, any attempt to modify a single bit of this memory via the AP will invalidate the CMAC tag. The next time the SEP reads that line, the MPE will detect the forgery and trigger a hardware panic, locking down the Enclave until a full system reset.

2.1.2 The Boot Monitor: Hardware Enforcement of OS-Bound Keys

On modern silicon (A13/M1 and later), Apple introduced the Secure Enclave Boot Monitor to mitigate the risk of Boot ROM exploits (like checkm8 ) compromising the chain of trust for key derivation.

In older architectures, the SEPROM would verify the sepOS signature and then jump to it. If the SEPROM was exploited, the attacker could jump to a malicious payload while retaining access to the hardware UID key. The Boot Monitor closes this gap by enforcing System Coprocessor Integrity Protection (SCIP) .

The Boot Flow:

  1. Payload Staging: The AP (iBoot) loads the sep-firmware.img4 payload into a region of physical memory.
  2. Mailbox Signal: The AP signals the SEP via a hardware mailbox register.
  3. Verification: The SEPROM parses the Image4 container. It verifies the signature against the Apple Root CA hash fused into the SEP's eFuses.
  4. The Handoff: Crucially, the SEPROM cannot simply jump to the loaded image. The SCIP hardware prevents execution of mutable memory.
  5. Monitor Intervention: The SEPROM invokes the Boot Monitor hardware block.
    • The Monitor resets the SEP core to a known clean state.
    • The Monitor calculates a cryptographic hash of the loaded sepOS memory range.
    • The Monitor updates the SCIP registers to permit execution of that specific range.
    • The Monitor locks the hash into a dedicated register in the Public Key Accelerator (PKA).

OS-Bound Key Derivation:
This finalized hash is the critical component. When the sepOS later requests keys (e.g., to decrypt user data), the hardware Key Derivation Function (KDF) mixes the hardware UID with this locked hash.

$$ K_{derived} = KDF(UID, Hash_{sepOS}) $$

If an attacker modifies a single byte of the sepOS (even with a Boot ROM exploit), the Boot Monitor calculates a different hash. Consequently, the KDF derives a different key, and the encrypted user data remains mathematically inaccessible. This is "Bound Security"—the data is bound not just to the device, but to a specific, signed software version.

2.1.3 Anti-Replay Mechanisms: The Integrity Tree

A classic attack vector against secure enclaves is the Replay Attack : capturing a snapshot of the encrypted RAM (e.g., when the passcode retry counter is 0) and restoring it later after the counter has incremented.

To prevent this, the SEP implements a hardware-enforced Integrity Tree (Merkle Tree).

  1. The Root of Trust: The root node of the integrity tree is stored in dedicated on-chip SRAM within the Secure Enclave complex. This memory is physically distinct from the main DRAM and cannot be addressed by the AP.
  2. Tree Structure: The protected memory region (where sepOS data and the Secure Storage Manager reside) is divided into blocks. Each block's hash is stored in a parent node, recursively up to the root.
  3. Atomic Updates: When the SEP writes to protected memory (e.g., incrementing a failed attempt counter), the MPE updates the data, recalculates the hashes up the tree, and atomically updates the root hash in the on-chip SRAM.
  4. Verification: On every read, the MPE verifies the path from the data block up to the SRAM root.

If an attacker replays an old DRAM state, the hash of the replayed block will not match the current root hash stored in the internal SRAM. The MPE detects the mismatch (Anti-Replay Violation) and halts the SEP. This mechanism ensures that the SEP has a strictly monotonic view of time and state, rendering snapshot fuzzing and counter rollbacks impossible.

2.2 SEP Runtime Architecture

Once the sepOS is bootstrapped and verified, the Secure Enclave transitions into its runtime state. At this point, it functions as a fully autonomous operating system running an Apple-customized variant of the L4 microkernel. For the reverse engineer, understanding the runtime architecture is crucial for analyzing how the SEP communicates with the hostile "Rich Execution Environment" (the AP running XNU) and how it persists sensitive state.

2.2.1 The Mailbox Interface: Analyzing the Shared Memory IPC

Communication between the Application Processor (AP) and the SEP is strictly asynchronous and interrupt-driven. Unlike the tight coupling of the SPTM (which uses synchronous instruction traps), the SEP interaction is mediated by a hardware mechanism known as the Mailbox , which relies on the proprietary Apple Interrupt Controller (AIC) to manage signaling.

The Physical Transport: AIC and Ring Buffers
There is no shared virtual memory space; the two processors exchange messages via physical memory ring buffers and hardware interrupts.

  1. The Ring Buffers (Inbox/Outbox):
    The system reserves two physical memory regions in DRAM, carved out by iBoot and described in the Device Tree (typically under the sep node as inbox-size and outbox-size ). These function as circular buffers.

    • Control Structures: Each buffer is managed by a pair of pointers (Write Head, Read Tail) stored in shared SRAM or at the start of the DRAM region.
    • Coherency: Because the SEP and AP share the same Unified Memory fabric, cache coherency is critical. The AppleSEPDriver must ensure appropriate barriers ( DMB , DSB ) are used when updating the Write Head to ensure the SEP sees the payload before the interrupt fires.
  2. The Doorbell (Apple Interrupt Controller):
    To signal a message, the sender cannot simply write to memory; it must trigger an exception on the receiver. On Apple Silicon, this is handled by the AIC , not a standard ARM GIC.

    • AP $\rightarrow$ SEP: The kernel writes to a specific AIC "Set" register (mapped in the AP's MMIO space). This asserts a hardware IRQ line wired to the SEP's core.
    • SEP $\rightarrow$ AP: When the SEP replies, it writes to its own AIC interface, which triggers a specific IRQ (often IRQ 0 or IRQ 1 relative to the SEP driver's view) on the AP. The kernel's interrupt handler ( AppleSEPDriver::interruptOccurred ) acknowledges this by writing to the AIC "Clear" register.

The L4 IPC Protocol (A7IOP):
The data payload within the ring buffers follows a serialized format, historically referred to as the Apple A7 I/O Protocol (A7IOP) , though it has evolved significantly.

  • Message Header: Every message begins with a 64-bit header containing:
    • Endpoint ID (8 bits): The destination service within the sepOS (e.g., 0x10 for Biometrics, 0x14 for Secure Storage).
    • Tag (8 bits): A transaction ID used to correlate asynchronous replies.
    • Length (16 bits): The size of the payload.
  • The AppleSEPDriver : On the XNU side, this kext manages the ring buffer logic. It serializes kernel requests (e.g., "Unlock Keybag") into the mailbox format, updates the Write Head, rings the AIC doorbell, and sleeps the calling thread on a condition variable until the SEP triggers the reply interrupt.

RE Focus: Fuzzing the Boundary
The mailbox is the primary attack surface for the SEP. Vulnerabilities here (parsing malformed messages) can lead to code execution within the Enclave.

  • Endpoint Fuzzing: The sepOS kernel dispatches messages to user-mode L4 tasks based on the Endpoint ID. Fuzzing specific endpoints (especially legacy or debug endpoints left enabled in production) is a standard methodology.
  • Shared Memory Hazards (DART): While the mailbox buffers are used for control messages (metadata), bulk data (like a firmware update or a large biometric template) is passed via DART-mapped shared memory . The AP maps a page, pins it, and passes the physical address to the SEP in a mailbox message. The SEP then maps this page into its own address space.
    • TOCTOU: Race conditions here are a classic target: modifying the data in the shared buffer after the SEP has validated the header/signature but before it processes the body. The SEP attempts to mitigate this by copying data to internal SRAM before processing, but large payloads may be processed in-place, exposing a window of opportunity.

2.2.2 The Secure Storage Component (xART): Encrypted Persistent Storage

The SEP has no internal non-volatile storage (NAND) of its own. It must rely on the AP's NAND flash to store persistent data (like the user's passcode hash, biometric templates, and credit card tokens). However, it cannot trust the AP to store this data securely.

To solve this, Apple introduced the Secure Storage Component , often referred to in firmware and kexts as xART (eXtended Anti-Replay Technology).

The Architecture:

  1. The Physical Chip: On modern devices, xART is backed by a dedicated, tamper-resistant EEPROM or NOR flash chip physically connected to the SEP via a private I2C or SPI bus. This chip is not accessible to the AP.
  2. The Logical Volume: The SEP treats the external NAND (managed by the AP) as a "dumb" block device. It encrypts its filesystem using keys derived from the hardware UID and the anti-replay counters stored in the dedicated xART chip.

The Anti-Replay Guarantee:
When the SEP writes a file (e.g., updating the failed passcode attempt counter):

  1. It encrypts the file data.
  2. It generates a new random nonce or increments a counter.
  3. It writes the encrypted file to the AP's filesystem (via the Mailbox).
  4. Crucially, it writes the hash of the file and the new counter to the dedicated xART storage chip.

When reading the file back:

  1. The SEP requests the file from the AP.
  2. It reads the expected hash/counter from the xART chip.
  3. It verifies the file against the xART record.

If the AP (or an attacker with physical access to the NAND) tries to revert the file to an older version (e.g., resetting the counter to 0), the hash will not match the record in the xART chip. The SEP will detect the rollback and reject the data. This mechanism ensures that the SEP's state is strictly monotonic and immune to external storage manipulation.

2.2.3 RE Focus: Reverse Engineering the sepOS L4 Syscall Table

For the advanced reverse engineer, the holy grail is understanding the sepOS kernel itself. Since it is based on L4, it relies heavily on synchronous IPC for system calls.

Identifying the Syscall Handler:
In the disassembled sepOS binary (which can be decrypted if you have a Boot ROM exploit for the specific device class, or by analyzing unencrypted development builds if available), the exception vector table is the starting point.

  • Look for the SVC (Supervisor Call) exception handler.
  • This handler typically branches to a dispatch table based on the immediate value in the SVC instruction or a register (often x0 or x8 ).

Mapping the Endpoints:
The sepOS is modular. It consists of the kernel and several user-mode "apps" or "tasks" running within the Enclave.

  • SEPOS Kernel: Handles memory management, scheduling, and IPC routing.
  • L4 Apps: Distinct binaries for distinct functions.
    • biometrickitd : Handles FaceID/TouchID processing.
    • securekeyvault : Manages the Keychain and Data Protection classes.
    • sigp : The Secure Enclave Signal Processor (for neural network operations).

By tracing the IPC messages dispatched from the Mailbox handler, you can map which L4 task handles which service. For example, a message with Endpoint ID 0x10 might route to the biometrickitd task. Analyzing the message parsing logic within that specific task reveals the proprietary protocol used for biometric authentication.

Tooling Note: Standard tools like IDA Pro or Ghidra require custom loaders for sepOS binaries. The memory layout is non-standard, and the binary format (Mach-O) often has stripped headers or non-standard segment protections that must be manually reconstructed based on the SCIP configuration found in the Boot Monitor logic.

3.0 The Chain of Trust: Firmware & Bootloaders

With the hardware root of trust established and the Secure Enclave operating as a parallel authority, the Application Processor begins the process of bootstrapping the mutable software stack. This phase is governed by the Image4 serialization format and a strict chain of cryptographic handover.

3.1 Low-Level Bootloader (LLB)

The Low-Level Bootloader (LLB) is the first piece of mutable code executed by the Application Processor. Loaded by the Boot ROM from the boot partition of the NAND (or NOR SPI on some development boards), it executes initially within the constraints of the SoC's SRAM. Its primary directive is architectural: it must bridge the gap between the raw silicon state and the feature-rich environment required by iBoot.

3.1.1 Parsing the Image4 ( img4 ) Container

To the reverse engineer, "firmware" on Apple Silicon is synonymous with Image4 . LLB is not a raw binary; it is encapsulated in an Image4 container, a format based on ASN.1 (Abstract Syntax Notation One) and DER (Distinguished Encoding Rules). Understanding this structure is prerequisite to any firmware analysis.

The Image4 container consists of three distinct sequences:

  1. IM4P (Payload): The actual executable code (the LLB binary).

    • Encryption: The payload is encrypted using AES-256. On production devices, this is wrapped with the GID Key . This means the payload is opaque to external analysis unless decrypted on-device (or via a GID oracle).
    • Compression: Once decrypted, the payload is typically compressed (LZSS or LZFSE).
    • Type Tag: A 4-character code (e.g., ibot , illb ) identifying the component.
  2. IM4M (Manifest): The signature and constraints, commonly known as the ApTicket .

    • The Signature: An RSA or ECDSA signature over the SHA-384 hash of the payload.
    • The Body: A set of entitlements and constraints (tags) that dictate where and how this payload can run.
    • Certificate Chain: The manifest includes the certificate chain leading back to the Apple Root CA (burned into the PKA fuses).
  3. IM4R (Restore Info): (Optional) Contains hardware-specific personalization data used during the restore process, such as the unique nonce generated by the SEP.

The Validation Logic:
When the Boot ROM loads LLB (and when LLB subsequently loads iBoot), it performs the following image4_validate routine:

  1. Parse the ASN.1 structure to separate IM4M and IM4P .
  2. Hash the IM4P (ciphertext).
  3. Locate the corresponding hash in the IM4M (under the specific tag, e.g., illb ).
  4. Verify the IM4M signature using the PKA.
  5. If valid, decrypt the IM4P using the AES engine (GID context).

3.1.2 DRAM Training and Memory Controller Configuration

The most critical hardware initialization task performed by LLB is DRAM Training .

When LLB starts, the system is running on internal SRAM (a few megabytes at most). The external LPDDR4X/LPDDR5 Unified Memory is inaccessible because the Memory Controller (MCU) is uninitialized. The physical characteristics of RAM—signal timing, voltage margins, and skew—vary slightly between every physical device due to manufacturing tolerances.

The Training Sequence:

  1. Reading SPD/Calibration Data: LLB reads calibration data from the device tree or dedicated EEPROM areas.
  2. PHY Configuration: It configures the Physical Layer (PHY) interface of the memory controller.
  3. Training Loop: LLB executes a complex algorithm that writes patterns to DRAM and reads them back, adjusting delay lines (DLLs) and drive strengths until the signal is stable.
  4. Remapping: Once training is complete, the MCU is brought online. LLB then reconfigures the Memory Management Unit (MMU) to map the vast expanse of DRAM into the address space.

RE Implication:
If you are attempting to exploit the Boot ROM or early LLB, you are constrained to SRAM. You cannot load large payloads or use heap spraying techniques that require gigabytes of memory until after LLB has successfully trained the DRAM. This creates a "choke point" for early-boot exploits.

3.1.3 Verifying the Exclusive Chip ID (ECID) and Board ID

Apple utilizes a mechanism called Personalization (or Taming) to prevent firmware replay attacks. You cannot simply take a valid, signed LLB from one iPhone and run it on another, nor can you downgrade to an older, vulnerable LLB version.

This enforcement happens inside the Image4 parser logic within LLB (checking the next stage) and the Boot ROM (checking LLB).

The Constraint Tags:
The IM4M manifest contains specific tags that bind the signature to the hardware:

  • ECID (Exclusive Chip ID): A 64-bit unique integer derived from the silicon die's coordinates on the wafer.
  • BORD (Board ID): Identifies the PCB model (e.g., 0x10 for a specific iPhone logic board).
  • CHIP (Chip ID): Identifies the SoC model (e.g., 0x8101 for M1).
  • SDOM (Security Domain): 0x1 for Production, 0x0 for Development.

The Check:
During boot, the executing code reads the actual values from the hardware fuses and compares them against the values present in the signed IM4M .

  • If Hardware.ECID != Manifest.ECID , the boot halts.
  • If Hardware.BORD != Manifest.BORD , the boot halts.

This mechanism, combined with the Nonce (a random value generated by the SEP during updates and baked into the IM4M ), ensures that the firmware is:

  1. Authentic: Signed by Apple.
  2. Targeted: Valid only for this specific device .
  3. Fresh: Valid only for this specific boot/update cycle (preventing downgrades).

Note: In the "Tahoe" architecture, this verification logic is hardened against fault injection. The comparison is often performed using redundant variables and bitwise checks that resist simple instruction skipping (e.g., glitching a B.NE instruction).

3.2 iBoot (Stage 2 Bootloader)

Once LLB has initialized the DRAM and verified the next stage, it hands off execution to iBoot . While LLB is a hardware-focused shim, iBoot is a sophisticated, single-threaded operating system in its own right. It contains a full USB stack, a display driver (for the Apple logo), a filesystem driver (APFS/HFS+), and the logic required to bootstrap the XNU kernel. In the Tahoe architecture, iBoot's role has expanded to become the orchestrator of the new security domains.

3.2.1 Device Tree (DT) Flattening

The hardware configuration of an Apple Silicon device is not discoverable via standard buses like PCI enumeration alone. Instead, iBoot relies on a Device Tree (DT) —a hierarchical data structure (similar to OpenFirmware or Linux FDT) that describes the SoC's topology.

The Source:
The raw Device Tree is embedded within the iBoot binary (or loaded as a separate devicetree.img4 payload). It contains nodes describing CPUs, memory maps, interrupt controllers (AIC), and peripherals.

The Flattening Process:
Before jumping to the kernel, iBoot "flattens" this tree into a binary format (FDT) and populates the /chosen node with runtime parameters.

  • kaslr-seed : A high-entropy random value generated by the TRNG. The kernel uses this to randomize its slide.
  • memory-map : A critical array of structures defining physical memory regions. iBoot marks regions used by the Boot ROM, LLB, and itself as reserved, ensuring the kernel does not overwrite them.
  • boot-args : The command-line arguments passed to the kernel (e.g., debug=0x14e , -v ). On production devices, these are strictly filtered; only specific flags are allowed unless the device is "demoted" or in a specific research state.

3.2.2 New in Tahoe: Loading the Security Monitors

In pre-Tahoe architectures (iOS 14 / macOS 11), iBoot would simply load the kernelcache and jump to it. In the Tahoe era (A15/M2+), iBoot must construct the Guarded Execution Environment before the kernel can exist.

Allocation and Reservation:
iBoot parses the device tree to identify physical memory ranges reserved for the new monitors. It carves these out of the available DRAM:

  • SPTM Region: Reserved for the Secure Page Table Monitor.
  • TXM Region: Reserved for the Trusted Execution Monitor.

Payload Loading:
iBoot locates the specific Image4 payloads within the kernelcache container:

  • Ap,SecurePageTableMonitor : The GL2 binary.
  • Ap,TrustedExecutionMonitor : The GL1 binary.

It decrypts and verifies these payloads just like any other firmware component. However, instead of loading them into standard memory, it loads them into the reserved physical regions identified above.

Locking SPRR Regions:
This is the critical security pivot. Before handing off control, iBoot configures the Shadow Permission Remapping Registers (SPRR) for the initial state.

  1. It configures the GL2 (SPTM) SPRR view to have Read/Write/Execute access to its own memory region.
  2. It configures the GL1 (TXM) SPRR view to have access to its region.
  3. Crucially, it configures the GL0 (Kernel) SPRR view to mark the SPTM and TXM regions as Inaccessible (No-Access) .

This ensures that when the processor eventually drops to EL1 (GL0) to run XNU, the kernel is physically incapable of reading or modifying the monitor code, even though it resides in the same physical DRAM.

3.2.3 LocalPolicy & BAA: The Shift to Local Signing

For macOS, Apple introduced a mechanism to allow users to boot older OS versions or custom kernels (Permissive Security) without breaking the hardware chain of trust. This is managed via LocalPolicy .

The Problem:
The Boot ROM and LLB enforce strict signature checks against Apple's global signing server (TSS). If you want to boot a custom kernel, you can't get a valid signature from Apple.

The Solution:

  1. LocalPolicy: A policy file stored on the Data Volume (in the iSCPreboot volume). It specifies the security mode (Full, Reduced, Permissive) and the hash of the custom kernel collection.
  2. BAA (Basic Attestation Authority): When a user authorizes a downgrade (via Recovery Mode authentication), the Secure Enclave generates a Local Signing Key .
  3. Re-Signing: The LocalPolicy is signed by this Local Key (inside the SEP).
  4. Boot Time: iBoot fetches the LocalPolicy. It asks the SEP to verify the signature. If the SEP confirms the policy is valid (and matches the user's intent), iBoot proceeds to load the custom kernel hash specified in the policy, effectively "blessing" it for this boot cycle.

This allows "Permissive Security" to exist while keeping the Boot ROM and LLB strictly locked down.

3.2.4 RE Focus: Decrypting iBoot Payloads via the AES MMIO Interface

To analyze iBoot, one must decrypt it. Since the GID key is fused into the silicon and physically disconnected from the CPU's register file, it cannot be extracted via software. Reverse engineers must instead turn the device into a Decryption Oracle by manipulating the dedicated AES hardware peripheral.

The kbag Mechanism:
The Image4 payload ( IM4P ) is encrypted with a random, per-file symmetric key (the target key). This target key is wrapped (encrypted) with the GID key and stored in the IM4P header as a Keybag ( kbag ) . To decrypt the firmware, one must unwrap this kbag.

The Hardware Distinction (ISA vs. MMIO):
It is critical to distinguish between the ARMv8 Crypto Extensions (instructions like AESE , AESD ) and the Apple AES Peripheral .

  • ARMv8 Crypto: Operates on keys loaded into standard NEON/SIMD registers ( v0 - v31 ). Useful for TLS or disk encryption where the key is known to the OS.
  • Apple AES Peripheral: A memory-mapped I/O (MMIO) block, typically located at a base offset like 0x23D2C0000 (on M1/T8103) or similar 0x2... ranges on newer SoCs. This peripheral has exclusive hardware access to the GID key fuses.

The Oracle Exploit:
Using a Boot ROM exploit (like checkm8 on A-series) or a specialized iBoot exploit, researchers execute a payload that drives this MMIO interface directly:

  1. Reset: Reset the AES peripheral via the AES_CTRL register to clear internal state.
  2. Key Selection: Write to the configuration register to select the GID Key as the decryption source. This sets an internal mux; the key itself is never exposed to the bus.
  3. FIFO Loading: Write the kbag (IV + Ciphertext) into the AES_DATA_IN FIFO registers.
  4. Execution: Trigger the engine. The hardware pulls the GID key from the fuses, performs the AES-256-CBC unwrap, and pushes the result to the output buffer.
  5. Extraction: Read the unwrapped target key (typically formatted as iv:key ) from the AES_DATA_OUT register.

Bit-Flipping and Timing Countermeasures:
Modern Apple Silicon (A12+/M1+) implements countermeasures against this oracle usage. The AES engine may enforce a state machine that requires the output of a GID decryption to be immediately DMA'd to executable memory and jumped to, rather than read back into a general-purpose register. Bypassing this often requires Fault Injection (voltage glitching) to corrupt the state machine or precise timing attacks to race the hardware's "sanitize on read" logic, allowing the extraction of the plaintext key before the hardware scrubs it.

4.0 The Security Monitor Layer (GL1/GL2): The New Hypervisor

In the "Tahoe" architecture, the XNU kernel has been demoted. It no longer possesses the ultimate authority to define the virtual memory layout of the system. That power has been migrated to a hardware-enforced monitor running in a proprietary execution state. This section dissects the mechanics of this new layer, which effectively functions as a lightweight, silicon-enforced hypervisor for the kernel itself.

4.1 The Secure Page Table Monitor (SPTM) - GL2

The Secure Page Table Monitor (SPTM) operates at Guarded Level 2 (GL2) . While architecturally sharing the EL1 exception level with the kernel, the Guarded Execution Feature (GXF) ensures that GL2 has a strict superset of permissions compared to the kernel's GL0. The SPTM is the sole entity permitted to write to the physical pages that constitute the translation tables (TTBR0/TTBR1).

4.1.1 The GENTER and GEXIT Instructions: Context Switching

Transitions into the SPTM are not handled by standard ARM exception vectors ( VBAR_EL1 ). Instead, they utilize the proprietary GENTER instruction, which performs a synchronous, atomic context switch.

The GENTER ABI:
To invoke the SPTM, the kernel populates specific registers and executes the opcode.

  • Opcode: 0x00201420 (Little Endian).
  • x15 (Selector): The Dispatch ID. This integer identifies which SPTM primitive to execute.
  • x0 - x7 (Arguments): The parameters for the call (e.g., physical addresses, permission flags).
  • x16 / x17 : Often used as scratch or secondary dispatch modifiers in newer revisions.

The Hardware Transition:
Upon execution of GENTER :

  1. SPRR Switch: The hardware swaps the active Shadow Permission Remapping Register configuration. The memory regions containing the SPTM code and data—previously invisible to the kernel—become Read/Write/Execute. Conversely, the kernel's own text might become Read-Only or Non-Executable depending on the monitor's logic.
  2. Stack Switch: The Stack Pointer ( SP ) is switched to the SP_GL2 register, pointing to a dedicated secure stack within the SPTM's private memory.
  3. PC Jump: Execution jumps to the vector defined in GXF_ENTRY_EL1 .

The Return:
The SPTM returns control to the kernel using GEXIT ( 0x00201400 ). This restores the GL0 SPRR configuration and the kernel's stack pointer. Crucially, GEXIT clears sensitive registers to prevent data leakage from the secure context.

4.1.2 The Frame Table (FTE): Tracking Physical Reality

To enforce security, the SPTM cannot rely on the kernel's data structures (like vm_page_t ), as they are mutable by a compromised kernel. Instead, the SPTM maintains its own "God View" of physical memory called the Frame Table .

The Frame Table is a linear array of Frame Table Entries (FTE) , located in SPTM-private memory. There is one FTE for every 16KB page of physical RAM.

FTE Structure (Reconstructed):
While the exact struct evolves, it generally contains:

  • Type (Bitfield): The strict classification of the page.
    • XNU_DATA : Generic kernel heap/stack.
    • XNU_TEXT : Immutable kernel code.
    • PAGE_TABLE : A page containing translation entries (TTEs).
    • USER_DATA : Memory mapped to EL0 processes.
    • SPTM_PRIVATE : Internal monitor structures.
  • Refcount : Tracks how many virtual mappings point to this physical page.
  • OwnerID : Identifies the Trust Domain (e.g., Kernel vs. Exclave).

The Security Invariant:
The SPTM enforces that a physical page can only be mapped into a virtual address space if the mapping permissions are compatible with the page's Type . For example, a page marked XNU_DATA in the FTE cannot be mapped as Executable. A page marked PAGE_TABLE cannot be mapped as Writable by the kernel.

4.1.3 The Dispatch Table: Reverse Engineering the Selectors

The interface between XNU and the SPTM is a strict, register-based API. However, unlike the stable syscall numbers of the BSD layer, the SPTM Dispatch IDs (Selectors) are not guaranteed to remain static across macOS versions. Apple frequently rotates these IDs between major (and occasionally minor) releases to frustrate static analysis tools and exploit chains that rely on hardcoded offsets.

The ABI:

  • x15 : The Dispatch ID (Selector).
  • x0 - x7 : Arguments (Physical Addresses, Permission Bitmasks, ASIDs).
  • x16 / x17 : Scratch registers, occasionally used for secondary modifiers or sub-ops.

Heuristic Identification:
Since relying on static IDs (e.g., 0x00 ) is brittle, reverse engineers must fingerprint the logic of the handler functions within the Ap,SecurePageTableMonitor binary to identify the primitives.

  • sptm_retype(ppn, old_type, new_type) (Often ID 0x00 or 0x01 ):

    • Fingerprint: Look for a function that accepts a Physical Page Number (PPN), reads the corresponding Frame Table Entry (FTE), and performs a Sanitization Loop . The SPTM must zero-fill ( bzero ) or cache-invalidate the page before transitioning it from XNU_DATA to PAGE_TABLE to prevent the kernel from initializing a page table with pre-computed malicious entries.
    • Logic: assert(refcount == 0); memset(pa, 0, PAGE_SIZE); fte->type = new_type;
  • sptm_map(asid, va, ppn, perms) :

    • Fingerprint: Look for a function that walks the translation tables (reading physical memory) and performs a Permission Check against the FTE. It will contain logic that explicitly compares the requested perms (e.g., Write) against the fte->type (e.g., XNU_TEXT ).
    • Logic: if (fte->type == XNU_TEXT && (perms & WRITE)) panic(); write_tte(...);
  • sptm_unmap(asid, va) :

    • Fingerprint: Look for the TLB Invalidation sequence. After clearing a TTE, the SPTM must execute TLBI instructions (e.g., TLBI ASIDE1IS ) to ensure the hardware translation lookaside buffer is coherent. The kernel is forbidden from executing TLBI instructions that affect the secure context; it must delegate this to the SPTM.
  • sptm_map_iommu(dart_id, context_id, dva, ppn, perms) :

    • Fingerprint: Look for writes to MMIO regions associated with DART controllers, rather than standard RAM. This function validates that the ppn is not a protected kernel page before mapping it into a device's IOVA space.

RE Implication:
Automated analysis scripts should not rely on CMP x15, #0x1 . Instead, they should symbolically execute the GENTER handler in the SPTM binary, identifying the dispatch table jump, and then classify the target functions based on the presence of DC ZVA (cache zero), TLBI , or FTE array access patterns.

4.1.4 RE Focus: Analyzing Panic Strings and the State Machine

The SPTM is designed to be Fail-Secure . Unlike standard kernel APIs that return KERN_FAILURE , the SPTM treats invalid requests as evidence of kernel compromise.

The Panic Mechanism:
If XNU sends a malformed request (e.g., trying to retype a page that is still mapped), the SPTM does not return. It triggers a system-wide panic.

  • Panic String: "received fatal error for a selector from TXM" or "invalid state transition" .
  • Analysis: These strings are gold for reverse engineers. They confirm that the SPTM enforces a strict Finite State Machine (FSM) for memory pages.

Mapping the State Machine:
By analyzing the panic logic, we can deduce the allowed transitions:

  1. FREE $\rightarrow$ XNU_DATA (Allocation)
  2. XNU_DATA $\rightarrow$ PAGE_TABLE (Retype for MMU use - requires sanitization)
  3. PAGE_TABLE $\rightarrow$ XNU_DATA (Teardown - requires unmapping all entries)
  4. XNU_DATA $\rightarrow$ XNU_TEXT (KEXT loading - One-way transition!)

Any attempt to deviate from this graph (e.g., trying to turn PAGE_TABLE directly into XNU_TEXT ) results in an immediate halt. This prevents "Page Table Spraying" and other heap manipulation techniques used to gain kernel execution.

4.2 The Trusted Execution Monitor (TXM) - GL1

If the SPTM is the brawn—enforcing the physics of memory mapping—the Trusted Execution Monitor (TXM) is the brains. Operating at Guarded Level 1 (GL1) , the TXM is the supreme arbiter of system policy. It represents the architectural decoupling of "mechanism" from "policy." While the SPTM handles how a page is mapped, the TXM decides if it is allowed to be mapped executable.

4.2.1 Decoupling AMFI: Moving Code Signature Verification

Historically, the Apple Mobile File Integrity (AMFI) kernel extension was the primary enforcement point for code signing. However, as a KEXT running in EL1, AMFI was susceptible to kernel-level memory corruption. Exploits like tfp0 could patch amfi_get_out_of_my_way or swizzle the MAC (Mandatory Access Control) hooks to bypass signature checks.

In the Tahoe architecture, the core verification logic has been lifted out of the kernel and placed into the TXM. Crucially, the TXM does not necessarily parse the full CMS blob on every request; its primary currency of trust is the Code Directory Hash (CDHash) .

The Verification Flow:

  1. Load & Hash: The kernel (XNU) loads a binary into memory (typed as XNU_DATA ). It parses the LC_CODE_SIGNATURE load command and calculates the CDHash (SHA-256) of the Code Directory.
  2. The Query: XNU issues a GENTER call to the TXM. It passes the CDHash and the physical address of the memory range.
  3. Trust Cache Lookup: The TXM first checks its internal Trust Caches (located in GL1 memory).
    • Static Trust Cache: Contains CDHashes for all immutable OS binaries (from the Cryptex).
    • Dynamic Trust Cache: Contains CDHashes for recently verified third-party apps.
  4. Cold Validation: If the CDHash is not found (a cache miss), the system enters a "Cold Start" validation path. The kernel (often aided by amfid in userland for complex policy checks) must provide the CMS signature blob to the TXM. The TXM performs the cryptographic verification of the blob against the Apple Root CA (or Developer ID Root) within the secure world. If valid, the CDHash is promoted to the Dynamic Trust Cache.
  5. Bless: Once the CDHash is validated (either via Cache Hit or Cold Validation), the TXM updates its internal state to "bless" the specific physical pages associated with that CDHash.
  6. Enforcement (GL2): When XNU subsequently asks the SPTM to map those pages as Executable ( RX ), the SPTM queries the TXM: "Is this physical page verified?" If the TXM has not blessed the CDHash associated with those pages, the SPTM denies the Execute permission.

RE Implication: Patching the kernel to ignore signature errors is now futile. Even if XNU "thinks" a binary is signed and attempts to set the executable bit in the PTE, it lacks the hardware authority to do so. The SPTM will simply refuse the PTE update because the TXM never validated the CDHash.

4.2.2 The Trust Cache: Static vs. Dynamic

To avoid the performance penalty of cryptographic verification on every page fault, the TXM manages the Trust Cache —a database of known-good CDHashes.

  • The Static Trust Cache:
    This is loaded by iBoot and passed to the TXM during the GL1 initialization. It contains the hashes of every binary in the OS (now encapsulated in the immutable Cryptexes ). This cache resides in GL1 memory and is strictly Read-Only.

  • The Dynamic Trust Cache:
    This handles third-party applications and JIT regions. When a user launches an app, the TXM verifies the signature once and adds the CDHash to the Dynamic Trust Cache.

    • Query Interface: The kernel queries the Trust Cache via a specific GENTER selector.
    • Attack Surface: The Dynamic Trust Cache is a mutable structure in GL1. A logic bug in the TXM's management of this cache (e.g., a race condition during entry removal or a hash collision attack) is a high-value target for persistence.

4.2.3 Developer Mode Enforcement and Downgrade Protection

The TXM is also the guardian of the device's security posture, specifically Developer Mode .

In previous iterations, enabling debugging capabilities was often a matter of setting nvram variables or boot-args (like cs_enforcement_disable=1 ). In Tahoe, these states are managed by the TXM.

The State Transition:
Enabling Developer Mode requires a reboot and explicit user authorization (Secure Intent via physical buttons). The TXM persists this state (likely via the Secure Enclave's secure storage).

Downgrade Protection:
The TXM enforces that the system cannot transition from a "Production" state to a "Developer" state without a full reboot and authentication ceremony. This prevents a kernel-level attacker from dynamically relaxing security policies to load unsigned modules.

Furthermore, the TXM validates the LocalPolicy (signed by the SEP) to determine if the system is booting in "Permissive Security" mode. If the LocalPolicy signature is invalid or missing, the TXM defaults to "Full Security," rejecting any code not signed by the Apple Root CA, regardless of what the kernel requests.

5.0 XNU Kernel Initialization: Entering EL1

The handoff from iBoot to the XNU kernel marks the transition from a single-threaded bootloader to a symmetric multiprocessing (SMP) operating system. However, in the Tahoe architecture, this is no longer a handover of absolute power. The kernel enters Exception Level 1 (EL1) not as a master, but as a client of the Guarded Level 2 (GL2) monitor.

The entry point is defined in osfmk/arm64/start.s . At this precise moment, the system state is fragile: the MMU is likely disabled (or running on an identity map provided by iBoot), interrupts are masked ( DAIF bits set), and the stack pointer is essentially arbitrary. The kernel's first objective is to orient itself within physical memory, calculate the KASLR slide, and establish the virtual memory structures required to turn on the lights.

5.1 The start routine and KASLR

The _start symbol is the architectural entry point. Unlike x86_64, where the kernel might handle its own decompression and relocation, the Apple Silicon kernel is loaded as a raw Mach-O executable (within the kernelcache container) directly into physical memory by iBoot.

The Register State at Entry:

  • x0 : Physical address of the boot_args structure (version 2).
  • x1 : Physical address of the Device Tree base (if not inside boot_args ).
  • x2 : 0 (Reserved).
  • x3 : 0 (Reserved).
  • sp : Invalid/Temporary.

5.1.1 Deriving the Kernel Slide: The Dual-Slide Paradigm

Kernel Address Space Layout Randomization (KASLR) on Apple Silicon is a cooperative effort between iBoot and XNU. Unlike x86_64, where the kernel might handle its own relocation, the Apple Silicon kernel is loaded as a raw Mach-O executable directly into physical memory by iBoot. However, in the Tahoe architecture, KASLR has evolved from a simple obfuscation technique into a compartmentalized security boundary.

The boot_args Structure:
Upon entry at _start , the kernel immediately parses the boot_args structure (version 2) pointed to by x0 . This structure acts as the handover manifest, containing the virtBase (the static link address, typically 0xFFFFFFF007004000 ) and the physBase (the actual physical load address in DRAM).

The Slide Calculation:
The slide is not generated by the kernel at runtime; it is consumed. iBoot generates a high-entropy value from the TRNG, populates the /chosen/kaslr-seed property in the Device Tree, and physically relocates the kernel text in DRAM to match this slide.

The kernel calculates its own slide using the delta between the compile-time base and the runtime physical base (adjusted for the static virtual-to-physical offset):

$$ \texttt{vm\_kernel\_slide} = \texttt{boot\_args.virtBase} - \texttt{CompileTimeBase} $$

The Tahoe Constraint: Entropy Decorrelation:
In the Tahoe architecture, the system operates under a Dual-Slide Paradigm . The SPTM (GL2) and the Kernel (GL0/EL1) reside in the same physical DRAM but operate in distinct translation regimes.

  1. Kernel Slide: Randomized by iBoot based on kaslr-seed .
  2. SPTM Slide: Randomized by iBoot based on a separate, decorrelated entropy source (or a cryptographic derivation of the master seed that is not exposed to EL1).

Security Implication:
This decorrelation is critical. A kernel-level memory leak (e.g., an infoleak revealing a kernel pointer) allows an attacker to calculate vm_kernel_slide . In previous architectures, if the monitor (PPL) was mapped at a fixed offset relative to the kernel, a kernel leak would instantly reveal the monitor's location.

In Tahoe, knowing vm_kernel_slide yields zero information about the virtual address of the SPTM. The SPTM's virtual mapping is established by iBoot in the GL2 translation tables ( TTBR1_EL2 context) before the kernel executes. The kernel is aware of the SPTM's physical pages (marked as "Reserved" in the memory map to prevent the VM subsystem from overwriting them), but it is architecturally blind to the SPTM's virtual location.

RE Focus: Finding the Slide:
For a reverse engineer with a kernel panic log or a JTAG connection, identifying these slides requires inspecting distinct registers:

  • Kernel Slide: Inspect TTBR1_EL1 . The translation table base points to the physical location of the kernel's L1 table. The high bits of the PC (Program Counter) at the exception vector reveal the virtual slide.
  • SPTM Slide: This is invisible from EL1. To find it, one must inspect TTBR1_EL2 (or the proprietary system register aliasing the GL2 translation base) via JTAG while the core is halted in the GL2 context.
  • Static Analysis: The vm_kernel_slide global variable in XNU is one of the first initialized. In a raw memory dump, locating the boot_args struct (often at the start of a physical page aligned to 16KB) will reveal the virtBase directly.

5.1.2 Initializing the MMU: TCR_EL1 and the SPTM Handshake

Before the kernel can execute C code safely, it must enable the Memory Management Unit (MMU). On standard ARMv8, this involves populating translation tables and writing to TTBR0_EL1 (User) and TTBR1_EL1 (Kernel), then setting SCTLR_EL1.M .

On Tahoe, this process is fundamentally altered because the kernel cannot write to its own page tables.

The Bootstrap Problem:
How does the kernel build its initial page tables if it requires the SPTM to map pages, but the SPTM requires the kernel to make hypercalls?

The Solution: The Bootstrap Tables:
iBoot constructs a set of initial "Bootstrap Page Tables" before handing off control. These tables are identity-mapped (Physical == Virtual) for the PC and stack, but also contain the kernel's high-virtual mappings.

  1. TCR_EL1 Setup: The kernel configures the Translation Control Register ( TCR_EL1 ) .
    • T1SZ / T0SZ : Defines the size of the virtual address space (typically 36-bit or 39-bit on iOS, 48-bit on macOS).
    • TG1 : Granule size (16KB is standard for Apple Silicon, unlike the 4KB standard of Android/Linux).
    • IPS : Intermediate Physical Address Size (matches the SoC capability, e.g., 40 bits).
    • TBI1 (Top Byte Ignore): Critical for PAC. This bit must be set to 1. It tells the MMU to ignore the top byte (bits 63-56) during address translation, allowing PAC signatures to exist in valid pointers without causing translation faults.

The SPTM Handshake (The First GENTER ):
Once TCR_EL1 is configured, the kernel must transition from the iBoot-provided bootstrap tables to its own managed tables.

  1. Allocation: The kernel allocates physical pages for the new L1/L2/L3 translation tables from the XNU_DATA pool.
  2. Sanitization: The kernel zeroes these pages.
  3. Retype: The kernel executes GENTER (Selector 0x00 - sptm_retype ) to convert these pages from XNU_DATA to PAGE_TABLE .
  4. Mapping: The kernel executes GENTER (Selector 0x01 - sptm_map ) to populate the entries, replicating the kernel text and static data mappings.
  5. Activation: Finally, the kernel writes the physical address of the new L1 table to TTBR1_EL1 .

The SCTLR_EL1 Lockdown:
The final step of start is writing to the System Control Register ( SCTLR_EL1 ) .

  • M (MMU Enable): Set to 1.
  • C (Cache Enable): Set to 1.
  • WXN (Write-XOR-Execute): Set to 1.

In Tahoe, the SPTM monitors writes to SCTLR_EL1 . If the kernel attempts to disable WXN (a common technique in older jailbreaks to patch kernel text), the SPTM intercepts the system register write and panics the device. The hardware configuration enforced by GL2 ensures that SCTLR_EL1 modifications are privileged operations that must comply with the system's security policy.

Once the MMU is active and the kernel is running on its own page tables (managed by SPTM), the start routine branches to arm_init , beginning the high-level initialization of the BSD subsystem and IOKit.

5.2 Hardware Security Enforcements (The "Kill Switch" Registers)

As the kernel initialization sequence progresses through start , it reaches a critical inflection point. The memory management structures are initialized, and the kernel is about to transition from a setup phase to a runtime phase. To prevent a compromised runtime kernel from modifying its own logic, the initialization routine must engage the hardware "Kill Switches."

These are proprietary system registers that, once written to, become immutable until a full system reset. In the pre-Tahoe era, these registers were the primary defense against persistent kernel compromises. In the Tahoe architecture, they serve as a hardware-enforced baseline that the SPTM relies upon to maintain the invariant of Kernel Text Immutability .

5.2.1 KTRR (Kernel Text Read-Only Region): The Physical Lock

Kernel Text Read-Only Region (KTRR) is Apple’s hardware solution to the "W^X" (Write XOR Execute) problem at the physical memory controller level. While the MMU (via page tables) controls virtual access permissions, page tables are mutable data structures. If an attacker gains arbitrary read/write (KRW) access to the kernel, they could theoretically modify the page tables to make the kernel text writable.

KTRR bypasses the MMU entirely. It enforces permissions based on Physical Addresses (PA) directly within the Memory Controller (MCU).

The Register Interface:
KTRR is controlled via a set of proprietary system registers, typically accessible via MSR instructions at EL1 (or GL2 in Tahoe).

  • KTRR_LOWER_EL1 ( S3_4_c15_c2_3 ): Defines the physical start address of the protected range.
  • KTRR_UPPER_EL1 ( S3_4_c15_c2_4 ): Defines the physical end address.
  • KTRR_LOCK_EL1 ( S3_4_c15_c2_2 ): The kill switch. Writing 1 to the lock bit enables the protection.

The "RoR" (Read-only Region) Mechanism:
Once the lock bit is set:

  1. Write Protection: Any write transaction (store instruction or DMA) targeting a physical address within the [LOWER, UPPER] range is dropped by the memory controller. Depending on the SoC generation, this either fails silently or triggers a synchronous external abort (SError).
  2. Execute Protection: The memory controller ensures that instruction fetches are permitted from this region.
  3. Immutability: The KTRR_LOCK_EL1 register itself becomes Read-Only. It cannot be unlocked by software.

The Tahoe Evolution (Virtualization of KTRR):
On M3/M4 chips running the SPTM, the kernel's interaction with KTRR changes. Since the kernel (EL1) is deprivileged, it cannot be trusted to set up KTRR correctly (it might set the range to zero, leaving itself unprotected).

  • SPTM Enforcement: The SPTM configures the physical KTRR registers during its own initialization (in Ap,SecurePageTableMonitor ).
  • Virtualization: When the XNU kernel executes the legacy instructions to write to KTRR_LOWER/UPPER in start , the hardware traps these accesses to GL2. The SPTM validates that the kernel is attempting to cover the correct physical range (matching the XNU_TEXT entries in the Frame Table) and effectively "mocks" the success of the operation to the kernel, while ensuring the hardware is actually locked down according to the SPTM's policy.

RE Focus: The KTRR Slide Alignment
Because KTRR operates on physical ranges, it lacks the granularity of 4KB/16KB pages. It typically operates on larger blocks (e.g., 1MB or L2 cache line boundaries). This forces the KASLR slide to be aligned to the KTRR granularity. If you are brute-forcing the KASLR slide, knowing the KTRR alignment constraint significantly reduces the entropy search space.

5.2.2 Kernel Integrity Protection (KIP): Extending the Shield

KTRR protects the static kernel binary ( kernelcache ). However, modern macOS relies heavily on the Boot Kernel Collection (BKC) and Auxiliary Kernel Collection (AKC) —large caches of drivers and extensions loaded during boot. These reside in memory adjacent to the kernel but are technically distinct payloads.

Kernel Integrity Protection (KIP) is the architectural evolution designed to protect these dynamic-but-immutable regions.

The Mechanism:
Unlike KTRR, which typically defines a single contiguous range, KIP (often implemented via the GXF or APRR logic on newer chips) allows for a more flexible definition of immutable regions.

  1. Registration: During the start routine, the kernel iterates over the loaded extensions (IOKit drivers).
  2. Sealing: Once the extensions are linked and relocated, the kernel issues a call to "seal" the region. In Tahoe, this is a GENTER call to the SPTM (Selector sptm_retype or sptm_protect ).
  3. Retyping: The SPTM updates the Frame Table Entries (FTE) for the physical pages backing the drivers. It transitions them from XNU_DATA (Writable) to XNU_TEXT (Executable/Read-Only).

The "One-Way" Door:
The security invariant enforced here is that memory typed as XNU_TEXT is never writable by EL1.

  • If the kernel attempts to write to a KIP-protected page, the SPRR configuration for GL0 (Kernel) triggers a permission fault.
  • If the kernel attempts to ask the SPTM to map it as writable, the SPTM checks the FTE, sees XNU_TEXT , and panics.

This effectively turns the kernel extensions into ROM. This mitigates a massive class of rootkits that historically operated by patching IOKit vtables or function pointers in memory.

5.2.3 The System Control Register ( SCTLR_EL1 ) Lockdown

The final "Kill Switch" is the configuration of the ARM processor itself. The SCTLR_EL1 register controls the MMU, caches, and alignment checks.

Critical Bits:

  • WXN (Write-XOR-Execute): Bit 19. When set, any memory region mapped as Writable is implicitly treated as Non-Executable ( XN ). This prevents the execution of shellcode on the heap or stack.
  • M (MMU Enable): Bit 0.
  • A (Alignment Check): Bit 1.

The Trap-and-Emulate Trap:
In a standard ARM system, EL1 can modify SCTLR_EL1 at will. An attacker with code execution could simply write to SCTLR_EL1 to disable WXN , map their shellcode, and execute it.

In the Tahoe architecture, SCTLR_EL1 is a trapped resource.

  1. The Trap: The hardware is configured (via HCR_EL2 or proprietary GXF controls) to trap writes to SCTLR_EL1 to the monitor layer (GL2/SPTM).
  2. The Policy: When the kernel executes MSR SCTLR_EL1, x0 , the CPU switches to the SPTM.
  3. The Check: The SPTM inspects the value in x0 . It enforces a strict policy: WXN must always be 1.
  4. The Verdict: If the kernel attempts to clear WXN , the SPTM denies the write and panics the system with a "Security Violation" code.

This ensures that the fundamental security properties of the execution environment (W^X) cannot be disabled, even by the kernel itself. The kernel is permanently locked into a secure configuration from the moment start completes.

5.3 Exclaves: The Microkernel within the Monolith

The introduction of Exclaves in the Tahoe architecture represents the most profound structural change to the Apple OS ecosystem since the transition from Mac OS 9 to OS X. It is an admission that the monolithic kernel architecture (XNU) has become too large, too complex, and too mutable to serve as the ultimate Trusted Computing Base (TCB) for high-value assets.

Exclaves introduce a Microkernel architecture running side-by-side with the monolithic XNU kernel on the same Application Processor cores. Unlike the Secure Enclave (which is a separate coprocessor with limited compute power), Exclaves harness the full performance of the M-series P-cores and E-cores while maintaining cryptographic isolation enforced by the SPTM.

5.3.1 The L4 Influence: Domains, Conclaves, and IPC

The architecture of the Exclave system is heavily indebted to the L4 microkernel family (specifically seL4). It prioritizes minimalism, capability-based security, and strict isolation.

The Hierarchy of Isolation:

  1. The Secure Kernel ( ExclaveOS ): A tiny, formally verifiable kernel that manages scheduling and IPC within the secure world. It runs at a privilege level guarded by the SPTM (likely GL1 or a restricted GL2 domain).
  2. Domains: The highest level of separation. The "Insecure Domain" hosts XNU and userland. The "Secure Domain" hosts Exclave workloads.
  3. Conclaves: Within the Secure Domain, workloads are siloed into Conclaves . A Conclave is a lightweight container consisting of an address space, a set of capabilities (handles to resources), and threads.
    • Example: The FaceID Conclave cannot access the memory of the Wallet Conclave, even though both run within the Exclave environment.

Memory Management via SPTM:
The isolation is enforced by the SPTM's Frame Table. Physical pages assigned to an Exclave are typed in the FTE (likely as EXCLAVE_DATA or SK_DATA ).

  • XNU View: The kernel sees these physical pages as "reserved" or "stolen" in the device tree. Any attempt by XNU to map these pages via sptm_map will result in a panic, as the SPTM forbids mapping Exclave-owned pages into the XNU_DOMAIN .
  • Exclave View: The ExclaveOS requests mappings from the SPTM. The SPTM updates the translation tables specific to the Exclave context (a distinct TTBR or context ID).

5.3.2 RE Focus: The RingGate Mechanism and XNUProxy

For the reverse engineer, the critical question is: How does the Kernel talk to an Exclave? They share no virtual memory, run in different hardware contexts, and the SPTM actively prevents XNU from mapping Exclave physical pages. The bridge is a mechanism internally referred to as RingGate , facilitated by a kernel extension named XNUProxy ( com.apple.driver.XNUProxy ).

The Communication Stack:

  1. Tightbeam (The IDL):
    Apple has introduced a new Interface Definition Language (IDL) called Tightbeam . It replaces the legacy Mach Interface Generator (MIG) for secure world communication. Unlike MIG, which is loosely typed and message-centric, Tightbeam is strongly typed and buffer-centric, designed to minimize parsing ambiguity.

    • Userland Analysis: The serialization logic is visible in /usr/lib/libTightbeam.dylib . Analyzing this library reveals the wire format: a header containing a protocol version and message ID, followed by a packed struct of arguments. Symbols like tb_message_encode , tb_client_call , and tb_endpoint_create are the primary indicators of this traffic.
    • Kernel Analysis: In XNUProxy , look for the TBTransport C++ classes that wrap the raw ring buffer management.
  2. The Downcall (XNU $\rightarrow$ Exclave):
    When XNU needs a service (e.g., "Authenticate this Passkey"), it cannot call the function directly.

    • Marshaling: XNUProxy serializes the request using Tightbeam into a shared memory ring buffer.
    • The Gate: The kernel executes a specific instruction to trigger the world switch. This is a GENTER instruction targeting a specific Dispatch ID reserved for the Secure Kernel (distinct from the SPTM/TXM selectors).
    • Context Switch: The hardware (mediated by SPTM) saves the EL1 state, switches the SPRR configuration to the Exclave view, and jumps to the ExclaveOS entry point.
    • Dispatch: ExclaveOS inspects the ring buffer and routes the message to the appropriate Conclave's IPC port.
  3. The Upcall (Exclave $\rightarrow$ XNU):
    Exclaves are not completely autonomous; they often rely on XNU for file system I/O or networking (since writing a full network stack in a microkernel is impractical).

    • The Exclave writes a request to the outbound ring buffer.
    • It triggers an interrupt or executes a GEXIT yield.
    • XNUProxy receives the notification, reads the request (e.g., "Write this blob to disk"), performs the operation via standard VFS calls, and returns the result via a Downcall.

Analyzing the XNUProxy Kext:
This kext is the Rosetta Stone for Exclave interaction. It maintains the mapping between Mach Ports (in XNU) and Tightbeam Endpoints (in Exclaves).

  • Endpoint Mapping: Reverse engineering the XNUProxy binary reveals structures that map 64-bit integer IDs to specific Exclave services. It effectively acts as a NAT (Network Address Translation) layer between Mach Ports and Exclave Endpoints.
  • Memory Loaning (The "DART" Window): While control messages go through ring buffers, large data transfers (like camera frames or biometric buffers) occur via Memory Loaning .
    • XNUProxy pins a userland page and retrieves its physical address.
    • It passes this Physical Page Number (PPN) to the Exclave via Tightbeam.
    • The Exclave requests the SPTM to map this specific PPN into its address space.
    • Vulnerability Class: This "Loaned Memory" is a prime target for TOCTOU (Time-of-Check to Time-of-Use) attacks. If the kernel (or a malicious user thread) modifies the content of the physical page after the Exclave has validated the header but before it processes the body, the Exclave's parser could be compromised. The SPTM likely enforces "Shared" vs "Exclusive" ownership states in the Frame Table to mitigate this, but the logic is complex and fragile.

5.3.3 Use Case: Secure Control of Privacy Indicators and Passkeys

The "Killer App" for Exclaves in macOS Tahoe is the hardware-enforced privacy indicator (the green/orange dots).

The Pre-Tahoe Vulnerability:
In previous macOS versions, the "Green Dot" (camera active) was drawn by WindowServer or the kernel. If an attacker achieved kernel-level code execution (Ring 0), they could hook the display driver and suppress the dot while recording the user.

The Tahoe Solution:

  1. Hardware Ownership: The physical frame buffer region corresponding to the status bar indicators is not mapped in the XNU domain. It is owned exclusively by a specific Privacy Conclave .
  2. The DART Lock: The Display Coprocessor's IOMMU (DART) is configured by the SPTM such that the main display pipe cannot write to the indicator pixels. Only the secure overlay pipe, controlled by the Exclave, can write there.
  3. Sensor Interrupts: When the camera sensor is energized, the GPIO interrupt is routed directly to the Exclave (via the AIC - Apple Interrupt Controller).
  4. The Drawing Path: The Exclave receives the "Camera On" interrupt. It writes directly to the secure frame buffer overlay to render the green dot.
  5. Immutability: Because XNU cannot map the physical memory of the secure overlay, and cannot reconfigure the DART (locked by SPTM), a compromised kernel is physically incapable of erasing the indicator.

Passkeys and FIDO:
Similarly, the cryptographic operations for Passkeys (WebAuthn) have moved to an Exclave.

  • Private Keys: The private keys for Passkeys are generated and stored within the Exclave's secure storage (mediated by SEP).
  • Biometric Binding: The Exclave communicates directly with the SEP to verify FaceID/TouchID.
  • Isolation: Even if malware injects code into LocalAuthentication daemon or the kernel, it cannot extract the private key material, as it resides in a memory domain that simply does not exist in the attacker's address space.

6.0 The Mach Subsystem: The Nervous System

While the SPTM and Exclaves represent the new fortress walls of the Apple Silicon architecture, the Mach subsystem remains the internal nervous system that coordinates activity within the XNU kernel. Originating from the NeXTSTEP era, Mach provides the fundamental primitives for Inter-Process Communication (IPC), thread scheduling, and virtual memory management.

For the reverse engineer, Mach is the primary vector for local privilege escalation (LPE). Despite decades of hardening, the complexity of state management in Mach messaging remains a fertile ground for logic bugs, race conditions, and reference counting errors. In the Tahoe era, Mach has been retrofitted with heavy PAC enforcement to protect its object graph.

6.1 Mach Ports & IPC Primitives

At the conceptual level, Mach is an object-oriented kernel. The fundamental unit of addressing is the Mach Port . To a userland process, a port is merely a 32-bit integer handle ( mach_port_name_t ). To the kernel, it is a complex, reference-counted data structure ( ipc_port ) that acts as a unidirectional communication channel.

6.1.1 Port Rights: Receive, Send, Send-Once, and Dead Names

The security model of Mach is capability-based. Possessing a port name is meaningless without the associated Port Right . The kernel tracks these rights in the process's IPC space.

  • MACH_PORT_RIGHT_RECEIVE : The ownership right. Only one task can hold the Receive right for a specific port at any given time. This task is the destination for messages sent to the port.
    • Kernel Structure: The ipc_port struct contains a pointer ( ip_receiver ) to the ipc_space of the task holding this right.
  • MACH_PORT_RIGHT_SEND : The ability to queue messages into the port. Multiple tasks can hold send rights to the same port. This is the standard "client" handle.
  • MACH_PORT_RIGHT_SEND_ONCE : A "fire-and-forget" right that vanishes after a single message is sent. This is critical for the Request/Reply pattern (RPC). When a client sends a message, it typically includes a MAKE_SEND_ONCE right to its own reply port. The server uses this to send exactly one reply, preventing the server from spamming the client later.
  • MACH_PORT_RIGHT_DEAD_NAME : If the task holding the Receive right dies or destroys the port, all outstanding Send rights in other tasks are instantly transmuted into Dead Names. Any attempt to send a message to a dead name returns MACH_SEND_INVALID_DEST .

RE Focus: The ipc_port Structure and PAC:
In previous generations, a common exploit technique involved "Fake Ports"—spraying the heap with crafted data that looked like an ipc_port struct and then tricking the kernel into using it.

In the arm64e/Tahoe architecture, the ipc_port structure is heavily fortified:

  1. ip_object : The base header of the port.
  2. ip_kobject : A pointer to the underlying kernel object (e.g., a task, a thread, or a user-client). This pointer is PAC-signed.
  3. ip_context : A 64-bit context value, also PAC-signed.

If an attacker attempts to forge a port, they must generate a valid signature for the ip_kobject pointer. Without the APDAKey (Data Key A), the kernel will panic upon AUT execution during message delivery.

6.1.2 The IPC Space ( ipc_space ) and the Global Name Server

Every task (process) in macOS has an associated IPC Space ( ipc_space ). This structure acts as the translation layer between the userland integer handles ( mach_port_name_t ) and the kernel pointers ( ipc_port * ).

The Translation Table ( is_table ):
The IPC space is implemented as a dynamic table (or radix tree for large spaces) of IPC Entries ( ipc_entry ).

  • Index: The userland handle (e.g., 0x103 ) is essentially an index into this table.
  • Entry: The ipc_entry struct contains:
    • ie_object : A pointer to the actual ipc_port (or ipc_pset ). Crucially, in Tahoe, this pointer is signed with PAC.
    • ie_bits : A bitfield storing the rights (Receive, Send, etc.) and the generation number (to detect stale handles).

The Lookup Process ( ipc_right_lookup_write ):
When a user executes mach_msg() , the kernel:

  1. Retrieves the current_task()->itk_space .
  2. Uses the handle passed in msgh_remote_port to index into the is_table .
  3. Validates the ie_bits to ensure the task actually possesses the required right (e.g., MACH_PORT_RIGHT_SEND ).
  4. Authenticates the ie_object pointer using AUT .
  5. Retrieves the ipc_port .

The Global Name Server:
Mach does not have a string-based global namespace in the kernel. "Service Discovery" is implemented in userland by launchd (the Bootstrap Server).

  • Special Ports: The kernel does maintain a small array of "Special Ports" attached to the Host and Task objects.
    • HOST_PORT : Represents the kernel itself.
    • HOST_PRIV_PORT : The "God Mode" port (see Section 6.2).
    • TASK_BOOTSTRAP_PORT : The handle to launchd .

When a process calls bootstrap_look_up("com.apple.foo") , it is actually sending a Mach message to the port stored in its TASK_BOOTSTRAP_PORT slot.

6.1.3 Copy-on-Write (CoW) optimizations in Out-of-Line (OOL) message passing

Mach messages are not limited to small scalars. They can transfer massive amounts of data using Out-of-Line (OOL) descriptors. This mechanism relies on Virtual Memory (VM) tricks rather than data copying, making it highly efficient but historically dangerous.

The Mechanism:

  1. Sender: Includes a mach_msg_ool_descriptor_t in the message, pointing to a buffer in its address space (e.g., 100MB of data).
  2. Kernel Processing: The kernel does not copy the 100MB. Instead, it walks the sender's VM map.
  3. Copy-on-Write (CoW): The kernel marks the physical pages backing that buffer as Read-Only in the sender's map.
  4. Receiver: The kernel maps those same physical pages into the receiver's address space, also as Read-Only .
  5. Faulting: If either the sender or receiver tries to write to the buffer, the MMU triggers a fault. The kernel catches this, allocates a new physical page, copies the data, and updates the mapping for the writer. This preserves the illusion of a copy.

The Tahoe/SPTM Intersection:
In the Tahoe architecture, this VM manipulation is complicated by the SPTM.

  • Permission Downgrade: When the kernel marks the pages as CoW (Read-Only), it cannot simply update the PTEs. It must issue a GENTER call ( sptm_protect or sptm_map ) to the SPTM to downgrade the permissions of the physical pages in the sender's address space.
  • The Vulnerability Class: This complexity introduces a race window. If the kernel logic fails to correctly lock the VM map object before requesting the SPTM update, or if the SPTM state machine has a logic flaw regarding shared pages ( refcount > 1 ), it might be possible to modify the physical page after the message has been "sent" but before the receiver reads it. This is known as a Double Fetch or Physically-Backed-Virtually-Disjoint attack.

RE Focus:
Analyze vm_map_copyin and vm_map_copyout in the XNU source (or binary). Look for how vm_map_entry structures are flagged with needs_copy and how these flags translate into SPTM calls. The interaction between Mach IPC (which assumes it controls VM) and the SPTM (which actually controls VM) is the friction point where new bugs will likely emerge.

6.2 The host_priv Port

In the lexicon of XNU exploitation, the Host Privilege Port ( host_priv ) has historically been synonymous with "Game Over." It is the Mach port representing the kernel task itself. Possessing a send right to this port allows a userland process to invoke the host_priv_server subsystem, granting the ability to manipulate physical memory, load kernel extensions (legacy), and control processor sets.

However, in the Tahoe architecture, the omnipotence of host_priv has been severely curtailed. The Secure Page Table Monitor (SPTM) and the Trusted Execution Monitor (TXM) have effectively neutered the "God Mode" capabilities traditionally associated with this handle.

6.2.1 The "God Mode" Handle: Generation and Restriction

The host_priv port is backed by the realhost kernel global structure. Unlike standard task ports, which map to a task_t , the host_priv port maps to the host object, but with a specific bit set in its ip_kobject pointer or context indicating privileged access.

Generation at Boot:
During the kernel bootstrap ( osfmk/kern/host.c ), the system initializes the host subsystem.

  1. host_init() : Allocates the realhost structure.
  2. Port Allocation: Calls ipc_port_alloc_kernel() to create the port.
  3. Kobject Association: The port is associated with the realhost structure.
    • PAC Enforcement: On arm64e, the ip_kobject pointer stored in the port structure is signed using the APDAKey (Data Key A) combined with the address of the port structure as the context. This prevents an attacker with a kernel read/write primitive from simply overwriting the ip_kobject of a user port to point to realhost .

Distribution:
The kernel is extremely stingy with this port. It is vended primarily to:

  • kernel_task : The kernel itself holds the receive right.
  • launchd (PID 1): Receives a send right during the userland bootstrap handoff.
  • kextd / kernelmanagerd : Required for managing kernel extensions (though this role is diminishing with DriverKit).

The "TFP0" Fallacy in Tahoe:
Historically, obtaining host_priv allowed an attacker to call task_for_pid(host_priv, 0, &kernel_task) . This returned the kernel_task port, granting full vm_read / vm_write access to the kernel's entire address space (TFP0).

In the Tahoe architecture, this chain is broken by the SPTM :

  1. task_conversion_eval : Even if you possess host_priv , the kernel checks the System Integrity Protection (SIP) policy managed by the TXM. If SIP is active, task_for_pid(0) returns KERN_FAILURE .
  2. The Immutable Kernel: Even if you bypass the SIP check (e.g., via a logic bug), and obtain the kernel_task port, the vm_write operations are intercepted.
    • If you attempt to write to kernel text ( XNU_TEXT ), the SPTM panics the system (Permission Violation).
    • If you attempt to write to page tables ( PAGE_TABLE ), the SPTM panics the system.
    • If you attempt to write to Data Const ( XNU_DATA_CONST ), the SPTM panics.

Thus, in Tahoe, host_priv degrades from a "Write-Anywhere" primitive to a "Logic Control" primitive. It allows you to invoke privileged kernel APIs, but it does not grant direct memory corruption capabilities against protected regions.

6.2.2 Task Special Ports: The Privileged Directory

While host_priv itself is restricted, it acts as the directory service for a set of Special Ports that control specific subsystems. These are accessed via host_get_special_port (MIG ID 205).

The realhost structure maintains a static array of these ports: ipc_port_t special[HOST_MAX_SPECIAL_PORT + 1] .

Critical Special Ports:

  • HOST_KEXTD_PORT (ID 1): The handle used to communicate with the kernel extension manager. Historically used to force the kernel to load a malicious kext. In Tahoe, kext loading is gated by the TXM, which verifies the signature and the LocalPolicy .
  • HOST_SEATBELT_PORT (ID 7): The control port for the Sandbox ( sandbox.kext ). Possession of this port allows a process to query and potentially manipulate sandbox policies (though policy application is usually one-way).
  • HOST_AMFID_PORT (ID 18): The communication channel for amfid . This port is critical for the Code Signing validation loop.
  • HOST_AUTOMOUNTD_PORT (ID 11): Used to trigger filesystem mounts.

RE Focus: The host_set_special_port Trap:
A common post-exploitation technique involves overwriting one of these special ports with a port controlled by the attacker (e.g., replacing the KEXTD port). This allows the attacker to intercept kernel upcalls intended for system daemons.

  • Mitigation: The host_set_special_port routine checks the caller's entitlements. Only a process with the com.apple.private.set-special-port entitlement (usually only launchd ) can overwrite these entries.
  • Bypass: Attackers look for logic bugs where a race condition allows replacing a port before the entitlement check completes, or finding a daemon that holds this entitlement and coercing it to perform the set operation (Confused Deputy).

6.2.3 RE Focus: Fuzzing Mach Message Parsing (MIG)

Since host_priv exposes a wide attack surface via the Mach IPC interface, it is a primary target for fuzzing. The kernel handles these messages using the Mach Interface Generator (MIG) .

The host_priv_server Routine:
When a message is sent to host_priv , the kernel's IPC dispatcher calls host_priv_server . This is an auto-generated function that deserializes the Mach message and dispatches it to the implementation (e.g., kern_host.c ).

Vulnerability Classes in MIG:

  1. Type Confusion: MIG relies on the message header to define the size and type of arguments. If the userland client sends a malformed message (e.g., claiming a descriptor is OOL memory when it is actually inline data), the kernel's unmarshaling logic might misinterpret the data, leading to heap corruption.
  2. Reference Counting Leaks: If a MIG routine returns an error (e.g., KERN_INVALID_ARGUMENT ) after it has incremented the reference count on a port or VM object but before it consumes it, the object leaks. In the kernel, this can lead to a refcount overflow (though 64-bit refcounts make this hard) or a Use-After-Free if the cleanup logic is flawed.
  3. TOCTOU on OOL Memory: As discussed in Section 6.1.3, if the message includes Out-of-Line memory, the kernel maps it Copy-on-Write. If the MIG handler verifies the content of the memory and then uses it later, the userland process might be able to race the kernel and modify the physical page (via a side-channel or SPTM state confusion) between the check and the use.

The Tahoe Hardening:
In the Tahoe kernel, MIG-generated code has been hardened with PAC .

  • Function Pointers: The dispatch tables used by host_priv_server are signed.
  • Context: The ipc_kmsg structure (representing the message in flight) is heavily protected to prevent modification of the message contents after validation but before processing.

However, logic bugs in the implementation of the host calls (the C functions called by MIG) remain reachable. For example, host_processor_set_priv allows manipulating CPU sets. If the logic fails to account for a processor being in a low-power state or being managed by an Exclave, it could trigger an inconsistent state in the scheduler.

7.0 IOKit & Driver Architecture

While the Mach subsystem provides the primitives for IPC and scheduling, IOKit provides the object-oriented framework for device drivers. Historically, IOKit has been the "soft underbelly" of the XNU kernel. Written in a restricted subset of C++, it relies heavily on virtual function dispatch, complex inheritance hierarchies, and manual reference counting ( OSObject::retain / release ).

In the Tahoe architecture, IOKit has undergone a radical hardening process. The transition to Apple Silicon has allowed Apple to enforce strict Control Flow Integrity (CFI) on C++ objects using PAC, while the SPTM enforces the immutability of the driver code itself.

7.1 IOKit Initialization

The initialization of IOKit is the bridge between the static hardware description provided by iBoot (the Device Tree) and the dynamic, runtime object graph that constitutes the macOS driver environment.

7.1.1 The IORegistry: Populating the Device Tree into C++ Objects

When the kernel boots, the hardware topology is described by the Flattened Device Tree (FDT) located at the physical address passed in x1 to _start . IOKit's first major task is to hydrate this binary blob into a live graph of IORegistryEntry objects.

The IOPlatformExpert :
The bootstrap process is driven by the IOPlatformExpert class (specifically IOPlatformExpertDevice on Apple Silicon).

  1. Unflattening: The kernel parses the FDT. For every node in the tree (e.g., arm-io , uart0 , aic ), it instantiates an IORegistryEntry .
  2. The IODeviceTree Plane: These objects are attached to the IODeviceTree plane of the Registry. This plane represents the physical topology as reported by iBoot.
  3. Property Mapping: Properties from the FDT (like reg , interrupts , compatible ) are converted into OSData , OSString , or OSNumber objects and attached to the registry entries.

Matching and Driver Loading:
Once the Registry is populated, IOKit begins the Matching phase ( IOService::startMatching ).

  1. The compatible String: IOKit iterates over the registry entries. It compares the compatible property (e.g., apple,s5l8960x-uart ) against the IOKitPersonalities dictionary defined in the Info.plist of every loaded driver.
  2. The Probe/Start Lifecycle: When a match is found, the driver's C++ class is instantiated.
    • init() : Constructor.
    • probe() : The driver verifies the hardware is actually present (rarely used on SoCs where hardware is static).
    • start() : The driver initializes the hardware, maps MMIO regions, and registers interrupts.

RE Focus: The "Missing" Hardware:
On Tahoe systems, you will notice gaps in the IORegistry compared to the raw Device Tree.

  • Reserved Regions: The SPTM and TXM reserve specific hardware blocks (e.g., the Secure Storage controller or specific GPIO banks for privacy LEDs).
  • Filtering: During the unflattening process, the kernel checks the memory map. If a device node's reg property overlaps with a physical range reserved by the SPTM, the kernel skips creating an IORegistryEntry for it. This prevents the kernel from even attempting to map the MMIO registers for secure hardware, effectively making that hardware invisible to the OS.

7.1.2 Boot Kernel Collection (BKC) vs. Auxiliary Kernel Collection (AKC)

Gone are the days of loading individual .kext bundles from /System/Library/Extensions . To optimize boot speed and enforce immutability, macOS now uses Kernel Collections .

The Boot Kernel Collection (BKC):
This is a single, massive Mach-O binary contained within the kernelcache Image4 payload ( Ap,BootKernelCollection ).

  • Content: It contains the XNU kernel and all "Essential" drivers required to mount the root filesystem, start launchd , and initialize the display.
  • Linkage: All internal symbols are pre-linked. There are no external relocations required at boot time for these drivers.
  • Protection (KIP): This is the critical security distinction. Because the BKC is loaded by iBoot, its physical pages are known before the kernel starts. The SPTM marks these pages as XNU_TEXT (Immutable) in the Frame Table. The kernel cannot modify the BKC code or read-only data, even with a write primitive. It is effectively ROM.

The Auxiliary Kernel Collection (AKC):
Drivers not required for early boot (e.g., Bluetooth, Wi-Fi, Audio) are packaged into the AKC.

  • Loading: The AKC is loaded later in the boot process by kernelmanagerd (userland).
  • Verification: When kernelmanagerd passes the AKC to the kernel, the kernel must verify its signature. In Tahoe, this verification is offloaded to the TXM . The TXM checks the signature against the LocalPolicy .
  • Sealing: Once verified and linked, the kernel issues a GENTER call to the SPTM to "seal" the AKC memory region. The SPTM transitions the pages from XNU_DATA to XNU_TEXT . Once sealed, the AKC becomes just as immutable as the BKC.

RE Implication:
If you are reverse engineering a driver, you must determine if it lives in the BKC or AKC.

  • BKC Drivers: Their offsets are static relative to the kernel slide.
  • AKC Drivers: They are loaded at a randomized offset after the kernel boot. You must traverse the kmod_info linked list in memory to find their load addresses.

7.1.3 RE Focus: PAC-signing of C++ Vtables ( OSObject ) and BLRAA

The OSObject class is the root of the IOKit inheritance hierarchy. In C++, dynamic dispatch is handled via Virtual Tables (vtables) —arrays of function pointers. Historically, attackers would overwrite the vtable pointer in an object to point to a fake vtable controlled by the attacker (vtable hijacking).

In the arm64e architecture, Apple has fundamentally altered the C++ ABI for kernel code to mitigate this.

The Signed Vtable Pointer:
In a standard C++ object, the first 8 bytes are the pointer to the vtable. In XNU on arm64e, this pointer is signed .

  • Key: APDAKey (Data Key A).
  • Context: The context is derived from the address of the object itself (specifically, the storage location of the vtable pointer) combined with a salt (often the type hash).

$$ \texttt{SignedPtr} = \texttt{PAC}(\texttt{VtableAddr}, \texttt{Key=APDA}, \texttt{Context}=\&Object) $$

Address Diversity:
Because the object's address is part of the signature context, an attacker cannot simply copy a valid vtable pointer from Object A to Object B . The signature for Object A is valid only at Object A 's address. Moving it invalidates the PAC. This kills "vtable reuse" attacks.

The BLRAA Dispatch:
When the kernel calls a virtual function (e.g., object->release() ), the compiler emits a specialized instruction sequence:

LDR     x0, [x20]       ; Load the object pointer
LDR     x16, [x0]       ; Load the signed vtable pointer
AUTDA   x16, x0         ; Authenticate Data Key A, Context = Object Address (x0)
LDR     x10, [x16, #0x18] ; Load the target function pointer from the vtable
BLRAA   x10, x16        ; Branch with Link, Authenticating Key A, Context = Vtable Address

Note the two-stage authentication:

  1. AUTDA : Authenticates that the vtable pointer belongs to this object. If the pointer was overwritten, x16 becomes a canonical non-valid pointer.
  2. BLRAA : The function pointers inside the vtable are also signed (using APIAKey ). The BLRAA instruction authenticates the function pointer (using the vtable address as context) and branches.

The "Recursive" PAC:
This creates a chain of trust:

  • The Object trusts the Vtable Pointer (via APDAKey ).
  • The Vtable trusts the Function Pointers (via APIAKey ).
  • The SPTM trusts the Vtable Memory (via XNU_TEXT immutability).

For a reverse engineer, this means that patching a vtable in memory is impossible (SPTM), and forging an object requires the ability to sign pointers with the APDAKey for an arbitrary address—a capability that requires a "Signing Oracle" gadget, which BTI aims to eliminate.

7.2 DriverKit (dexts)

The introduction of DriverKit represents a strategic retreat for the XNU kernel. For decades, the kernel’s attack surface was effectively the sum of the core kernel plus every third-party driver loaded into the address space. A vulnerability in a Wacom tablet driver or a USB-to-Serial adapter was functionally identical to a vulnerability in the scheduler: both yielded EL1 code execution.

DriverKit bifurcates this model by moving hardware drivers into userland, executing as System Extensions ( .dext ) . While they look and feel like drivers to the developer (using a C++ subset similar to Embedded C++), architecturally they are unprivileged processes. In the Tahoe architecture, this isolation is not merely a sandbox; it is a hardware-enforced chasm guarded by the TXM and SPTM.

7.2.1 Moving drivers to userland: IOUserClient and Entitlement Checks

A dext does not have direct access to the kernel's task port, nor does it run with root privileges by default. It operates within a tightly constrained sandbox, communicating with the kernel via a specialized IPC bridge.

The IOUserServer Proxy:
When a dext is matched and loaded (managed by kernelmanagerd ), the kernel instantiates a shadow object known as IOUserServer . This kernel-side object acts as the proxy for the userland driver.

  • The Shim: When the kernel needs to call a function in the driver (e.g., Start() ), it calls a method on IOUserServer .
  • Serialization: IOUserServer serializes the arguments into a specialized Mach message format (distinct from standard MIG).
  • The Upcall: The message is sent to the dext process. The DriverKit runtime (linked into the dext ) deserializes the message and invokes the implementation of the IOService subclass in userland.

The IOUserClient Interface:
Conversely, when the dext needs to talk to the kernel (e.g., to register an interrupt handler or map memory), it cannot call kernel APIs directly. It uses IOUserClient .

  • Restricted API Surface: The dext can only invoke a specific subset of kernel functionality exposed via IOUserClient traps. These traps are heavily scrutinized.
  • OSAction : Interrupts are no longer handled via ISRs (Interrupt Service Routines) in the driver. Instead, the kernel handles the physical IRQ, masks it, and dispatches an OSAction event to the dext via a Mach notification. This eliminates the entire class of vulnerabilities related to interrupt context safety and spinlock deadlocks in third-party code.

Entitlements as the Gatekeeper (TXM Enforcement):
In Tahoe, the ability of a dext to bind to specific hardware is governed by Entitlements .

  • Hardware Binding: A dext cannot simply mmap any MMIO region. It must possess specific entitlements (e.g., com.apple.developer.driverkit.transport.usb ) to access specific device families.
  • TXM Verification: When kernelmanagerd launches the dext , the signature and entitlements are validated by the Trusted Execution Monitor (TXM) .
    • The TXM verifies the CDHash against the Trust Cache.
    • The TXM validates that the provisioning profile allows the specific HID or USB entitlements requested.
    • If the TXM returns a failure, the kernel refuses to establish the IOUserServer link, and the driver fails to start. This prevents a compromised kernelmanagerd from launching a rogue driver with elevated hardware access.

RE Focus: The IOUserClass Metaclass:
Reverse engineering a dext requires understanding the OSMetaClass infrastructure in userland. The dext binary contains OSMetaClass information that describes the RPC interface. By parsing the __DATA,__const sections, one can reconstruct the vtables and the mapping between the kernel-side dispatch IDs and the userland C++ methods.

7.2.2 Memory Mapping Constraints and IOMMU (DART) Protection

The most dangerous capability of a driver is Direct Memory Access (DMA) . A malicious or buggy driver could program a peripheral (like a GPU or Network Card) to write data to physical address 0x0 (or wherever the kernel text resides), bypassing CPU-enforced protections like KTRR.

To mitigate this, Apple Silicon employs a pervasive IOMMU architecture known as DART (Device Address Resolution Table) .

The DART Architecture:
Every DMA-capable peripheral on the SoC sits behind a DART. The device does not see Physical Addresses (PA); it sees I/O Virtual Addresses (IOVA) . The DART translates IOVA $\rightarrow$ PA, enforcing permissions (Read/Write) at the page level.

DriverKit Memory Model:

  1. IOMemoryDescriptor : When a dext allocates a buffer for DMA, it creates an IOMemoryDescriptor .
  2. Mapping: The dext calls IOMemoryDescriptor::CreateMapping . This triggers a call into the kernel.
  3. The Kernel's Role: The kernel allocates physical pages ( XNU_DATA ) and pins them.
  4. DART Programming: The kernel programs the DART associated with the specific hardware device controlled by the dext . It maps the physical pages to an IOVA range visible to that device.

The Tahoe/SPTM Enforcement:
In the Tahoe architecture, the kernel is no longer trusted to program the DARTs directly. If the kernel could write to DART registers, it could map the kernel's own text segment as writable to the GPU, then tell the GPU to overwrite it (a DMA attack).

  • SPTM Ownership: The physical pages containing the DART translation tables (or the MMIO registers controlling the DART) are typed as SPTM_PRIVATE or a specific hardware-protected type in the Frame Table.
  • The sptm_map_iommu Selector: When the kernel needs to map a buffer for a dext , it issues a GENTER call to the SPTM.
    • Validation: The SPTM verifies that the physical pages being mapped are owned by the dext (or are valid shared memory). It strictly forbids mapping any page typed XNU_TEXT , PAGE_TABLE , or SPTM_PRIVATE into a DART.
    • Execution: The SPTM performs the write to the DART hardware.

MMIO Mapping Restrictions:
Similarly, when a dext needs to control hardware registers (MMIO), it requests a mapping.

  • The kernel cannot simply map physical device memory into the dext 's address space.
  • The SPTM validates the physical address against a Device Allow List (embedded in the Device Tree and signed/verified by iBoot).
  • This ensures that a USB driver can only map the USB controller's registers, and cannot map the registers for the Secure Enclave Mailbox or the KTRR controller.

RE Implication:
Exploiting a dext to gain kernel privileges is exponentially harder in Tahoe. Even if you gain code execution in the dext (Userland), you cannot issue arbitrary syscalls (sandbox), you cannot map kernel memory (VM isolation), and you cannot use the hardware device to perform DMA attacks against the kernel (SPTM-enforced DART). The attacker is contained within a hardware-enforced cage, limited to the specific capabilities of that one peripheral.

7.3 The Graphics Stack (AGX)

If the XNU kernel is the central nervous system, the Apple Graphics (AGX) stack is a secondary, alien brain grafted onto the SoC. On M-series silicon, the GPU is not merely a peripheral; it is a massive, autonomous compute cluster running its own proprietary operating system, managing its own memory translation, and executing a command stream that is almost entirely opaque to the main OS.

For the reverse engineer, AGX represents the largest and most complex attack surface in the kernel. The driver ( AGX.kext ) is enormous, the firmware is encrypted (until load), and the hardware interface is undocumented. In the Tahoe architecture, Apple has moved to aggressively sandbox this beast, wrapping the GPU's memory access in strict DART (Device Address Resolution Table) policies enforced by the SPTM to prevent DMA-based kernel compromises.

7.3.1 RTKit: The Proprietary RTOS running on the GPU Coprocessor (ASC)

The GPU does not execute driver commands directly. Instead, the M-series SoC includes a dedicated Apple Silicon Coprocessor (ASC) —typically a hardened ARMv8-R or Cortex-M class core—that manages the GPU hardware. This coprocessor runs RTKit , Apple’s proprietary Real-Time Operating System.

The Firmware Blob:
The kernel driver does not contain the logic to drive the GPU hardware registers directly. Instead, upon initialization ( AGX::start ), it loads a firmware payload from the filesystem (e.g., /usr/share/firmware/agx.bin or embedded in the kext).

  • Format: The firmware is a standard Mach-O binary, often multi-architecture.
  • Sections: It contains __TEXT and __DATA segments just like a userland program.
  • RTKit Structure: Reverse engineering the firmware reveals a microkernel architecture. It has a scheduler, an IPC mechanism, and a set of "Endpoints" (services).

The RTKit IPC Protocol:
Communication between the XNU kernel ( AGX.kext ) and the ASC ( RTKit ) occurs via a shared memory mailbox protocol.

  1. Mailbox Registers: The AP writes to a specific MMIO register to ring the doorbell of the ASC.
  2. Message Buffer: The message payload is placed in a shared memory ring buffer.
  3. Endpoints: The protocol is endpoint-based. The kernel sends messages to specific service IDs running on the ASC:
    • EP_PM : Power Management (Voltage/Clock gating).
    • EP_GR : Graphics Rendering (Command submission).
    • EP_COMP : Compute (GPGPU/Metal).

RE Focus: The RTKit State Machine:
The AGX.kext contains extensive logging strings and state tracking for RTKit. By analyzing the RTKit class in the kext, one can reconstruct the message structures.

  • Crash Logs: When the GPU hangs, RTKit writes a "Coredump" to a shared buffer. The kernel captures this. Analyzing these logs reveals the internal memory layout of the ASC and the state of the GPU pipeline at the time of the crash.
  • Filter Bypass: Historically, vulnerabilities existed where the kernel could send malformed IPC messages to the ASC, causing memory corruption inside the GPU firmware. While this doesn't directly yield Kernel R/W, compromising the ASC allows an attacker to use the GPU as a confused deputy for DMA attacks (see 7.3.3).

7.3.2 IOMobileFramebuffer (IOMFB): Secure Framebuffers and Exclave Compositing

While AGX.kext handles rendering, IOMobileFramebuffer.kext (IOMFB) handles the display controller (DCP). This driver is responsible for the "Swap Chain"—taking the rendered frames and scanning them out to the display panel.

The Unified Memory Architecture (UMA):
On Apple Silicon, the Framebuffer is just a region of system DRAM. WindowServer (userland) renders into an IOSurface . The physical pages backing this surface are passed to IOMFB, which programs the Display Coprocessor (DCP) to read from them.

The Security Criticality:
IOMFB is a high-value target because it handles complex shared memory structures ( IOMFBSharedMemory ) mapped into both the kernel and userland ( WindowServer ).

  • Fuzzing Surface: The Connect method and external methods of IOMobileFramebufferUserClient have historically been riddled with race conditions and bounds-checking errors.

Tahoe and the "Secure Overlay":
In the Tahoe architecture, IOMFB's control over the display is no longer absolute. To support the Hardware Privacy Indicators (Green/Orange dots), the display pipeline has been bifurcated.

  1. Standard Pipe: Managed by IOMFB/WindowServer. Draws the desktop/apps.
  2. Secure Pipe: Managed by an Exclave . Draws the privacy indicators.

Hardware Compositing:
The compositing of these two pipes happens in the display hardware, not in memory.

  • The Exclave owns a small, reserved framebuffer region.
  • The Display Controller overlays this region on top of the standard framebuffer during scanout .
  • Immutability: Because the Secure Pipe's framebuffer memory is owned by the Exclave (and protected by the SPTM), neither the kernel nor the GPU can write to it. This guarantees that if the camera is on, the green dot will be visible, even if the kernel is compromised.

7.3.3 DART: The IOMMU Wall and DMA Containment

The GPU is effectively a DMA engine with the capability to read and write vast swathes of system memory. Without restriction, a compromised GPU firmware (or a malicious shader exploiting a GPU hardware bug) could overwrite kernel text or page tables.

To prevent this, the AGX hardware—and indeed every DMA-capable peripheral on the Apple Silicon SoC—sits behind a strict IOMMU known as the DART (Device Address Resolution Table) .

DART Architecture and Stream IDs (SIDs):
The DART translates Device Virtual Addresses (DVA) used by the peripheral into Physical Addresses (PA) in DRAM. However, the translation is not monolithic; it is segmented by the source of the traffic.

  • Stream IDs (SIDs): Every transaction on the SoC's Network-on-Chip (NoC) carries a hardware-generated Stream ID identifying the initiator (e.g., GPU Firmware, Vertex Fetcher, Display Controller).
  • Context Banks: The DART maintains multiple translation contexts (similar to distinct TTBR roots).
  • SID Matching: The DART hardware is configured to map specific SIDs to specific Context Banks. This allows isolation between different workloads on the same peripheral (e.g., isolating WindowServer rendering commands from a background compute shader).

The Tahoe Enforcement (SPTM):
In pre-Tahoe systems, the kernel ( AGX.kext or IOMapper ) managed the DART page tables and the SID configuration registers directly. This meant a kernel attacker could disable DART, remap SIDs to privileged contexts, or map kernel memory into the GPU's address space to bypass KTRR.

In Tahoe, DART management is privileged to the SPTM.

  1. Ownership: The physical pages containing the DART translation tables (L1/L2 TTEs) and the MMIO registers controlling SID configuration are typed as SPTM_PRIVATE (or a specific IOMMU_TABLE type) in the Frame Table.
  2. Mapping Requests: When AGX.kext needs to map a user's IOSurface for GPU access:
    • It pins the user pages.
    • It issues a GENTER call (Selector sptm_map_iommu ).
    • It passes the DART ID, the Context ID, the DVA, and the PA.
  3. Validation: The SPTM verifies:
    • The PA is valid USER_DATA (not Kernel Text, not Page Tables).
    • The DART ID corresponds to the GPU.
    • SID Integrity: Crucially, the SPTM enforces the immutable binding between SIDs and Contexts. It ensures that the kernel cannot reconfigure the DART to allow an untrusted SID (e.g., the Neural Engine) to write to a Context Bank reserved for the Secure Enclave or Display Pipe.
  4. Execution: The SPTM writes the DART PTE.

RE Focus: The "GART" Attack Surface:
Despite SPTM protection, the logic requesting the mapping still resides in the kernel.

  • Aliasing: Can the kernel trick the SPTM into mapping the same physical page to two different DART contexts with different permissions?
  • Stale TLBs: Does the SPTM correctly flush the DART TLB ( tlb_flush ) immediately after unmapping? If not, the GPU might retain access to a page that has been freed and reallocated to the kernel, leading to a Use-After-Free via DMA.
  • Side Channels: The DART configuration registers (e.g., TCR , TTBR , and SID match registers) are trapped by the hardware to GL2. Attempting to write to the DART control base address from EL1 should trigger a synchronous exception. Reverse engineering the IOMapper class in IOKit will reveal the specific GENTER trampolines used to bridge these operations.

8.0 Userland Bootstrap: The Birth of PID 1

The initialization of the XNU kernel concludes with the mounting of the root filesystem (the Signed System Volume) and the spawning of the first userland process. In the UNIX tradition, this is init (PID 1). In macOS, this is launchd .

However, launchd is far more than a SysV init replacement. It is the Mach Port Broker , the central registry for the operating system's IPC namespace, and the gatekeeper for service lifecycle management. In the Tahoe architecture, launchd 's role is further hardened; it is the first process to run under the full scrutiny of the Trusted Execution Monitor (TXM), and its ability to bootstrap the rest of the system relies on a complex dance of entitlements and port rights.

8.1 launchd : The Mach Port Broker

The transition from kernel mode to user mode is a singularity. The kernel thread executing bsd_init constructs a user context and never returns. From this point forward, the system's security relies on the correct enforcement of Mach port rights and the immutability of the bootstrap namespace.

8.1.1 Transition from kernel to userland: The first execve

The kernel routine bsd_init (in bsd/kern/bsd_init.c ) is responsible for hand-crafting the first process.

The Loading Sequence:

  1. Path Resolution: The kernel looks for the init binary. Historically /sbin/launchd , in the Tahoe/Cryptex era, this path is a firmlink or symlink resolving into the OS Cryptex (e.g., /System/Cryptexes/OS/sbin/launchd ).
  2. load_init_program : This kernel function mimics an execve syscall from within the kernel. It allocates a proc_t structure and a task_t structure.
  3. TXM Validation (The First Check):
    Before the kernel can map launchd into memory, it must validate the code signature.
    • XNU parses the launchd binary to extract the CDHash.
    • XNU issues a GENTER to the TXM .
    • The TXM verifies that the launchd CDHash exists in the Static Trust Cache (loaded by iBoot). Since launchd is part of the immutable OS, it must be in the static cache. If it is not, the system panics immediately (integrity violation).
  4. SPTM Mapping:
    Once validated, XNU requests the SPTM to map the physical pages containing the launchd text segment.
    • The SPTM updates the Frame Table Entries (FTE) for these pages to USER_DATA (or a specific immutable user type).
    • The SPTM writes the translation table entries (TTEs) for the new process's address space, marking the text segment as RX (Read-Execute) and the data segment as RW (Read-Write).

The host_priv Handoff:
Crucially, launchd is the only process in the system that receives the Host Privilege Port ( host_priv ) directly from the kernel without asking for it.

  • During load_init_program , the kernel calls ipc_port_alloc_kernel to retrieve the host_priv port.
  • It inserts a send right to this port into launchd 's IPC space ( ipc_space ).
  • This right is placed at a fixed, known port name (often mach_task_self_ + n ), or retrieved by launchd immediately upon startup via task_get_special_port .

This handle gives launchd the authority to configure the rest of the system, including setting the system clock, loading kernel extensions (via kernelmanagerd ), and shutting down the system.

8.1.2 Initializing the Bootstrap Port (subset of the Name Service)

Mach does not have a global, string-based name service in the kernel. If Process A wants to talk to "com.apple.windowserver", the kernel has no idea what that string means. The mapping of String -> Mach Port is the responsibility of the Bootstrap Server . launchd is the Bootstrap Server.

The Bootstrap Port:
Every task in XNU has a special port slot called TASK_BOOTSTRAP_PORT .

  • Initialization: When launchd starts, it allocates a receive right for a new port. It calls task_set_special_port(mach_task_self(), TASK_BOOTSTRAP_PORT, new_port) to register this port as the bootstrap port for itself .
  • Inheritance: When launchd spawns a child process (e.g., syslogd ), the child inherits this bootstrap port handle. This connects every process in the system back to launchd .

The Namespace Hierarchy:
The bootstrap namespace is not flat; it is hierarchical to support security domains and user separation.

  1. Startup Domain: The root domain, populated by LaunchDaemons (system-wide services running as root or specific users).
  2. User Domain: Created when a user logs in. Populated by LaunchAgents .
  3. Session Domain: Specific to a GUI login session (Aqua).

RE Focus: The bootstrap_check_in Routine:
When a daemon starts (e.g., locationd ), it must retrieve the receive rights for the Mach ports it is supposed to service.

  1. The daemon calls bootstrap_check_in(bootstrap_port, "com.apple.locationd", &service_port) .
  2. launchd receives this message.
  3. Security Check: launchd inspects the Audit Token (appended to the Mach message trailer by the kernel). It verifies that the PID sending the check-in request matches the PID that launchd just spawned for that label.
  4. Port Transfer: If the check passes, launchd moves the Receive Right for the service port (which it created earlier) into the daemon's IPC space.

This prevents "Port Squatting," where a malicious process tries to register "com.apple.password-server" before the real daemon starts. Only the process spawned by launchd as that service can claim the port.

8.1.3 Parsing launchd.plist and the binary protocol for XPC service lookups

launchd configuration is driven by Property Lists ( .plist ) located in /System/Library/LaunchDaemons , /Library/LaunchDaemons , etc. However, launchd does not simply read these files at runtime; it compiles them into an internal graph.

The MachService Dictionary:
The key element for reverse engineers is the MachServices dictionary in the plist.

<key>MachServices</key>
<dict>
    <key>com.apple.securityd</key>
    <true/>
</dict>

This entry tells launchd : "Allocate a Mach port. Associate it with the string 'com.apple.securityd'. If anyone sends a message to this port, launch this binary."

Demand Launching (The Lazy Allocation):
launchd does not start all daemons at boot. It creates the ports and listens on them.

  1. The Lookup: A client (e.g., Safari) calls xpc_connection_create_mach_service("com.apple.securityd", ...) .
  2. The Resolution: Under the hood, libxpc sends a message to the bootstrap port asking for the send right associated with that string.
  3. The Trigger: launchd looks up the string. It sees the service is not running. It performs posix_spawn to start the daemon.
  4. The Handoff: While the daemon is launching, launchd holds the client's request. Once the daemon checks in (see 8.1.2), launchd hands the send right to the client.

The Binary Protocol (XPC vs. MIG):
Historically, launchd used the Mach Interface Generator (MIG) for these lookups ( bootstrap.defs ). In modern macOS, this has largely been superseded by the XPC protocol, which runs over raw Mach messages but uses a proprietary serialization format.

  • xpc_pipe : This is the low-level mechanism. Messages sent to launchd are often XPC dictionaries serialized into a Mach message body.
  • Magic Bytes: XPC messages start with a specific magic header (often 0x42774242 - 'BwBB' in ASCII for binary blobs).
  • Structure: The payload is a serialized tree of objects (dictionaries, arrays, strings, file descriptors, and Mach ports ).

RE Focus: launchd 's job_dispatch :
Reverse engineering the launchd binary (which is stripped, but symbols can often be correlated from open-source releases of liblaunch ) reveals the job_dispatch or demux routine.

  • This routine parses the incoming Mach message.
  • It identifies if it is a legacy MIG request (subsystem 400) or an XPC request.
  • It routes the request to the appropriate subsystem (Job Management, Service Lookup, or XPC Domain management).

The "Domain" Concept:
In Tahoe, launchd enforces strict Domain Isolation . A process in the "App Sandbox" domain cannot look up services in the "System" domain unless those services are explicitly allowlisted in the sandbox profile ( com.apple.security.exception.mach-lookup.global-name ). launchd enforces this by checking the sender's entitlements (via the kernel audit token) against the requested service name before returning the port right. If the check fails, launchd returns BOOTSTRAP_UNKNOWN_SERVICE (or effectively 0 ), and the client sees "Connection Invalid."

8.2 The Dynamic Linker ( dyld )

If launchd is the architect of the userland process hierarchy, dyld (the dynamic linker) is the mason that lays the bricks for every single process. In the macOS ecosystem, dyld is not merely a library loader; it is a privileged extension of the kernel’s execution logic, responsible for enforcing the final mile of the platform’s security policies, including Library Validation, symbol resolution, and the application of Address Space Layout Randomization (ASLR).

On Apple Silicon and the Tahoe architecture, dyld has evolved significantly. It no longer relies on legacy load commands for rebasing; it utilizes Chained Fixups to enable page-in linking, and its operations are tightly coupled with the kernel’s memory management subsystem, which is in turn subservient to the Secure Page Table Monitor (SPTM) .

8.2.1 Mapping the Dyld Shared Cache (DSC)

The Dyld Shared Cache (DSC) is the defining characteristic of the macOS memory layout. It is a massive, pre-linked artifact (often exceeding 4GB) containing the combined text and data segments of nearly all system frameworks ( CoreFoundation , libSystem , Foundation , UIKit , etc.).

The Shared Region:
To optimize memory usage, the kernel maintains a Shared Region . This is a reserved area of the virtual address space (typically starting at 0x180000000 on arm64) where the DSC is mapped.

  • Physical Sharing: The physical pages backing the DSC are shared across all processes.
  • Split Segments: The DSC is split into three primary mappings to satisfy W^X (Write XOR Execute) requirements:
    1. __TEXT : Read-Execute (RX). Contains immutable code.
    2. __DATA_CONST : Read-Only (RO). Contains pointers and data that are resolved at link time and never change.
    3. __DATA_DIRTY : Read-Write (RW). Contains data that must be modified per-process (e.g., Objective-C class realization).

The Tahoe/SPTM Constraint:
In the Tahoe architecture, the mapping of the DSC is a privileged operation guarded by the SPTM.

  1. Boot Load: The DSC is loaded from the OS Cryptex ( /System/Cryptexes/OS/System/Library/dyld/ ).
  2. Immutable Mapping: When the kernel initializes the Shared Region, it issues GENTER calls to the SPTM. The SPTM updates the Frame Table Entries (FTE) for the physical pages backing the DSC __TEXT segment to USER_TEXT (or a specific shared immutable type).
  3. Enforcement: Once mapped, the SPTM forbids any entity (including the kernel) from writing to these physical pages. This renders the system libraries immutable even in the face of a kernel compromise. Hooking a system function like open in the shared cache via kernel memory writing is physically impossible; the attacker must use __DATA_DIRTY indirection or userland hooks.

8.2.2 RE Focus: Code Signature Validation ( fcntl(F_ADDFILESIGS) ) and the Call to TXM

dyld is the primary enforcer of Library Validation . This security hardening ensures that a process can only load libraries that are signed by Apple or signed by the same Team ID as the main executable.

The Validation Flow:
When dyld loads a Mach-O image (via dlopen or load command parsing):

  1. mmap : It maps the file into memory. At this stage, the pages are not yet executable.
  2. Signature Registration: dyld calls fcntl(fd, F_ADDFILESIGS_RETURN, ...) passing a fsignatures_t struct. This tells the kernel: "I want to execute code from this file. Here is the offset to the code signature (CMS blob)."

The Kernel $\rightarrow$ TXM Handoff:
In previous architectures, the kernel (specifically AMFI.kext ) would parse the blob and verify the signature. In Tahoe, the kernel is untrusted for verification.

  1. Marshaling: The kernel identifies the physical pages containing the signature blob.
  2. The Gate: The kernel executes GENTER to invoke the Trusted Execution Monitor (TXM) at GL1.
  3. TXM Verification:
    • The TXM parses the CMS blob in the secure world.
    • It verifies the cryptographic chain of trust against the Trust Cache (for platform binaries) or the Apple Root CA (for third-party).
    • It checks for constraints (e.g., is this a restricted binary? Does it have the com.apple.private.security.no-sandbox entitlement?).
  4. The Verdict: The TXM returns a verdict to the kernel. Crucially, it also updates the SPTM's view of those physical pages.
    • If valid, the TXM "blesses" the pages associated with the code directory hash (CDHash).
  5. Page Fault Enforcement:
    • When dyld later jumps into the library, a page fault occurs.
    • The kernel attempts to map the page as Executable ( RX ).
    • The SPTM intercepts the PTE update. It checks if the physical page has been blessed by the TXM.
    • If yes, the Execute bit is set. If no (e.g., the kernel lied about the signature verification), the SPTM denies the mapping, and the process crashes with SIGKILL (Code Signature Invalid) .

RE Tip: If you are analyzing a crash where dyld terminates a process immediately after loading a dylib, check the system logs for amfid or kernel messages related to TXM rejection. The error code returned by fcntl is often sanitized; the real reason (e.g., "Revoked Certificate") lies in the secure monitor logs, which may be redacted in production builds.

8.2.3 ASLR in Userland: Chained Fixups and the Death of dyld_info

Modern dyld (dyld 4.0+) on Apple Silicon has deprecated the legacy LC_DYLD_INFO rebasing opcodes in favor of Chained Fixups ( LC_DYLD_CHAINED_FIXUPS ). This architectural shift is not merely a performance optimization to reduce dirty pages; it is a fundamental change in how the binary format represents memory addresses, tightly coupling ASLR with Pointer Authentication (PAC).

The Problem with Legacy Rebasing:
In the legacy model, dyld parsed a list of opcodes ( REBASE_OPCODE_DO_* ) to locate pointers in the __DATA segment and add the ASLR slide. This required dyld to touch every page containing a pointer at launch time, triggering massive I/O and dirtying pages that might never be used.

The Chained Fixup Solution:
In a Chained Fixup binary, the "pointers" stored in the __DATA segment on disk are not memory addresses . They are 64-bit metadata structures that form a linked list (a chain) within each 16KB page.

The On-Disk Structure:
The LC_DYLD_CHAINED_FIXUPS load command points to a header ( dyld_chained_fixups_header ) containing offsets to "Starts" arrays ( dyld_chained_starts_in_image ).

  • Page-Level Granularity: The metadata identifies the offset of the first fixup in every page.
  • The Chain: The data at that offset is a 64-bit struct (e.g., dyld_chained_ptr_64_rebase ).
    • next (Bits 51-62): The offset (in 4-byte strides) to the next fixup in the same page. A value of 0 terminates the chain for that page.
    • target (Bits 0-35): The offset in the __TEXT segment (for rebasing) or the ordinal in the import table (for binding).
    • high8 (Bits 56-63): Additional attributes (e.g., auth diversity).

Page-In Linking (Lazy Fixups):
When dyld loads a library, it does not iterate the whole file. It sets up the memory mapping.

  1. The Fault: When the process execution flow hits a pointer in __DATA (e.g., a vtable entry or a global string pointer), a page fault occurs.
  2. The Trap: The kernel (or dyld via a user-fault handler) detects that the page requires fixups.
  3. The Walk: The fixup logic looks up the "Start" offset for that specific page. It walks the linked list defined by the next bits.
  4. The Patch: For each entry in the chain:
    • It extracts the target .
    • It adds the ASLR slide (for rebase) or resolves the symbol (for bind).
    • Crucially for arm64e: It calculates the PAC signature.
  5. The Write: It overwrites the 64-bit metadata struct with the final, signed, slid pointer. The page is now "dirty" and valid.

PAC Integration ( DYLD_CHAINED_PTR_ARM64E ):
On Apple Silicon, the fixup format is often DYLD_CHAINED_PTR_ARM64E . This struct contains the diversity data (salt) for the pointer.

  • dyld does not just write Base + Offset .
  • It writes PAC(Base + Offset, Key, Context) .
  • This binds the ASLR slide to the process-specific PAC keys ( APIAKey , APDAKey ) at the exact moment of paging in.

RE Implication: The "Broken" Binary:
For the reverse engineer, this mechanism means that static analysis of a raw Mach-O binary from disk is misleading.

  • If you open a raw binary in an older disassembler (or hexdump ), the global pointers will look like garbage (e.g., 0x0040... ). These are the chain metadata structs, not addresses.
  • CFG Destruction: Without processing these chains, the Control Flow Graph (CFG) is broken. Virtual function calls and global callbacks point to nowhere.

Tooling Requirement:
To analyze these binaries statically, you must "unchain" them—simulating the dyld fixup process to convert metadata back into generic, unslid pointers.

  • ipsw : The ipsw tool (specifically ipsw dyld fixup ) can parse the chains and output a "fixed" Mach-O where the __DATA segment contains valid pointers relative to the file base.
  • dyld_shared_cache_util : Apple's own tool (if available) or open-source equivalents must be used to extract and fixup binaries from the DSC.
  • IDA Pro / Ghidra: Modern versions automatically detect LC_DYLD_CHAINED_FIXUPS and apply the fixups in the database view, but the underlying file on disk remains "chained."

8.3 Cryptexes (Cryptographic Extensions)

The introduction of the Signed System Volume (SSV) in macOS Big Sur solved the problem of persistence; by rooting the filesystem trust in a cryptographic hash verified by iBoot, Apple ensured that the OS partition was immutable. However, this immutability introduced a significant logistical problem: patching a single binary (e.g., WebKit or dyld ) required a full OS update, a re-hashing of the entire volume, and a new signature from Apple’s Taming Server (TSS).

To resolve this tension between security and agility, Apple introduced Cryptexes (Cryptographic Extensions). A Cryptex is a cryptographically sealed, versioned filesystem image that is grafted onto the system hierarchy at boot or runtime. In the Tahoe architecture, the Cryptex subsystem is the primary mechanism for the "Split OS" design, decoupling the core kernel/BSD environment from the rapidly evolving userland frameworks.

8.3.1 The "Split OS" Architecture: /System/Cryptexes/OS

In modern macOS, the root filesystem ( / ) is essentially a skeleton. It contains configuration files and the kernel, but the vast majority of executable code—including dyld , libSystem , and the frameworks—resides within the OS Cryptex .

The Image4 Container:
A Cryptex is distributed as an Image4 ( img4 ) container.

  • Payload ( IM4P ): A disk image (typically APFS or HFS+) containing the directory hierarchy.
  • Manifest ( IM4M ): The signature. Crucially, the OS Cryptex is often Personalized . It is signed by TSS with the device's ECID, binding the specific version of the userland libraries to the specific silicon, preventing "library replay" attacks where a user might try to load a vulnerable dyld from an older OS version on a newer kernel.

The Mounting Process ( apfs_graft ):
The kernel does not mount Cryptexes using standard mount syscalls. It uses a specialized mechanism within the APFS kext known as Grafting .

  1. iBoot Handoff: For the OS Cryptex, iBoot loads the image into memory (or identifies its location on NAND) and verifies the IM4M signature against the hardware root of trust.
  2. Trust Cache Extraction: The Cryptex contains a wrapper/trustcache file. This is a binary blob containing the CDHashes of every executable inside the Cryptex.
  3. TXM Ingestion: Before the kernel grafts the filesystem, it extracts this Trust Cache and passes it to the Trusted Execution Monitor (TXM) via GENTER . The TXM verifies the signature of the Trust Cache itself. If valid, these hashes are added to the Static Trust Cache in GL1 memory.
  4. Grafting: The kernel calls apfs_graft . This stitches the Cryptex filesystem into the Virtual File System (VFS) namespace.
    • The OS Cryptex is grafted at /System/Cryptexes/OS .
    • Firmlinks: To maintain POSIX compatibility, the system uses APFS Firmlinks. When a process accesses /usr/lib/libSystem.B.dylib , the VFS transparently redirects the lookup to /System/Cryptexes/OS/System/Library/Frameworks/... .

RE Focus: The "Real" Path:
For reverse engineers, this abstraction can be confusing.

  • dyld : The dynamic linker itself lives in the Cryptex. The binary at /usr/lib/dyld on the SSV is often a shim or a hardlink to the Cryptex mount.
  • Debugging: When attaching a debugger to launchd or early boot processes, path resolution may return the /System/Cryptexes/... path. Scripts that rely on hardcoded paths to /System/Library must be updated to handle this redirection.

8.3.2 Rapid Security Response (RSR): Patching via Overlay Mounts

The Cryptex architecture enables Rapid Security Response (RSR) , allowing Apple to ship security fixes (e.g., for a WebKit zero-day) without a full OS update.

The Patch Cryptex:
An RSR update is essentially a "Sparse Cryptex." It does not contain a full OS; it contains only the binaries that have changed.

The Overlay Mechanism:
When an RSR is applied:

  1. cryptexd : The system daemon cryptexd (running as root) manages the staging of the update.
  2. Verification: The RSR Cryptex is verified by the TXM. Since RSRs are often generic (not personalized to ECID to allow faster distribution), the TXM verifies them against a specific "RSR Root" or a sub-CA in the Apple PKI.
  3. Union Mounting: The kernel mounts the RSR Cryptex over the existing OS Cryptex mount point using a union-like filesystem strategy.
    • If a file exists in the RSR Cryptex (e.g., WebKit.framework ), the VFS layer serves that file.
    • If it does not, the VFS falls back to the base OS Cryptex.

The "Antipatch" and Reversibility:
A critical security requirement for RSR is reversibility. If a security patch causes a boot loop, the system must be able to recover.

  • The Antipatch: RSRs include logic to revert the state.
  • Pre-Boot Authentication: Because the RSR modifies the executable code of the system, its presence must be attested to during boot. The state of "RSR Applied" vs. "Base OS" is managed by LocalPolicy and verified by iBoot/LLB. If the boot fails repeatedly, iBoot can discard the RSR overlay and boot the "Known Good" base OS Cryptex, effectively removing the patch.

Security Implications for the Researcher:

  • Atomicity: The RSR update is atomic. You cannot have a "half-patched" system. The TXM ensures that either the full RSR Trust Cache is loaded, or none of it is.
  • Persistence: RSRs do not modify the SSV. They persist in the Preboot volume and are re-grafted at every boot.
  • Analysis: To analyze an RSR patch, one cannot simply diff the filesystem. One must extract the RSR Cryptex ( .dmg inside the update payload), mount it, and compare the binaries against the base OS Cryptex. The "delta" is the patch.

9.0 The Security Daemon Hierarchy

While the kernel and the hardware monitors (SPTM/TXM) enforce the immutable laws of the system physics (memory protections, page table integrity), the complex, mutable business logic of macOS security is delegated to a hierarchy of userland daemons. These daemons operate with high privileges, often holding special ports or entitlements that allow them to influence kernel policy. For the reverse engineer, these daemons represent the "Policy Engine" of the OS—and historically, the most fertile ground for logic bugs and sandbox escapes.

9.1 amfid (Apple Mobile File Integrity Daemon)

The Apple Mobile File Integrity Daemon ( amfid ) is the userland arbiter of code execution policy. While the TXM (GL1) handles the cryptographic verification of platform binaries and the enforcement of the Static Trust Cache, it lacks the context to evaluate the complex web of third-party provisioning profiles, developer certificates, and MDM constraints.

In the Tahoe architecture, amfid functions as the Policy Decision Point (PDP) for third-party code, while the kernel and TXM act as the Policy Enforcement Points (PEP) .

9.1.1 The Interaction between launchd , the Kernel (MACF), and amfid

amfid does not poll for binaries; it is interrupt-driven by the kernel via the Mandatory Access Control Framework (MACF) hooks.

The Bootstrap Race:
amfid is a critical system daemon launched by launchd early in the boot process. Because amfid is responsible for verifying signatures, it presents a bootstrap paradox: Who verifies amfid ?

  • The Solution: amfid is a platform binary located in the OS Cryptex . Its CDHash is present in the Static Trust Cache loaded by iBoot.
  • TXM Verification: When launchd spawns amfid , the kernel consults the TXM. The TXM finds the hash in the immutable static cache and blesses the pages immediately. No upcall is required.

The Verification Upcall (The "Slow Path"):
When a user launches a third-party application (e.g., /Applications/Calculator.app ), the flow traverses the boundary between Kernel and Userland multiple times.

  1. The Hook: The kernel executes execve . The MACF hook mpo_vnode_check_signature is triggered in AppleMobileFileIntegrity.kext .
  2. TXM Miss: The kernel queries the TXM via GENTER . The TXM checks the Static Trust Cache and the Dynamic Trust Cache. For a newly launched third-party app, this lookup fails.
  3. The Upcall: The kernel constructs a Mach message containing the file's path, the CDHash, and the detached signature blob (if applicable). It sends this message to the HOST_AMFID_PORT (Special Port 18).
  4. amfid Processing: amfid receives the message. It performs the heavy lifting of parsing the CMS blob, validating the certificate chain (via IPC to trustd ), and checking provisioning profiles.
  5. The Verdict: amfid returns a boolean verdict to the kernel via host_set_exception_ports or a dedicated MIG reply.
  6. TXM Update: If amfid approves the binary, the kernel issues a second GENTER call to the TXM, instructing it to add the validated CDHash to the Dynamic Trust Cache .
    • Security Note: In Tahoe, the TXM likely verifies the signature blob again independently to ensure the kernel/amfid didn't lie about the cryptographic validity, but it relies on amfid for the policy decision (e.g., "Is this developer ID allowed on this system?").

RE Focus: The MIG Interface:
The communication interface is defined in mfi_server.defs (reverse engineered). The key routine is verify_code_directory .

  • Input: audit_token_t , mach_port_t file_port , off_t offset , int64_t file_size .
  • Attack Surface: Malformed Mach messages sent to amfid 's service port can trigger type confusion in the daemon. However, amfid checks the sender's audit token to ensure requests originate only from the kernel ( kernel_task ).

9.1.2 Validating Code Directories (CDHash), Entitlements, and Provisioning Profiles

The core logic of amfid resides in its ability to link a binary's Code Directory (CD) to a valid Provisioning Profile . This is the mechanism that enforces the "Walled Garden" on iOS and the Notarization requirements on macOS.

The Validation Logic ( MISValidateSignatureAndCopyInfo ):
amfid links against libmis.dylib (Mobile Installation Service), which exports the symbol MISValidateSignatureAndCopyInfo . This function is the heart of the check.

  1. CDHash Extraction: amfid reads the LC_CODE_SIGNATURE load command from the binary. It hashes the Code Directory slots to compute the CDHash.
  2. Entitlement Extraction: It parses the embedded entitlements (XML/plist) from the signature blob.
  3. Profile Correlation: If the binary is signed by a developer certificate (not Apple), amfid looks for an embedded embedded.mobileprovision file.
    • PKCS#7 Verification: It verifies the signature of the profile itself (signed by Apple).
    • Allowlist Check: It compares the entitlements requested by the binary against the Entitlements dictionary in the profile.
    • Device Check: For development profiles, it verifies the device's UDID is present in the ProvisionedDevices array.
  4. Constraint Enforcement:
    • Restricted Entitlements: Certain entitlements (e.g., com.apple.private.security.no-sandbox ) are "Restricted." They can only be granted if the provisioning profile is signed by a specific Apple internal certificate chain. amfid enforces this hierarchy.

The "Union" of Trust:
On macOS, amfid also interfaces with the Ticket system (Notarization).

  • It checks if a valid Notarization Ticket is stapled to the binary or present in the system's ticket database ( /var/db/SystemPolicy ).
  • If a ticket is found, amfid treats the binary as "Apple-Notarized," relaxing certain constraints compared to a purely ad-hoc signed binary.

RE Focus: libmis.dylib Reversing:
This library is heavily obfuscated and stripped. However, analyzing the failure paths of MISValidateSignature... is crucial for understanding why a specific binary is being killed.

  • Return Codes: The function returns an integer status. 0 is success. Other values map to specific failures (e.g., MIS_PROFILE_EXPIRED , MIS_ENTITLEMENT_MISMATCH ).
  • Logging: amfid writes detailed failure reasons to the system log ( os_log ), but often only if a specific boot-arg or configuration profile is installed ( AmfiDeveloperMode ).

9.1.3 Exception Handling: How get-task-allow and Debugging Entitlements are Processed

One of amfid 's most critical roles is gating access to process debugging. The ability to attach a debugger ( task_for_pid ) is effectively a full compromise of the target process.

The get-task-allow Entitlement:
In a standard production environment, a process cannot be debugged unless it possesses the com.apple.security.get-task-allow entitlement.

  • Xcode Builds: When you build and run an app from Xcode, it signs the binary with a development certificate and injects this entitlement.
  • App Store Builds: The App Store distribution process strips this entitlement.

The amfid Decision Matrix:
When the kernel parses a binary, it extracts the entitlements and passes them to amfid . amfid validates that the get-task-allow entitlement is permitted by the provisioning profile.

  • Valid: amfid tells the kernel to set the CS_GET_TASK_ALLOW flag in the process's cs_flags (Code Signing Flags).
  • Invalid: amfid strips the entitlement or rejects the signature entirely.

The task_for_pid Check:
When debugserver (or lldb ) calls task_for_pid(target_pid) , the kernel checks the target's cs_flags .

  • If CS_GET_TASK_ALLOW is set: Access granted (subject to root/procmod checks).
  • If CS_GET_TASK_ALLOW is clear: Access denied ( KERN_FAILURE ).

Developer Mode (The Tahoe Shift):
In the Tahoe architecture, the mere presence of the entitlement is insufficient. The system must be in Developer Mode .

  1. State Check: amfid queries the TXM to determine if Developer Mode is active.
  2. Enforcement: If Developer Mode is disabled , amfid will reject the execution of any binary signed with a development certificate, even if the profile is valid. This prevents an attacker from side-loading a malicious "debuggable" app onto a user's device to inspect memory, unless the user has explicitly lowered the device's security posture via the TXM-mediated reboot ceremony.

Unrestricted Debugging (SIP Disable):
On macOS, disabling System Integrity Protection (SIP) historically allowed unrestricted debugging ( csrutil disable ).

  • Legacy: The kernel checked the CSR_ALLOW_UNRESTRICTED_DEBUGGING bit in the NVRAM boot-args.
  • Tahoe: The SIP state is managed by the LocalPolicy and enforced by the TXM . amfid retrieves this policy state. If SIP is disabled, amfid relaxes the signature validation logic, effectively telling the kernel "Everything is valid," allowing the execution of unsigned code and the attachment of debuggers to system processes (except those protected by Exclaves).

9.2 sandboxd & The Seatbelt Policy

If amfid is the bouncer checking IDs at the door, the Seatbelt subsystem (marketed as App Sandbox) is the straightjacket strapped to the guest once they are inside. Originating from the TrustedBSD project, the macOS Sandbox is a Mandatory Access Control (MAC) mechanism that restricts a process's access to resources—files, network sockets, Mach ports, and IOKit drivers—regardless of the user's UID.

In the Tahoe architecture, the Sandbox has evolved from a path-based filter into a semantic, metadata-driven enforcement engine that is tightly coupled with the kernel's VFS layer and the new Data Vault primitives.

9.2.1 Compiling SBPL (Sandbox Policy Language) to Byte Code

The definition of a sandbox profile is written in SBPL (Sandbox Policy Language) , a Scheme-like (LISP) dialect. However, the kernel does not contain a LISP interpreter. The translation from human-readable policy to kernel-enforceable logic is the responsibility of sandboxd and the libsandbox.dylib runtime.

The Compilation Pipeline:

  1. Profile Selection:

    • App Store Apps: The system applies the generic container profile.
    • System Daemons: Daemons specify their profile name in launchd.plist (e.g., com.apple.syslogd.sb ).
    • Platform Profile: In modern macOS, individual SBPL files for system services are increasingly deprecated in favor of a monolithic Platform Profile embedded directly into the Boot Kernel Collection . This reduces the attack surface by removing the need to parse text files at runtime.
  2. The Compiler ( sandboxd ):
    When a process initializes the sandbox (via sandbox_init_with_parameters ), an XPC call is made to sandboxd .

    • sandboxd parses the SBPL.
    • It resolves variable expansions (e.g., ${HOME} , ${TemporaryDirectory} ).
    • It compiles the rules into a proprietary Byte Code format (often referred to as "TinyScheme" or "Filter Machine" code).
  3. The Byte Code Structure:
    The compiled blob is a serialized state machine. Reverse engineering this format reveals a graph of operations:

    • Opcodes: OP_MATCH_PATH , OP_MATCH_PATTERN , OP_CHECK_ENTITLEMENT , OP_ALLOW , OP_DENY .
    • Filters: The logic is optimized into a decision tree. For example, file operations are often grouped by path prefix to minimize evaluation time.

RE Focus: Reversing the Binary Blob:
The kernel receives this blob via the __mac_syscall interface (specifically the sandbox_set_profile command).

  • Extraction: You can intercept these blobs by hooking sandbox_check_common in the kernel or sandbox_compile_entitlements in userland.
  • Decompilation: Tools like sbs (Sandbox Scrutinizer) or custom IDA scripts can lift the bytecode back into a pseudo-SBPL format.
  • The "Profile Validation" Check: In Tahoe, the kernel performs a sanity check on the bytecode. It verifies that the profile does not contain "impossible" instructions or infinite loops that could hang the kernel thread (a DoS vector).

9.2.2 The Sandbox Kernel Extension: Hooking Syscalls via the MAC Framework

The enforcement engine is Sandbox.kext . It hooks into the XNU kernel using the Mandatory Access Control Framework (MACF) .

The MACF Hooks:
XNU is instrumented with hundreds of mac_ hooks placed at critical security bottlenecks.

  • Filesystem: mpo_vnode_check_open , mpo_vnode_check_rename , mpo_vnode_check_unlink .
  • IPC: mpo_mach_port_check_send , mpo_mach_port_check_receive .
  • IOKit: mpo_iokit_check_open_service .

The Evaluation Flow:

  1. Trigger: A sandboxed process calls open("/etc/passwd", O_RDONLY) .
  2. Hook: The kernel executes mac_vnode_check_open .
  3. Dispatch: The MAC framework iterates through registered policies. Sandbox.kext claims the hook.
  4. Credential Lookup: The kext retrieves the kauth_cred_t of the calling process. Attached to this credential is the Sandbox Label , which contains the pointer to the compiled bytecode profile.
  5. Evaluation: The Sandbox engine executes the bytecode against the request arguments.
    • Input: Operation ( file-read ), Path ( /etc/passwd ).
    • Logic: The engine traverses the decision tree.
  6. Caching (The Performance Critical Path):
    Evaluating bytecode on every syscall is prohibitively expensive. Sandbox.kext maintains a Check Cache .
    • If the decision is made, the result is cached in the vnode 's label or a per-process cache.
    • Subsequent accesses to the same file bypass the bytecode engine and use the cached verdict.
    • RE Vulnerability: Cache invalidation bugs are critical. If a file is moved or permissions change, but the Sandbox cache remains stale, enforcement can be bypassed.

The Tahoe/SPTM Intersection:
While the Sandbox policy is software-defined, the integrity of the hooks is hardware-enforced.

  • Immutable Hooks: The function pointers in the mac_policy_conf structure (which point to Sandbox.kext functions) reside in XNU_DATA_CONST .
  • SPTM Enforcement: If a rootkit attempts to unhook the Sandbox by overwriting these pointers, the SPTM will panic the system, as XNU_DATA_CONST is read-only to EL1. This prevents the classic "unhooking" attacks used by older jailbreaks.

9.2.3 Containerization: Data Vaults and Group Containers

In the Tahoe era, Apple has moved beyond simple path-based rules (which are fragile and prone to symlink attacks) toward Semantic Containerization .

Data Vaults:
Data Vaults are the "Nuclear Option" for privacy. They protect sensitive user data (e.g., Messages, Photos, Health) from everyone , including the root user.

  • Implementation: A Data Vault is a directory flagged with a specific VFS attribute (ACL or extended attribute).
  • Enforcement: The kernel checks for this attribute during lookup. Access is denied unless the calling process possesses a specific, restricted entitlement (e.g., com.apple.private.security.storage.AppDataVault ).
  • The "Root" Fallacy: Even if you run as root (UID 0) with host_priv , you cannot ls or cat a Data Vault. The Sandbox check happens before the DAC (Discretionary Access Control) check.
  • RE Focus: Look for rootless_check_datavault_flag in the kernel. This function verifies the entitlement against the vnode's flags.

Group Containers:
To allow IPC and data sharing between apps and their extensions (e.g., a Widget and its parent App), the Sandbox enforces Group Containers .

  • The application-group Entitlement: Defines the shared identifier (e.g., group.com.example.app ).
  • Container Manager: The system daemon containermanagerd manages the lifecycle of these directories ( ~/Library/Group Containers/ ).
  • Sandbox Logic: The profile compiler automatically injects rules allowing read/write access to ${HOME}/Library/Group Containers/<group-id> . This injection is based on the entitlements present in the binary's code signature, binding the filesystem access directly to the cryptographic identity of the executable.

If amfid validates the code's identity and sandboxd restricts the code's reach, tccd is the daemon responsible for managing the most fragile component of the security model: the user.

The Transparency, Consent, and Control (TCC) subsystem is the "User Intent Oracle." It governs access to privacy-sensitive sensors (Camera, Microphone), personal data (Contacts, Calendars, Photos), and privileged system capabilities (Full Disk Access, Screen Recording). In the Tahoe architecture, tccd has evolved from a simple prompt generator into a complex attribution engine that must defend against "Consent Hojacking" and "Attribution Spoofing."

9.3.1 The TCC Database: Schema, Integrity, and SIP

The state of user consent is persisted in SQLite databases. There is a bifurcation of state:

  • System TCC: /Library/Application Support/com.apple.TCC/TCC.db (Root-owned, system-wide permissions like Full Disk Access).
  • User TCC: ~/Library/Application Support/com.apple.TCC/TCC.db (User-owned, per-session permissions like Camera/Microphone).

The Schema:
The core of the database is the access table. For the reverse engineer, the critical columns are:

  • service : The string identifier of the privilege (e.g., kTCCServiceSystemPolicyAllFiles , kTCCServiceMicrophone ).
  • client : The bundle identifier or path of the requesting binary.
  • client_type : 0 (Bundle ID) or 1 (Absolute Path).
  • auth_value : 0 (Denied), 1 (Unknown), 2 (Allowed), 3 (Limited).
  • csreq : The Code Signing Requirement .

The csreq Blob (The Cryptographic Anchor):
TCC does not trust paths. If it did, an attacker could simply overwrite /Applications/TrustedApp.app with malware and inherit its camera permissions.
Instead, TCC stores a compiled Code Signing Requirement (CSReq) blob in the database.

  • When a process requests access, tccd obtains the process's code signature.
  • It validates the running code against the csreq stored in the database row.
  • Logic: (Current_Code_Signature) satisfies (Stored_CSReq) .

Protection Mechanisms:
While the database is a standard SQLite file, access to the file handle is heavily guarded.

  • SIP (System Integrity Protection): The System TCC database is protected by SIP. Even root cannot write to it. Only tccd (which has the com.apple.private.tcc.manager entitlement) and specific Apple-signed updaters can modify it.
  • Data Vaults: In newer macOS versions, the directory containing the database is often flagged as a Data Vault, requiring kernel-enforced entitlements just to traverse the directory structure.

RE Focus: The tccutil Fallacy:
The command-line tool tccutil is often used to reset permissions. However, it communicates with tccd via XPC. It does not touch the DB directly. Reverse engineering tccd reveals that it holds an exclusive lock on the DB handle, and any attempt to modify the DB file directly (e.g., via a bypass) usually results in database corruption or the daemon crashing and reloading from a backup, due to WAL (Write-Ahead Log) integrity checks.

9.3.2 The Attribution Chain: Determining Who is Asking

The hardest problem tccd solves is Attribution . When a user clicks a button in an app, and that app asks a daemon to take a photo, who is the "Client"? The App? Or the Daemon?

The XPC Audit Token:
When a message arrives at tccd (via com.apple.tccd.system or com.apple.tccd ), the primary identifier is the Audit Token attached to the Mach message trailer.

  • tccd parses the token to retrieve the PID, UID, and GID.
  • It uses csops (or the kernel-backed CS_VALID flag) to ensure the PID maps to a valid, signed binary.

The "Responsible Process" Problem:
Consider Terminal.app running a shell script that calls curl . If curl asks for network access, the user should see "Terminal.app would like to access...", not "curl".
To handle this, TCC supports Attribution Chains .

  1. responsible_pid : A process can designate another process as responsible for its actions (using launchd attributes or specific XPC flags).
  2. Validation: tccd verifies the relationship. It checks if the requesting process is a child of the responsible process, or if the responsible process possesses the com.apple.private.tcc.allow entitlement which permits it to proxy requests.

The Access Object:
Internally, tccd creates a TCCAccessRequest object.

  • Subject: The process requesting access.
  • Accessor: The process that will actually touch the resource (often the same as Subject).
  • Attributor: The process to be displayed in the UI.

RE Focus: The tccd State Machine:
By analyzing the service_listener function in tccd , one can map the decision logic:

  1. Entitlement Check: Does the client have com.apple.private.tcc.allow for this service? If yes, Grant immediately (no prompt).
  2. Database Lookup: Does a row exist in TCC.db ?
    • If yes, validate csreq . If valid, return auth_value .
  3. User Prompt: If no record exists, construct a UserNotification .
    • Secure UI: The prompt is not drawn by tccd . It is handed off to UserNotificationCenter (and CoreServicesUIAgent ). This prevents the requesting app from drawing over the prompt (Clickjacking) because the prompt window resides in a higher window level (Shield Window) managed by WindowServer.

9.3.3 RE Focus: XPC Attacks against TCC Endpoint Validation

For the vulnerability researcher, tccd is a prime target. If you can trick tccd into believing you are Finder.app , you gain Full Disk Access.

Attack Vector 1: XPC Injection:
Many apps load bundles or plugins. If an app with TCC permissions (e.g., a Video Editor with Camera access) loads a malicious dylib (via dylib hijacking or unsafe dlopen ), that dylib runs inside the privileged process.

  • TCC's View: tccd sees the PID of the Video Editor. It grants access.
  • Mitigation: Hardened Runtime and Library Validation (enforced by dyld and TXM) prevent loading unsigned dylibs. However, if the host app has the com.apple.security.cs.disable-library-validation entitlement, this vector is wide open.

Attack Vector 2: The "Fake" Attributor:
Attackers attempt to spoof the XPC dictionary sent to tccd .

  • The XPC message often contains a TCCAccessRequest dictionary.
  • Historically, researchers found they could inject a fake target_token into this dictionary.
  • Current Hardening: tccd now ignores user-supplied tokens in the XPC payload for most checks. It relies almost exclusively on the kernel-supplied Audit Token from the IPC trailer, which cannot be spoofed from userland.

Attack Vector 3: Semantic Confusion (The "Open" Trap):
An attacker uses the open command or AppleEvents to coerce a privileged app to perform an action.

  • Example: Telling Terminal to execute a script.
  • Mitigation: macOS introduced "Automation" permissions ( kTCCServiceAppleEvents ). App A cannot send AppleEvents to App B unless explicitly authorized by the user. tccd enforces this by intercepting the AppleEvent manager's IPC.

The "Tahoe" Impact:
On Apple Silicon, the TXM plays a subtle but vital role here. tccd relies on the Code Signing flags ( CS_VALID , CS_HARD , CS_KILL ) to trust the client. In Tahoe, these flags are ultimately enforced by the TXM.

  • If an attacker manages to patch the kernel to set CS_VALID on a malicious binary, the SPTM will still refuse to map the pages as executable if the TXM hasn't blessed the CDHash.
  • Therefore, tccd 's reliance on code signing is anchored in the hardware root of trust. A kernel compromise does not trivially allow bypassing TCC, because the attacker cannot easily forge the immutable identity of a system binary that tccd trusts.

10.0 User Session, Authentication & Data Protection

The transition from the system bootstrap phase to the interactive user session represents a critical boundary crossing. Up to this point, the system has operated primarily in the Startup domain, managed by the root launchd context. The instantiation of a user session requires the creation of a new security context—the Audit Session —and the decryption of user-specific cryptographic material anchored in the Secure Enclave.

In the Tahoe architecture, this process is not merely a check of a password hash against a file. It is a hardware-mediated ceremony involving the unwrapping of the Secure Token , the derivation of the User Keybag , and the establishment of a Kerberized identity that binds the user's biological presence (Biometrics) to their digital authority.

10.1 loginwindow & opendirectoryd

The graphical login experience is orchestrated by two primary userland daemons: loginwindow , which manages the session lifecycle and UI, and opendirectoryd , which serves as the abstraction layer for authentication and identity services. While these components are legacy codebases dating back to NeXTSTEP, their internals have been aggressively refactored to support the hardware-backed security model of Apple Silicon.

10.1.1 The Audit Session ID (ASID) and Kernel Tracking

In XNU, the concept of a "User" is tracked via the Audit Session ID (ASID) . This is distinct from the UNIX UID/GID. The ASID identifies a specific login instance (e.g., the graphical console session vs. an SSH session).

The setaudit_addr Syscall:
When loginwindow successfully authenticates a user, it does not simply setuid . It invokes the setaudit_addr syscall (wrapped by libbsm ).

  • Kernel Structure: This syscall populates the audit_token_t structure associated with the process.
  • Immutability: Once an ASID is set for a process, it is inherited by all children. Crucially, XNU enforces that a process cannot change its ASID once set (with very specific exceptions for sshd and loginwindow holding the com.apple.private.audit.session.create entitlement).

The Security Implications of ASID:
The ASID is the primary signal used by tccd and WindowServer to determine "Console Ownership."

  • TCC: When a process requests Camera access, tccd checks if the requesting process's ASID matches the ASID of the active graphical session. If it matches, the user is prompted. If it is a background session (e.g., SSH), the request is auto-denied or routed differently.
  • WindowServer: Only processes belonging to the active Console ASID are permitted to connect to the WindowServer and draw on the screen.

RE Focus: The security_authtrampoline :
Historically, the transition from the loginwindow context (root) to the user context involved a setuid binary called security_authtrampoline . In modern macOS, this logic is largely internalized within loginwindow 's interaction with launchd .

  • loginwindow calls xpc_session_create .
  • This triggers launchd to spawn a new instance of itself (the User launchd ) running as the target user.
  • This User launchd becomes the root of the user's process tree.

10.1.2 loginwindow : The Shield Window and Session State

loginwindow (located at /System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow ) is the session leader. It is responsible for drawing the login GUI, validating credentials, and managing the "Shield Window."

The Shield Window (Anti-Overlay):
To prevent "Clickjacking" or "Fake Login Screen" attacks where a malicious application draws a window over the login prompt to capture keystrokes, loginwindow utilizes a privileged connection to SkyLight (the WindowServer framework).

  • Window Level: The login UI is drawn at the kCGShieldingWindowLevel (or kCGCursorWindowLevel - 1). This is a Z-order reserved for the system.
  • Event Isolation: When the Shield Window is active, the WindowServer routes all keyboard and mouse events exclusively to loginwindow , regardless of what other applications might be running in the background.

The State Machine:
Reverse engineering loginwindow reveals a complex state machine driven by AppleEvents and Notificiations .

  • LoginHook / LogoutHook : While officially deprecated, the code paths for these legacy customization scripts still exist, though they are heavily sandboxed.
  • TAL (Transparent App Lifecycle): loginwindow manages the "Resume" feature (re-opening windows after reboot). This state is persisted in ~/Library/Preferences/ByHost/com.apple.loginwindow.*.plist , protected by the user's Data Protection keys.

10.1.3 opendirectoryd : The Authentication Broker

opendirectoryd is the daemon responsible for answering the question: "Is this password correct?" It is a modular daemon that loads plugins (bundles) to handle different directory services (LDAP, Active Directory, Local).

The Local Node ( /Local/Default ):
On a standalone Mac, authentication is handled by the Local Node . The data store for this node is located at /var/db/dslocal/nodes/Default/ .

  • Users: Stored as individual Property List ( .plist ) files in users/ .
  • Shadow Hash: The user's password hash is not stored in the plist. It is stored in a separate, restricted file structure known as the Shadow Hash , often embedded as binary data within the plist but stripped of read permissions for non-root users.

The Shadow Hash Data:
The ShadowHashData key in the user's plist contains a binary blob. In the Tahoe era, this blob is not a simple SHA-512 hash. It is a complex structure containing:

  1. SALTED-SHA512-PBKDF2: A standard hash for legacy compatibility (often disabled or randomized on FileVault-enabled systems).
  2. Secure Token Wrapper: The primary authentication secret. This is a blob encrypted by a key derived from the Secure Enclave .

The Verification Flow ( ODRecordVerifyPassword ):
When loginwindow passes a password to opendirectoryd :

  1. od_verify_password : The request is routed to the Local Node module.
  2. AppleSEPKeyStore Interaction: opendirectoryd cannot verify the Secure Token wrapper itself. It passes the plaintext password and the wrapped blob to the kernel extension AppleSEPKeyStore.kext .
  3. SEP Round Trip: The kernel marshals this request to the Secure Enclave via the Mailbox.
  4. Unwrapping: The SEP mixes the hardware UID with the user's password (tangling). It attempts to unwrap the blob.
  5. Verdict: If the unwrap succeeds, the SEP returns a success signal (and potentially a derived key for FileVault). If it fails, it returns an authentication error.

RE Implication:
This architecture means that offline cracking of macOS password hashes (extracted from dslocal ) is effectively impossible for FileVault-enabled users on Apple Silicon. The hash is not mathematically complete without the hardware UID of the specific SEP that created it.

10.1.4 Kerberos and the Local KDC (Heimdal)

A little-known fact about modern macOS is that every machine runs a Local Key Distribution Center (LKDC) . This is a stripped-down Heimdal Kerberos KDC running within opendirectoryd .

Why Kerberos on a Laptop?
Apple uses Kerberos to handle "Single Sign-On" between system services without passing the plaintext password around.

  1. Initial Login: When the user logs in, opendirectoryd obtains a Ticket Granting Ticket (TGT) from the Local KDC using the user's password.
  2. Credential Cache: This TGT is stored in the kernel-managed credential cache (accessible via the CCACHE server).
  3. Service Auth: When the user tries to unlock a System Preference pane, use Screen Sharing, or access the Keychain, the system requests a Service Ticket from the Local KDC.
  4. Validation: The service verifies the ticket. If valid, access is granted.

The "Smart Card" Enforcement:
In high-security environments (and increasingly the default in Tahoe), the Local KDC is backed by PKINIT .

  • If the user logs in with a Smart Card (or the built-in PIV token backed by the Secure Enclave), the KDC validates the x.509 certificate chain instead of a password hash.
  • The TXM plays a role here by enforcing that the Smart Card driver (running as a dext ) is valid and has not been tampered with, ensuring the certificate presented to the KDC is authentic.

RE Focus: Heimdal Framework:
The Heimdal.framework in the PrivateFrameworks directory contains the logic for this. Hooking krb5_get_init_creds_password in loginwindow is a common technique for Red Teams to intercept plaintext credentials before they are hashed, though this requires bypassing SIP and the hardened runtime.

10.2 Biometric Unlock (Touch ID / Face ID / Optic ID)

Biometric authentication on Apple platforms is frequently misunderstood as a replacement for the passcode. Architecturally, it is a convenience mechanism that authorizes the Secure Enclave to perform a cryptographic operation (unwrapping a key) that would otherwise require the user's manual secret entry.

In the Tahoe architecture, the biometric stack is a high-stakes interplay between userland daemons, kernel drivers, the Secure Enclave, and—in the case of Face ID and Optic ID—the Secure Neural Engine (SNE) .

10.2.1 The Daemon Hierarchy: coreauthd $\rightarrow$ biometrickitd $\rightarrow$ SEP

The userland implementation is split into two primary daemons to enforce separation of concerns: coreauthd (Policy) and biometrickitd (Mechanism).

coreauthd (The Policy Engine):
Located at /System/Library/Frameworks/LocalAuthentication.framework/Support/coreauthd , this daemon manages the Local Authentication (LA) context.

  • Context Management: When an app calls LAContext.evaluatePolicy , coreauthd creates a unique context ID ( ACMContext ).
  • ACL Evaluation: It parses the Access Control Lists (ACLs) attached to Keychain items. It determines what is required (e.g., "Biometry OR Passcode").
  • The ACM (Access Control Module): Inside coreauthd , the ACM is a library that mirrors the logic of the Secure Enclave. It prepares the request structures that will eventually be sent to the SEP.

biometrickitd (The Hardware Manager):
Located at /usr/libexec/biometrickitd , this daemon is responsible for the physical sensors.

  • Sensor Abstraction: It loads the specific plugin for the device's hardware (Mesa for Touch ID, Pearl for Face ID, Jade for Optic ID).
  • Hardware State: It manages the power state of the sensor and the emitter (e.g., the Flood Illuminator or IR Projector).
  • Data Relay: Crucially, biometrickitd does not process biometric data . It acts as a dumb pipe. It sets up a Direct Memory Access (DMA) channel or a shared memory buffer between the sensor hardware and the Secure Enclave.

The Handshake:

  1. coreauthd receives a request and validates the PID/ASID.
  2. coreauthd sends an XPC request to biometrickitd to "Arm" the sensor.
  3. biometrickitd issues an IOConnectCall to the kernel driver ( AppleBiometricSensor.kext ).
  4. The kernel driver wakes the SEP via the Mailbox and prepares the hardware interface.

10.2.2 The Hardware Path: Sensor-to-SEP Pairing

A critical security property of the biometric stack is the Hardware Pairing between the sensor and the SEP. This prevents "Evil Maid" attacks where an attacker replaces the fingerprint sensor with a malicious FPGA that replays a stored "match" signal.

The Factory Pairing:
During manufacturing, the sensor (e.g., the Touch ID button or the TrueDepth camera module) and the SEP exchange public keys. A shared secret is established and burned into the sensor's controller.

The Encrypted Channel:
When the sensor captures an image (fingerprint ridge map or IR depth map):

  1. Encryption: The sensor hardware encrypts the raw image data using the shared pairing key before it leaves the sensor module.
  2. Transport: The encrypted data travels over the SPI or MIPI bus to the Application Processor.
  3. Passthrough: The AP (kernel) writes this encrypted blob into a physical memory buffer shared with the SEP. The AP cannot decrypt this data; it sees only high-entropy noise.
  4. Decryption: The SEP reads the buffer, decrypts it using the pairing key, and processes the raw biometric data.

RE Implication:
If you hook the AppleBiometricSensor kext and dump the data buffers, you will not get a fingerprint image. You will get ciphertext. Furthermore, if you replace the sensor with a component from another iPhone (without running the Apple System Configuration tool to re-pair), the SEP will fail to decrypt the data, and biometrics will be disabled.

10.2.3 The Secure Neural Engine (SNE) & Optic ID

With the advent of Face ID and Optic ID (Vision Pro), the computational cost of biometric matching exceeded the capabilities of the general-purpose SEP core. Apple introduced the Secure Neural Engine (SNE) .

Architecture:
The Neural Engine (ANE) on the SoC is partitioned.

  • Standard Mode: Accessible to userland (CoreML) for standard ML tasks.
  • Secure Mode: A dedicated slice of the ANE is reserved for the SEP.

The Optic ID Flow (Tahoe/Vision Pro):

  1. Capture: The iris cameras capture spatiotemporally modulated IR images.
  2. Transfer: The encrypted images are DMA'd to the SEP's protected memory.
  3. SNE Handoff: The SEP sanitizes the ANE (flushing previous weights/data) and loads the Optic ID neural network model.
  4. Processing: The SEP feeds the image data into the SNE. The SNE performs the feature extraction (generating the mathematical representation of the iris).
  5. Matching: The SNE returns the feature vector to the SEP. The SEP compares this vector against the enrolled templates stored in the Secure Storage Component (xART) .
  6. Liveness: Simultaneously, the SEP analyzes the "spatiotemporal modulation" (the reaction of the pupil to light pulses) to ensure the subject is a living eye and not a contact lens or high-res display.

Memory Protection:
On M3/M4 chips, the memory used by the SNE during this process is protected by a dedicated Hardware Security Controller . If the AP attempts to read SNE memory while it is in Secure Mode, the transaction is blocked, and a system panic is triggered.

10.2.4 Secure Intent: The GPIO Hardline

For high-value transactions (Apple Pay, Smart Card pairing), biometric matching is insufficient. Malware could theoretically trick the user into looking at the phone (Face ID match) to authorize a background payment.

To solve this, Apple implements Secure Intent via a physical side-channel.

The Physical Button:
The power button (or side button) is wired to the Always-On Processor (AOP) and the AP, but it also has a dedicated GPIO line connected directly to the Secure Enclave .

The Logic:
When a transaction requires Secure Intent (flagged in the LAContext ):

  1. The SEP performs the biometric match.
  2. If successful, the SEP holds the authorization. It does not release the token.
  3. The SEP monitors its private GPIO pin.
  4. The Window: The user must double-click the physical button within a specific time window (usually < 2 seconds) of the biometric match.
  5. Release: Only when the SEP detects the electrical signal of the double-click on its private line does it sign the payment token or release the key.

RE Focus:
This mechanism renders UI spoofing attacks ineffective for Apple Pay. Even if an attacker controls WindowServer and draws a fake "Double Click to Pay" overlay, and even if the user looks at the phone (satisfying Face ID), the attacker cannot generate the electrical signal on the SEP's GPIO pin.

10.2.5 The LAC (Local Authentication Context) and Token Binding

When authentication succeeds, the SEP does not simply return true . It returns a cryptographic token.

The ACMHandle :
coreauthd receives an opaque handle from the SEP. This handle refers to a session in the SEP's memory.

Token Binding:
When accessing a Keychain item protected by kSecAccessControlUserPresence :

  1. securityd (Keychain daemon) sends the encrypted keyblob (wrapped key) and the ACMHandle to the SEP.
  2. The SEP verifies the ACMHandle is valid and associated with a recent biometric match.
  3. The SEP uses its internal hardware key to unwrap the keyblob.
  4. The SEP returns the raw AES key to securityd (or performs the crypto operation internally if the key is non-extractable).

The "Backoff" Strategy:
The SEP enforces the retry policy in hardware.

  • Counter: Stored in xART .
  • Logic:
    • 1-2 failures: Retry allowed immediately.
    • 3+ failures: Force delay.
    • 5 failures: Biometrics disabled. Passcode required.
    • Global Reset: If the device is rebooted, or if 48 hours pass, the SEP invalidates the biometric keys in memory, forcing a passcode entry to re-derive the master keys. This logic is encoded in the sepOS and cannot be patched by the kernel.

10.3 Data Protection & FileVault

On Intel Macs, FileVault was a distinct full-disk encryption layer (CoreStorage) that operated below the filesystem. On Apple Silicon, this distinction has vanished. FileVault is simply Data Protection applied to the Volume Key.

The architecture of macOS Tahoe unifies the security model of iOS and macOS. Every file on the SSD is individually encrypted with a per-file key, wrapped by a class key, protected by the Secure Enclave. "Turning on FileVault" merely changes the protection policy of the Volume Encryption Key (VEK) from "Protected by Hardware UID" to "Protected by User Password."

This section dissects the cryptographic hierarchy, the hardware-accelerated AES path that keeps keys out of RAM, and the Sealed Key Protection (SKP) mechanism that binds data availability to the system's boot measurement.

10.3.1 Unwrapping the User Keybag: The Class Key Hierarchy

The central data structure for Data Protection is the Keybag . It is a binary property list stored on disk (typically in /private/var/db/UserKeyStash or similar locations depending on the boot volume layout), containing the wrapped class keys.

The Hierarchy:

  1. Hardware UID: The root of trust, fused into the SEP. Accessible only to the SEP's AES engine.
  2. User Password: The user's login secret.
  3. Passcode Derived Key (PDK): The SEP mixes the UID and the User Password using a proprietary KDF (Tangling).
    $$ \text{PDK} = \text{Tangle}(\text{UID}, \text{PBKDF2}(\text{Password}, \text{Salt})) $$
  4. Class Keys: The Keybag contains the keys for specific protection classes (A, B, C, D), wrapped by the PDK.

The Classes:

  • Class A (Complete Protection): Keys are evicted from SEP memory immediately upon device lock.
  • Class B (Protected Unless Open): Keys are evicted on lock, but open file handles retain an ephemeral unwrapped key in the AES engine context to allow background writes to finish.
  • Class C (First Unlock): Keys remain in SEP memory after the first successful authentication until reboot. This is the class used for the FileVault Volume Key.
  • Class D (No Protection): Keys are wrapped only by the UID. They are accessible as long as the device is booted, even without a password. Used for system daemons that must start before login.

The Unwrapping Ceremony:
When the user logs in:

  1. loginwindow passes the password to the kernel ( AppleSEPKeyStore ).
  2. The kernel passes the password and the Keybag blob to the SEP via Mailbox.
  3. The SEP performs the Tangle operation to derive the PDK.
  4. The SEP attempts to unwrap the Class Keys in the Keybag using the PDK.
  5. Crucially: The unwrapped Class Keys are never returned to the kernel. They remain resident in the SEP's protected SRAM. The SEP returns a "Handle" (Key ID) to the kernel.

10.3.2 Sealed Key Protection (SKP): Binding Data to Measurement

In the Tahoe architecture, Apple introduced Sealed Key Protection (SKP) to mitigate "Evil Maid" attacks where an attacker boots a malicious operating system (or a vulnerable older version) to brute-force the keybag.

SKP binds the availability of the keys to the Measurement of the boot chain.

The Measurement Flow:

  1. Boot Monitor: Measures the sepOS .
  2. sepOS : Measures the KernelCache and the LocalPolicy .
  3. PCR Extension: These measurements are extended into registers within the SEP (conceptually similar to TPM PCRs).

The SKP Key:
When the Volume Encryption Key (VEK) is generated (at install time), it is wrapped not just by the UID/PDK, but by a key derived from the Current System Measurement .

$$ \text{WrappedVEK} = \text{Encrypt}(\text{VEK}, \text{KDF}(\text{UID}, \text{PDK}, \text{Measurement})) $$

The Security Guarantee:
If an attacker attempts to boot a custom kernel (Permissive Security) or a downgraded kernel to attack the Data Protection implementation:

  1. The Boot Monitor/ sepOS calculates a different measurement hash.
  2. The KDF derives a different wrapping key.
  3. The SEP fails to unwrap the VEK.
  4. The Data Volume remains inaccessible.

This forces the attacker to possess the user's credentials and boot a signed, measured, and authorized operating system to access the data.

10.3.3 The Hardware AES Engine & The "Wrapped Key" Path

A common misconception is that the kernel decrypts file data. In reality, the XNU kernel never sees the raw AES keys for file contents. Apple Silicon utilizes a dedicated AES DMA Engine that sits between the Memory Controller and the Storage Controller (ANS2).

The Inline Encryption Path:

  1. File I/O: When a user process reads a file, the kernel looks up the file's metadata to find the Per-File Key (wrapped by a Class Key).
  2. The Request: The kernel sends the Wrapped Per-File Key and the Class Key Handle to the SEP.
  3. SEP Translation: The SEP unwraps the Per-File Key using the resident Class Key.
  4. The Side Channel: The SEP does not return the raw key to the kernel. Instead, it programs the AES Engine's key registers directly via a dedicated hardware bus (or re-wraps the key with an ephemeral key known only to the AES Engine).
  5. DMA: The kernel initiates the DMA transfer from NAND.
  6. Decryption: The AES Engine decrypts the data on-the-fly as it flows into DRAM. The data lands in the page cache as plaintext, but the key that decrypted it effectively vanishes.

RE Implication:
Memory forensics on a running macOS kernel cannot recover the file encryption keys. They simply do not exist in kernel memory. To recover the keys, one must compromise the SEP firmware and dump its SRAM.

10.3.4 RE Focus: Analyzing the AppleSEPKeyStore Kernel Extension

The interface between the kernel and the SEP's key management logic is the AppleSEPKeyStore kext. This binary is a prime target for understanding the proprietary IPC protocol.

Key Symbols:

  • aks_unwrap_key : The primary function for unwrapping a keybag element.
  • aks_get_lock_state : Queries the SEP for the current lock state (Locked/Unlocked).
  • sep_key_store_client_handle_message : The demuxer for messages received from the SEP.

The Attack Surface:
The communication protocol involves complex Type-Length-Value (TLV) structures.

  • Handle Confusion: The kernel refers to keys by 32-bit integer handles. A potential vulnerability class involves guessing or forging a handle to trick the SEP into using a key belonging to a different security domain (e.g., using a System Key to unwrap User Data).
  • Race Conditions: The AppleSEPKeyStore maintains a shadow state of the lock status. Discrepancies between the kernel's view of the lock state and the SEP's actual state can lead to logic bugs where the kernel allows an operation that the SEP should have blocked, or vice versa.

Tahoe Hardening:
In Tahoe, AppleSEPKeyStore operations are increasingly gated by the TXM . Critical key operations (like enabling FileVault or changing the recovery key) require a GENTER transition to GL1 to verify the policy before the request is forwarded to the SEP, adding an additional layer of authorization check that cannot be bypassed by kernel patching.

11.0 Conclusion: The Attack Surface Landscape

The architectural transformation introduced with macOS Tahoe and the M3/M4 silicon generation signifies the end of the "Kernel is King" era. We have moved from a monolithic trust model, where uid=0 and tfp0 were the ultimate objectives, to a federated security model where the kernel is merely a highly privileged, yet strictly supervised, tenant within a hardware-enforced hypervisor.

For the vulnerability researcher, this necessitates a shift in methodology. Fuzzing syscalls is no longer sufficient to compromise the system's root of trust. The new frontier lies in the Boundary Crossings —the specific, hardware-mediated bridges that allow data and execution flow to traverse the isolated domains.

11.1 Summary of Boundary Crossings

The following matrix details the architectural boundaries, the mechanisms used to traverse them, and the specific attack surface exposed at each junction.

11.1.1 Userland (EL0) $\leftrightarrow$ Kernel (EL1)

The Traditional Boundary, Hardened by Silicon.

  • Transition Mechanism:
    • Entry: SVC (Supervisor Call) instruction triggering a synchronous exception to VBAR_EL1 + 0x400 .
    • Exit: ERET (Exception Return) restoring PC and PSTATE from ELR_EL1 and SPSR_EL1 .
  • Hardware Enforcement:
    • PAC: Entry points are signed. The kernel verifies the thread state signature ( kauth_thread_state ) on return.
    • PPL/SPTM: The kernel cannot modify its own text or page tables to disable SMEP/SMAP equivalents ( PAN / PXN ).
  • The Tahoe Shift:
    • The kernel is no longer the final arbiter of virtual memory. When a user process requests mmap(RWX) , the kernel cannot simply write to the translation table. It must request the SPTM to map the page.
    • Attack Surface:
      • Logic Bugs: Standard memory corruption (UAF, Heap Overflow) in kernel extensions still yields EL1 execution.
      • PAC Bypasses: Forging pointers to survive the ERET or function pointer authentication.
      • Argument Sanitization: The kernel must sanitize user pointers before passing them to the SPTM. A "Confused Deputy" attack where the kernel is tricked into asking the SPTM to map a privileged page into user space is the new tfp0 .

11.1.2 Kernel (EL1) $\leftrightarrow$ Secure Page Table Monitor (GL2)

The "Mechanism" Boundary: The New Hypervisor.

  • Transition Mechanism:
    • Entry: GENTER (Opcode 0x00201420 ) with Selector ID in x15 .
    • Exit: GEXIT (Opcode 0x00201400 ).
  • Hardware Enforcement:
    • GXF: Hardware context switch of SP_EL1 $\rightarrow$ SP_GL2 .
    • SPRR: Atomic switch of permission views. Kernel text becomes RO/NX; SPTM text becomes RX.
  • Data Exchange:
    • Registers: x0 - x7 carry physical addresses (PPNs) and permission bitmaps.
    • Shared Memory: None. The SPTM reads physical memory directly via its own linear map.
  • Attack Surface:
    • State Machine Confusion: The SPTM enforces a Finite State Machine (FSM) on every physical page (Frame Table). The primary attack vector is finding a sequence of retype / map / unmap calls that desynchronizes the SPTM's view of a page from the hardware's actual state (e.g., aliasing a PAGE_TABLE as XNU_DATA ).
    • Input Validation: Passing invalid physical addresses or edge-case permission combinations to sptm_map .
    • Panic-as-Oracle: Since SPTM panics on failure, timing side-channels or fault injection during the GENTER window are potential vectors to infer the memory layout of GL2.

11.1.3 Kernel (EL1) $\leftrightarrow$ Trusted Execution Monitor (GL1)

The "Policy" Boundary: The Signature Oracle.

  • Transition Mechanism:
    • Entry: GENTER (Targeting the TXM Dispatch Table).
  • Hardware Enforcement:
    • SPRR: TXM memory is visible only in GL1 context.
  • Data Exchange:
    • Physical Pointers: The kernel passes the physical address of a Code Directory or CMS blob.
    • Trust Cache: The kernel requests updates to the Dynamic Trust Cache.
  • Attack Surface:
    • Parsing Complexity: The TXM must parse complex, legacy formats (ASN.1, CMS, Mach-O headers) to verify signatures. This parsing happens in the highest privilege level. A buffer overflow in the TXM's ASN.1 parser is a "God Mode" exploit.
    • TOCTOU: The kernel loads the signature into memory, passes the physical address to TXM, and then TXM verifies it. If the kernel (or a DMA device) can modify that memory during the TXM's verification window, the signature check can be bypassed.
    • Logic Downgrades: Tricking the TXM into believing the device is in "Developer Mode" or "Permissive Mode" via corrupted NVRAM or LocalPolicy parsing.

11.1.4 Kernel (EL1) $\leftrightarrow$ Secure Enclave (SEP)

The "Air Gap" Boundary: The Parallel Computer.

  • Transition Mechanism:
    • Asynchronous IPC: Mailbox Registers (Doorbell) + Shared Memory Buffers (DART-mapped).
  • Hardware Enforcement:
    • Physical Isolation: Distinct CPU core, distinct MMU.
    • Memory Protection Engine: SEP memory is encrypted/authenticated inline.
  • Data Exchange:
    • Serialized Messages: L4 IPC format (Endpoints, TLV payloads).
    • Wrapped Keys: Keys are passed as opaque blobs; raw key material never crosses this boundary.
  • Attack Surface:
    • Message Parsing: Fuzzing the sepOS endpoint handlers (e.g., biometrickitd , securekeyvault ).
    • Shared Memory Races: Modifying the contents of a DART-mapped buffer after the SEP has validated the header but before it processes the payload.
    • Anti-Replay Logic: Attempting to rollback the xART storage state to force the SEP to reuse old nonces or counters.

11.1.5 Kernel (EL1) $\leftrightarrow$ Exclaves (Secure Domain)

The "Microkernel" Boundary: The RingGate.

  • Transition Mechanism:
    • RingGate: XNUProxy kext marshals data $\rightarrow$ GENTER (to Secure Kernel) $\rightarrow$ IPC to Conclave.
  • Hardware Enforcement:
    • SPTM: Enforces physical memory isolation between XNU_DOMAIN and SK_DOMAIN .
  • Data Exchange:
    • Tightbeam: A strongly-typed IDL serialization format.
  • Attack Surface:
    • Proxy Confusion: Exploiting XNUProxy to route messages to the wrong Conclave.
    • IDL Deserialization: Bugs in the Tightbeam generated code within the Exclave.
    • Resource Exhaustion: Flooding the Secure Kernel with Downcalls to starve secure workloads (DoS).

11.2 The "Intel Gap": Security Disparities between x86 and Apple Silicon

While macOS Tahoe presents a unified user experience across architectures, the underlying security reality is a tale of two operating systems. On Apple Silicon, macOS is a hypervisor-managed, hardware-attested fortress. On Intel (x86_64), it remains a traditional monolithic kernel relying on legacy protection mechanisms. This divergence has created a massive "Intel Gap"—a disparity in exploit mitigation so severe that the same vulnerability often yields a trivial root shell on Intel while resulting in a harmless panic on Apple Silicon.

For the reverse engineer, understanding this gap is essential for targeting. The Intel architecture represents the "Soft Target," lacking the silicon-enforced boundaries of the SPTM, TXM, and PAC.

11.2.1 The Absence of Lateral Privilege (Ring 0 is King)

The most profound difference lies in the privilege hierarchy.

  • Apple Silicon: As detailed in Section 4.0, the kernel (EL1) is deprivileged. It is subservient to the SPTM (GL2) and TXM (GL1). Achieving arbitrary kernel read/write (KRW) is merely the first step in a long chain of exploitation. The attacker must still bypass the hardware-enforced page table protection and code signing logic.
  • Intel x86: The kernel runs in Ring 0 (Supervisor Mode). There is no "Ring -1" hypervisor enforcing integrity on the host macOS kernel. Apple never implemented a Windows VBS/HVCI-equivalent on Intel Macs.
    • Consequence: On Intel, KRW is Game Over. If an attacker can write to kernel memory, they can disable SIP, unhook the Sandbox, patch amfid , and overwrite page tables directly. There is no hardware entity above Ring 0 to say "No."

11.2.2 Static vs. Dynamic Kernel Integrity (KTRR vs. SPTM)

Both architectures attempt to enforce Kernel Text Read-Only Region (KTRR) , but the implementation differs fundamentally in flexibility and robustness.

  • Intel (Hardware KTRR): On recent Intel Macs, KTRR is implemented via proprietary memory controller registers (configured via MSR ).
    • Mechanism: The firmware locks a physical range of memory as Read-Only/Executable.
    • Limitation: This is Static . Once the range is locked at boot, it cannot change. This forces the kernel to fit all immutable code into a contiguous block. It cannot protect dynamically loaded drivers (KEXTs) with the same hardware rigor. KEXTs rely on software-managed page tables ( CR0.WP bit), which a compromised kernel can disable.
  • Apple Silicon (SPTM):
    • Mechanism: The SPTM manages the Frame Table.
    • Advantage: This is Dynamic . The kernel can load a new extension (AKC), link it, and then ask the SPTM to "Seal" it. The SPTM transitions those specific pages to XNU_TEXT . This allows the "Immutable Kernel" coverage to extend to late-loaded drivers, a feat impossible on the static Intel KTRR implementation.

11.2.3 The CFI Chasm: PAC vs. CET

Control Flow Integrity (CFI) is the primary defense against ROP/JOP.

  • Apple Silicon: Pointer Authentication (PAC) is ubiquitous. It protects return addresses (stack), function pointers (heap/data), and C++ vtables. It provides cryptographic diversity based on pointer context.
  • Intel x86: Intel Macs support Control-flow Enforcement Technology (CET) , specifically Shadow Stacks ( IBT support is limited).
    • The Gap: CET Shadow Stacks protect return addresses effectively, but they do not protect Forward-Edge transfers (function pointers) with the same granularity as PAC.
    • Data Pointers: Crucially, Intel has no equivalent to APDAKey (Data Key). An attacker on Intel can still perform Data-Oriented Programming (DOP) —swapping valid object pointers or corrupting decision-making flags—without triggering a hardware fault. On Apple Silicon, these pointers are signed; forging them requires a signing gadget.

11.2.4 The Root of Trust: T2 vs. On-Die Boot ROM

The boot chain trust anchor differs physically.

  • Intel: The Root of Trust is the Apple T2 Security Chip (on models 2018-2020).
    • The Weakness: The T2 is a discrete bridge. It verifies the boot.efi and kernelcache signature before the Intel CPU starts. However, once the Intel CPU is executing, the T2 is effectively a peripheral connected via USB/PCIe. It cannot introspect the Intel CPU's execution state. It cannot stop a runtime kernel exploit.
  • Apple Silicon: The Root of Trust is the AP Boot ROM .
    • The Strength: The security logic (SEP, PKA, Boot Monitor) is on the same die . The Secure Enclave can monitor the power and clock lines of the AP. The SPTM (running on the AP) enforces the boot measurements continuously. The trust chain is not "handed off"; it is maintained throughout the runtime lifecycle.

11.2.5 I/O Security: VT-d vs. DART

DMA attacks are a classic method to bypass CPU memory protections.

  • Intel: Uses VT-d (Intel Virtualization Technology for Directed I/O).
    • Configuration: The kernel configures the IOMMU tables.
    • Vulnerability: If the kernel is compromised, it can reconfigure VT-d to allow a malicious Thunderbolt device to overwrite kernel memory (unless strict "DMA Protection" is enabled and locked, which relies on the kernel's integrity).
  • Apple Silicon: Uses DART (Device Address Resolution Table).
    • Enforcement: As detailed in Section 7.2.2, the kernel cannot write to DART registers. Only the SPTM can map I/O memory.
    • Result: Even a compromised kernel cannot weaponize a peripheral to perform a DMA attack against the monitor or the kernel text, because the SPTM will reject the mapping request.

11.2.6 Summary Table: The Security Disparity

Feature Intel Mac (x86_64) Apple Silicon (arm64e)
Highest Privilege Ring 0 (Kernel) GL2 (SPTM)
Page Table Protection Software ( CR0.WP ) Hardware (SPRR + SPTM)
Kernel Integrity Static KTRR (Boot only) Dynamic KIP (Runtime Sealing)
CFI CET (Shadow Stack) PAC (Cryptographic Signatures)
Vtable Protection None (Plaintext Pointers) Signed Pointers ( APDAKey )
Code Signing Kernel-enforced ( AMFI.kext ) Monitor-enforced (TXM)
DMA Protection Kernel-managed VT-d Monitor-managed DART
Secure Enclave Discrete (T2 Chip) Integrated (On-Die)
Exploit Consequence Persistent Rootkit Possible System Panic / Non-Persistence

Conclusion for the Researcher:
The "Intel Gap" means that legacy Intel Macs are essentially running a different, far more vulnerable operating system, despite sharing the macOS version number. Exploits that require complex, multi-stage chains on M3 (e.g., bypassing PAC, confusing SPTM, racing TXM) can often be reduced to a single Use-After-Free and a ROP chain on Intel. As Apple phases out Intel support, the "easy mode" of macOS exploitation is rapidly vanishing.

The trajectory of macOS security architecture is not asymptotic; it is directional. Apple is not merely patching vulnerabilities in XNU; they are actively architecting its obsolescence as a security boundary. The "Tahoe" architecture provides the silicon primitives (SPTM, TXM, GL2) required to execute a long-term strategy of Architectural Attrition .

The future of macOS exploitation lies in understanding two concurrent trends: the ossification of the XNU kernel into a static, immutable appliance, and the migration of high-value logic into the opaque, hardware-isolated world of Exclaves.

11.3.1 The Deprecation of kmod_load : The Static Kernel

For decades, the ability to load Kernel Extensions (KEXTs) was a defining feature of macOS. It was also its Achilles' heel. KEXTs run at EL1, share the kernel's address space, and historically lacked the rigorous code review applied to the core kernel.

The mechanism for this—the kmod_load syscall (and the associated kmod_control traps)—represents a massive attack surface. It requires the kernel to possess a runtime linker ( kld ), capable of resolving symbols, applying relocations, and modifying executable memory.

The DriverKit End-Game:
Apple has systematically introduced userland replacements for kernel drivers: USBDriverKit , HIDDriverKit , AudioDriverKit , and NetworkingDriverKit .

  • Current State: In Tahoe, third-party KEXTs are deprecated. The userland tool kmutil manages the policy, but the actual loading still relies on the kernel's ability to link code. Loading a legacy KEXT now requires reducing system security (disabling SIP/Secure Boot) and interacting with the TXM via LocalPolicy to explicitly authorize the hash.

Future State: The Death of the Runtime Linker:
We are approaching a point where the kernel will effectively lose the ability to load dynamic code entirely in "Full Security" mode. The goal is to remove the kmod_load logic from the kernel entirely.

  • The "Sealed" Kernel: The Boot Kernel Collection (BKC) (loaded by iBoot) and the Auxiliary Kernel Collection (AKC) (loaded early by kernelmanagerd ) will be the only permitted executable kernel code.
  • Pre-Linked Immutability: By moving all linking to build-time (kernelcache generation) or boot-time (iBoot verification), Apple can strip the dynamic linker logic ( kld ) from the runtime kernel. If the kernel doesn't know how to link a Mach-O, it cannot load a rootkit.
  • SPTM Enforcement: The SPTM already enforces that XNU_TEXT is immutable. The logical next step is for the SPTM to reject any sptm_retype request that attempts to create new XNU_TEXT pages after the initial boot sealing phase is complete.

RE Implication:
The era of the "Rootkit" is ending. If you cannot introduce new code into EL1 via kmod_load , and you cannot modify existing code due to KTRR/SPTM, persistence in the kernel becomes impossible. Attackers will be forced to live entirely within data-only attacks (DOP) or move their persistence to userland (which is easier to detect) or firmware (which is harder to achieve).

11.3.2 Exclave Expansion: Eating the Monolith

If XNU is the "Insecure World," Exclaves are the "Secure World." Currently, Exclaves are used for high-sensitivity, low-complexity tasks (Privacy Indicators, Passkeys). However, the architecture is designed to scale. Apple is effectively strangling the monolithic kernel by slowly migrating critical subsystems out of EL1 and into Exclaves.

Candidates for Migration:

  1. The Network Stack ( skywalk ):
    Apple has already introduced skywalk , a userland networking subsystem. The logical evolution is to move the TCP/IP stack and packet filtering logic into an Exclave.
    • Benefit: A remote code execution vulnerability in the Wi-Fi firmware or the TCP stack would compromise an isolated Exclave, not the entire kernel. The SPTM would prevent the compromised network stack from touching system memory.
  2. Filesystem Encryption (APFS):
    Currently, AppleSEPKeyStore handles key wrapping, but the bulk encryption happens via the AES Engine managed by the kernel. Moving the filesystem driver's cryptographic logic to an Exclave would ensure that even a kernel compromise cannot exfiltrate file keys, as the keys would exist only within the Exclave's memory domain.
  3. Audio and Media Processing:
    To protect DRM content and prevent microphone eavesdropping, the entire CoreAudio engine could be moved to a "Media Conclave."

The "Dark Matter" OS:
As more logic moves to Exclaves, a significant portion of the OS execution flow becomes invisible to standard introspection tools.

  • No DTrace: You cannot DTrace an Exclave.
  • No kdebug: Kernel tracing will show a "black hole" where the request enters XNUProxy and vanishes until the result returns.
  • Opaque State: The memory of an Exclave is physically unmappable by the kernel. A kernel memory dump (coredump) will contain gaps where the Exclave memory resides.

11.3.3 The "Hollow Kernel" Hypothesis

Extrapolating these trends leads to the Hollow Kernel Hypothesis .

In this future architecture, XNU (EL1) is demoted to a Compatibility Shim . Its primary role is to:

  1. Provide POSIX system call semantics for legacy userland applications.
  2. Manage coarse-grained scheduling of CPU resources.
  3. Act as a message bus (via XNUProxy ) between userland applications and the real system logic running in Exclaves.

The Security Inversion:
In the traditional model, the Kernel protects the User. In the Hollow Kernel model, the Hardware (SPTM/TXM) protects the System from the Kernel.

  • The kernel is treated as untrusted code.
  • The TCB (Trusted Computing Base) shrinks from "The entire Kernel" to "The SPTM, TXM, and specific Exclaves."
  • A kernel compromise becomes a "Local DoS" or "Privacy Violation" rather than a "System Compromise."

11.3.4 The Visibility Gap: The End of Passive Analysis

For the reverse engineer, this shift is catastrophic for visibility.

  • Tightbeam IDL: The interface between XNU and Exclaves is defined by Tightbeam. Unlike MIG, which was relatively static, Tightbeam protocols can evolve rapidly. Reverse engineering the system will require constantly reconstructing these serialization formats.
  • The "Intel Gap" Closure: As Apple phases out Intel support completely, they will likely remove the legacy code paths in XNU that supported the "un-isolated" model. This will make the kernel source code (if still released) increasingly divergent from the binary reality running on M-series chips.
  • Hardware-Locked Debugging: Debugging an Exclave likely requires "Red" (Development) fused silicon. Researchers working on retail "Green" (Production) hardware will be effectively locked out of analyzing the internal logic of these secure subsystems, forced to treat them as black boxes and fuzz their inputs via XNUProxy .

Final Thought:
macOS is no longer just a Unix system. It is a distributed system running on a single die, governed by a hypervisor that doesn't exist in software. The kernel is dead; long live the Monitor.

Show HN: RealDeed – Tokenize Real Estate into Digital Assets

Hacker News
www.realdeed.co
2025-11-22 20:46:43
Comments...
Original Article

T o k e n i z e R e a l E s t a t e i n t o D i g i t a l A s s e t s

Global-Ready Tokenization for Real Estate in the UAE & MENA, U.K, India & beyond.

Simple 3 step process

Properties Visual Studio

Note : This is for virtual representation purposes only it is not intended for sale, solicitation, or to attract any form of investment.

Properties Secured on Blockchain

Properties Secured on Blockchain

image for your logo

Rapid & Secure Tokenization

Rapid & Secure Tokenization

innovative image

Web 3.0
DLT Innovative Solution

Web 3.0
DLT Innovative Solution

Projects are approved for tokenization

blured persona image

sharpen persina image

image with details

Real Time Title Deed Verification

In our clients’ words, for pilot real estate tokenization.

  • Real Deed Tokenisation Pilot Service is Amazing!

  • I got my villa tokenized in just 1 day.

  • Real Deed’s tokenisation engine helped us tokenize our towers.

    Dino Daniel

    Property Advisor

  • Worked with RealDeed to tokenize my villa remotely.

    Jason Eining

    Townhouse Owner

RealDeed is your trusted partner in transforming your property into a digital asset.

The Go-Between

Hacker News
theamericanscholar.org
2025-11-22 20:24:17
Comments...
Original Article

Late in August 1939, just before Germany invaded Poland, the Chicago Daily News broke an ironclad policy to make the American reporter Helen Paull Kirkpatrick its first and only woman correspondent abroad. She was nearly 30 and had been living in London since 1937 after two years in Geneva, circulating with patrician ease among the native and expat upper classes. With two English colleagues, she had cofounded The Whitehall Letter, a successful weekly digest of world affairs with a strong antifascist bent, and on her own wrote two books: one about Prime Minister Neville Chamberlain and the prewar Munich Agreement, the other about the British after the war began. She also started an American edition of the newsletter run by her younger brother, Lyman Bickford Kirkpatrick Jr., whose eventual experiences in the American intelligence apparatus would figure heavily in her own working life.

Helen Kirkpatrick’s start date in the Daily News London bureau could not have been better timed. The Duke and Duchess of Windsor had just returned to England after a period of semi-exile in France. By way of Helen’s frequent weekends with Waldorf and Nancy Astor at Cliveden and with Ronald and Nancy Tree at Ditchley Park, she happened to know something most others did not: that the Windsors were staying at South Hartfield House, the grand home of Edward Dudley Metcalfe, the duke’s former equerry, whom she also happened to know. Bill Stoneman, the Daily News London bureau chief, thought her foolish to trek the 40 miles south to Sussex in what was sure to be a failed attempt to interview the Duke of Windsor—the man who had been the nation’s king until abdicating the throne in 1936. But off she went. She arrived at twilight, buoyed with anticipation until it emerged that the duke had already declared that he would not give interviews during his stay. And yet, struck by the dismay on the face of this visitor, the duke devised a gallant way to both keep his word and salvage Helen’s hopes for a triumphant debut in the Daily News —and the many other papers that subscribed to its well-regarded foreign news service. “He seemed to have decided that even though I was not to be allowed to interview him, he would interview me,” she wrote. Her story appeared on page two, September 18, 1939, under the headline, “Duchess of Windsor to Run Hospital, Duke May Join Army.”

This flashy little royal scoop became the first of scores of exclusives during Helen’s seven years with the paper, most of them far more substantive in news value. For her first anniversary on staff, the editors featured her in a five-column promotional house ad titled “War and a Woman,” which called her articles “clear as crystal, accurate as a radio beam, prompt as the crashing impact of the happenings they record”; her dispatches arrived in Chicago by cablegram at “machine-gun tempo”—three, four, sometimes five times a day. Her beat was all of England and Ireland and General Charles de Gaulle and his London-based Free French Movement. Later, there would be lengthy forays to Algeria, Italy, France, Germany, and elsewhere in Europe—wherever war news flowed.

Least known to the public among her journalistic virtues, the ad disclosed, was how often her stories had to be “splashed across eight-columns, front page headlines, without a hint of credit to the razor-edged intelligence that rifled them home.” For example, eight days before the German blitzkrieg of May 10, 1940, Helen was alone among correspondents to report that King Leopold III of Belgium had privately informed the U.S. government that the Germans were sure to invade his country next. Clearly to avoid linking the news to Helen or revealing how or where she had obtained the information, the story ran buried in a column of items under someone else’s byline, attributed only to “private sources in Chicago.” The point was, the impeccable confidential sources cultivated by “Our Helen,” as Daily News headlines would sometimes dub her in the years to come, were either newsmakers themselves or those just as likely to know of what they spoke. “Not even now,” the ad went on, “—not until war and war’s tongue-stilling offspring, strict censorship, have lifted—can the complete story of Helen Kirkpatrick’s incredible war coverage be told.”

The siblings’ connectedness casts a hazy light on how the realms of intelligence-gathering and major media reportage have sometimes intersected in times of war and tyranny.

It is fair to say that as a class, American women correspondents during World War II were not held in particularly high regard, so Helen Kirkpatrick’s outsize ability to garner not only respect but also major governmental and military awards does conjure up questions. Of the more than 1,600 U.S.-accredited World War II correspondents, only 19 received the coveted U.S. Medal of Freedom—and of those 19, Helen was the only woman. This despite similar barrier-breaking reportage by Margaret Bourke-White, Ann Stringer, Lee Carson, Lee Miller, Iris Carpenter, Marguerite Higgins, and others, none of whose names even appear in archived military lists of suggested nominees. How did she manage to have so much swift, direct access to so many top-line political, diplomatic, military, and intelligence sources? Was it just her keen reporting, or did the close social relationships she developed with important men set her apart?

Lyman Kirkpatrick’s time in Europe during the war either ran parallel to hers or was intertwined. And yet in her lengthiest latter-day interviews, references to her brother are few and casual. In his books, Lyman’s mentions of his sister were similarly spare. Both of their names appear in the wartime memoirs of friends and colleagues, but never in the same account. Only a careful rereading of the Washington Press Club Foundation’s oral history of Helen revealed an offhand mention of a brother in the CIA. Did Lyman’s rapid rise through the U.S. intelligence ranks figure in her successes, and if so, how? Had Helen broken with journalism’s established codes of conduct and crossed the line into espionage? In a more general way, the siblings’ connectedness during the war years casts a hazy light on how the realms of intelligence-gathering and major media reportage, meant to be strictly separate, have sometimes intersected in times of war and tyranny.

In interviews, Helen’s answers to questions about how she achieved such high recognition were veiled, self-deprecating. Sometimes she’d deflect. Did she do anything heroic? No, she’d reply, adding that she could not remember why she was honored by the French and with a U.S. Medal of Freedom or if she ever even knew the reasons. Check her papers at Smith College, she would suggest. They offer little. The questions lingered. For even her nearest living relations, they linger still.


The Kirkpatricks of Rochester, New York, were a prominent family, albeit no
longer monied by the time Helen and Lyman’s parents married. (They later divorced, then married each other again.) On their father’s side, the siblings were mindful and proud of a Scottish lineage that pre-dated Robert the Bruce; I Mak Siccar (“I’ll make sure”) was the family motto. Their maternal heritage came via the Paulls of Wheeling, West Virginia; they often spent holidays at the family home in Hawthorne Court in Woodsdale, on Wheeling’s outskirts. Their ancestors included Virginia unionists like Colonel James Paull, a soldier in the Revolutionary War, and Colonel Joshua Fry, who, alongside the father of Thomas Jefferson, created the original map of Virginia. A long list of judges, lawyers, military men, and other public servants followed.

With tuition help from grandparents and scholarships, and after a couple of time-outs for Helen for financial hiccups, both siblings completed their education with prestigious degrees. Helen attended The Masters School in Dobbs Ferry, New York, and then studied history at Smith, graduating Phi Beta Kappa in 1931; Lyman went to Deerfield and Princeton, where he majored in politics and graduated in 1938. Helen looked after Lyman like an ambitious other mother. In letters home, she lovingly mocked her parents’ adoration of him, which she shared, referring to him as “Little Man” or “Son.”

To intimates, Lyman went by Kirk; Helen, they called Pat. Seven years apart in age, they were as tall as they were driven, smart, and talented. Lyman was a handsome six-foot-five; Helen, a stately five-foot-10. Both were appealing in style and manner, always commanding notice. Her friend and colleague Ben Robertson found her “extraordinary” and “beautiful” with “a first-class mind” that she put to use, he later wrote, in “taking up matters with people at the top, with cabinet ministers and the like.” Her only known detractor in print was General Raymond E. Lee, a U.S. military attaché in London whom she met in December 1940. He found her “rather clever,” his journals record, but “far from being as attractive and alluring as she thinks she is.”

It was during her student days that Helen first spent time in Geneva, which fed her fascination with the comings and goings of the crowd that convened around the still-promising League of Nations. “There were all kinds of operators around,” she once said, “people whose jobs were a little obscure as to what they were. They were oil merchants or they were spies for one side or another. It was a very exciting place to be.” In the summer of 1935, she returned to the city as a tour leader for a group of high-school girls, but really to flee an unhappy marriage. “Not returning,” read the two-word transatlantic cable she sent to Victor Polachek Jr., the husband of two years she’d left behind in New York City. She found work writing policy papers for the Geneva Research Center, an affiliate of the Foreign Policy Association, and as a newspaper stringer who covered the League of Nations for the New York Herald-Tribune and a few other American and British papers. That she spoke French was an enormous advantage, she said, but not monetarily. She lived on Brussels sprouts and cottage cheese.

By 1937, Helen was 28. Most journalists of her era and caliber would have advanced to staff positions with journeymen status by that age, but she had not yet hit her stride. Her working life had zigzagged a few times before she settled in Europe: There was the year she spent with her mother in a management training program at Macy’s; her nondegree graduate study on a fellowship in Geneva; the marriage to Polachek, during which they lived in a charming but oh-so-narrow three-story townhouse in Greenwich Village, once rented for a couple of years by Edna St. Vincent Millay and her husband. Through the Polacheks’ next-door neighbors, Osgood and Alice Field, Helen had taken a job helping to organize an exhibition on Soviet education at the Museum of Natural History. This was under the auspices of the Soviet All-Union Society for Cultural Relations with Foreign Countries, also called VOKS. However, the Communist Party affiliation of many of its members left her dubious and made her steer clear of any further involvement with the organization.

In Geneva, her stringer status meant that her copy appeared unsigned. Newspaper archives yield her name only once from those days: She is quoted in other reporters’ stories about the July 4, 1936, suicide of Štefan Lux, a Czech journalist who shot himself in the assembly room of the League of Nations to protest the League’s inaction on Germany’s treatment of Jews. Of the League’s reporters, only Helen—“an American girl attached to the Geneva Research Center,” The New York Times called her—was within sight of Lux and within earshot of his last words: “ C’est le dernier coup. ” To have no outlet that would publish her firsthand report, she later said, was especially frustrating.

Victor Gordon-Lennox, a well-born diplomatic correspondent for The Daily Telegraph, often reported from Geneva. It was he who persuaded Helen to join him and Graham Hutton of The Economist in the Whitehall Letter venture in London. For Helen, this was a fresh and appealing opportunity. In part, she owed her swift entrée into London’s sought-after social circles to “V.G.L.,” as letters home so often refer to him in reports of her weekends and glittering evenings out.

Meanwhile, the timing of Helen’s move to London coincided with intensified familial concern over young Lyman’s professional prospects. Helen was in the best position to help. The summer after his graduation from Princeton, he went biking in England with a college mate. This was when the siblings’ adult lives began to converge. Helen introduced Lyman to Kermit “Kim” Roosevelt, son of Theodore by his second marriage, who was seeking a summer tutor to prepare his son, Dirck, for admission to Groton. Helen would later say Lyman’s fine manners got him the position over several Rhodes scholars. Living with the Roosevelts meant that Lyman needed the right clothing—a tux, for example, which would have taken too long to arrive from home. Helen arranged to borrow one from someone at the U.S. Embassy. “The name escapes me,” Lyman wrote years later, “but my foggy recollection is that it might have been a tuxedo of the ambassador himself.” And yet, Ambassador Joseph P. Kennedy stood five inches shorter than Lyman, so perhaps not. Helen also sent her brother new white flannels and tennis shirts. “He’s coming to town one day soon [to] be fitted for a suit and have some tweed he got in Scotland made into another jacket,” she wrote home. Once Lyman was back in the United States that September, a Princeton connection recommended him to David Lawrence, who hired him for his company, United States News, the eventual publisher of U.S. News & World Report . Lyman edited the U.S. edition of The Whitehall Letter on the side, with typing and administrative help from his wife, the former Jeanne Barclay Courtney, whom he married in February 1939.


In London, Helen knew both William “Wild Bill” Donovan and David K. E. Bruce well. Donovan headed up the Office of the Coordinator of Information, known as the COI, which evolved into the Office of Strategic Services, the OSS. He once tried to recruit her to the agency, but as she later told an interviewer, her response was, “To do what?” Donovan said he didn’t know, but he would find a spot for her. She declined, telling him she thought she was as useful to the war effort doing what she was doing as she would be in the OSS. Bruce, whom she met when he first arrived in England as chief representative of the American Red Cross, served both the COI and OSS as London branch chief and deputy executive director. Published entries from his diaries mention Helen innocuously only a couple of times, but his biographer described her as Bruce’s old friend, who often joined him at the home of Nancy and Ronald Tree.

In August 1940, Bruce, then still with the Red Cross, would go with Helen and other correspondents to Shakespeare Cliff, facing the Dover Strait, to watch the RAF fighters confront the invading Luftwaffe overhead. The memoirists among them later name-dropped their colleagues in passages about these deeply affecting days. “We lived in expectation of a full German attack any night,” Vincent “Jimmy” Sheean wrote, “and the nerves of some of our friends grew so exacerbated by suspense that they actually said they would welcome it.” Not Helen; she was steely, and was in fact first to suggest the Dover vigils. Ben Robertson said that by then she had already become one of the best American journalists, woman or man, “and in the weeks that were to follow she was to add to her already established reputation.”

Another crucial contact of Helen’s was David Gray, the “openly pro-British” U.S. minister plenipotentiary to Ireland. She stayed with the Grays whenever she was in Dublin, which was often. It was Gray who told her, as she reported on June 12, 1940, how much evidence there was of German infiltration in Ireland. She mentioned the tide of fifth columnists who secretly wanted a British defeat, although she does not attribute her source. To avoid Irish censorship, she filed the story with a Dublin dateline once she got back to England. Ireland’s minister in Washington, Robert Brennan, wrote to the editor, conveying his country’s ire and calling out what he said were misstatements in the story. The Daily News published his missive promptly, adding this note: “Miss Kirkpatrick’s article was based on careful investigation in England and in Eire, and, The Daily News has no reason to doubt the accuracy of her reports. It is obvious that a diplomat in Washington cannot be better informed on London opinion than a resident correspondent in London.”

For Helen, breaking stories for the Daily News enlarged her already formidable social standing and long list of contacts and sources who cultivated her as eagerly as she cultivated them. What she gleaned during a Christmastime weekend at Ditchley Park in 1940 about secret negotiations between Britain and Vichy France almost got her thrown out of England. Another guest told her that Vichy had said it would not aid the Nazis in attacking Britain; Helen reported it, enraging Winston Churchill. “Miss Helen Kirkpatrick should be shipped out of the country at the earliest moment,” the prime minister’s minutes record. “It is very undesirable to have a person of this kind scouting about private houses for copy regardless of British interests.” Duff Cooper, the British minister of information, whom Helen considered “a great close personal friend,” vouched for her; the prime minister reconsidered.

In Dublin late in the summer of 1941, David Gray tapped Helen to take a secret message to London and hand it to Britain’s cabinet minister for the colonies. It concerned a scheme devised by Gray and Sir John Maffey, Britain’s chief diplomatic representative to Ireland, to force the Irish into the war or at least to give up some rights. Helen obliged, despite the line-crossing this involved. By Gray’s mistake, however, a copy of the memo ended up in an envelope sent to Joe Walshe, the secretary of the Irish Free State’s Department of External Affairs—Gray had meant to send him only a copy of one of Helen’s articles from Dublin. “Well, the fat was in the fire,” she recalled. Even in wartime, for a correspondent to be seen as engaging in the trading of unpublished information or in espionage of any sort was and is a clear violation of every established journalistic code of ethics. Especially in conflict situations, any insinuation of such a sideline, even a comment tossed off in jest, could be as destructive to a reporter’s reputation inside the press corps as it was potentially lethal outside it.

Gray, knowing his telephones were bugged, traveled to Belfast to call Helen about what had happened and then wrote to her to apologize for putting her in such a terrible position. He offered to resign or do whatever she suggested because, he wrote, “I never can square myself for this stupidity.” She told him she was the expendable one, not he. In the end, nothing came of the episode beyond the doubt it sowed in the minds of Dubliners that Helen “was at least an intelligence officer or a spy. And,” she recalled, “I was treated that way.” The British, she said, were especially dubious of Americans who traveled often between England and Ireland, and the IRA went so far as to accuse her of being the harbinger of an American invasion of their country. “But I continued to go regularly,” she said, “and I became quite good friends with the man in the Foreign Office.”

It remained Helen’s style, at every opportunity, to share with officials among the Allies whatever she had learned that they might be able to make good use of, sometimes preparing lengthy unrequested written memos about hard-won information, reports that just as often went nowhere. The Americans were invariably unreceptive—“We have our own sources” was the official attitude, Helen would later say—but the British and French were keen to learn whatever they could. Years later, she would reflect on the “strong feeling” she and Lyman shared about “the importance of intelligence, of knowing what’s going on, and finding out from people, asking what they’ve seen, what they’ve heard. Then you evaluate it as to whether it’s of real interest or not.”

Especially in conflict situations, even a comment tossed off in jest could be as destructive to a reporter’s reputation inside the press corps as it was potentially lethal outside it.

Given these predilections, it is not surprising that Helen would see intelligence as a career route for Lyman, if not for herself. Once again, she flung a handful of sisterly pixie dust onto her brother’s job prospects. “I’ve cast a few flies in [the COI’s Bill] Donovan’s direction with no results so far,” she wrote to her brother in Washington early in January 1942, just weeks after Pearl Harbor was attacked and after the Naval Reserve Air Force rejected his enlistment because he was red-green colorblind. “Everyone in the service branches of the [U.S.] embassy has suddenly blossomed into uniform, etc., and our English friends have ceased to be polite. … Most of us here wanted to get home—or into some job other than our present ones.” However, she went on, the U.S. Ambassador, John G. Winant, “asked us all to sit still and keep on with what we’re doing for the time being.” Helen felt out David Gray to see whether he could make use of her in Dublin, but he demurred, affirming her own growing sense that she was better off where she was. From Colonel Frank Knox, her publisher, came cajoling affirmation. “I can think of no war work in which a courageous and intelligent woman could be engaged that would be more valuable to the country in the present circumstances than that you are doing for us,” he wrote. “Us” in this case seems to have meant more than the foreign news service of the Chicago Daily News: In 1941, Knox had also become the secretary of the U.S. Navy.

Two months later, Lyman, still at his publishing job in Washington, got a call from Helen’s friend David Bruce, whom Lyman had met back in 1940. Bruce offered Lyman a post with the COI. Lyman later wrote of some “flattering and undeserved attention” that had come his way, at about the same time, from Colonel Knox himself, the Navy secretary–cum– Daily News publisher. This happened via Bill Stoneman, Helen’s London bureau chief, who mentioned to Knox the Navy’s rejection of Lyman. Knox then called Lyman into his office and offered to rescind the decision, but by that point, Lyman had accepted the job with the COI. Knox cautioned Lyman against “that crazy outfit” but did not convince him otherwise. Knox further offered to help if Lyman changed his mind.

That, in essence, is how their world worked.

Lyman was soon deployed to London and later to the continent as the COI became the OSS. He was considered a master at postmortem assessments, a specialty that years later would include a seminal report on the failed Bay of Pigs invasion of 1961. Among his wartime assignments was the task of traveling with the 12th Army as General Omar Bradley’s briefer. He also worked in close liaison with the French and Belgian resistance, serving as a conduit for intelligence reports on the resistance and from OSS agents to the 12th Army Group.

His duties included giving the enemy order of battle at morning briefings each day. According to Helen, General Bradley marveled at Lyman’s ability to reel off—from memory, without notes—the entire litany of intelligence gathered about German operations across and up and down the continent from one day to the next. One of Lyman’s favorite memories, meanwhile, involved General George S. Patton, who was often at Army group headquarters. Shortly before Christmas 1944, Patton asked for a briefing on when the Russian forces would be able to resume their offenses in the East, thus taking some pressure off the western front. The Russians shared little, despite repeated requests from SHAEF, the Supreme Headquarters Allied Expeditionary Force. As Lyman explained in his memoir, most information came from listening to German and Russian communiqués about the battles in the East “and then plotting the map based upon what seemed most plausible from what they were both saying and then balancing this with what was known from our own intelligence sources.” The intelligence team prepared an answer, which they provided close to New Year’s Day, when the generals had time to hear it. As the briefing concluded, Patton asked one question: “When do you think the Russians are going to launch their next offensive?” Lyman gave the reply. The second week in January, he said, based on the case study the team had prepared of the time lags between Soviet offensives, coupled with knowledge that in Poland, the Russians were along the bank of the Vistula and that they would want the river frozen to a sufficient depth to allow tanks to cross.

The Russians indeed launched on January 11. “The next time General Patton came to a briefing,” Lyman recalled, “he marched into the room, pointed directly at me, and said, ‘Brad, you’d better watch that fellow—he must be a Communist. He knew when the Russians were going to attack!’ ” Sometime later, Lyman learned of a pair of high compliments: that Patton asked Bradley to transfer Lyman to Patton’s staff and that Bradley declined.

For the D-Day invasion, 1944, Lyman and most of the OSS did not get to Normandy until “D-Day plus 17,” or June 23. That day, the Daily News carried a story from Helen datelined “Normandy Beachhead, June 21.” It appeared on the front page with an editor’s note above it, saying that she was among the first 10 women correspondents permitted to make an in-and-out visit to Normandy. Throughout the war, women reporters were forbidden to go closer to any front than the nurses were allowed to venture, although unauthorized breaches did occur. (Martha Gellhorn, for example, stowed away on a hospital ship to report the beginning of the D-Day invasion and lost her military privileges as a result.) In this case, the women arrived on an unarmed Douglas transport to pick up the American wounded. Throughout most of the days leading up to and after the invasion, Helen’s datelines were either London or some version of “Supreme Allied Expeditionary Force Headquarters in England.” Gossips passed word, and the Daily News reported that Helen “plays bridge with Eisenhower and Churchill calls her ‘The Kirk.’ ”

Not until July 11 was she more permanently in France. Above her first Cherbourg dateline, July 19, her editors trumpeted in italics that she was the “first war correspondent”— woman or otherwise—to be assigned to the headquarters of General Marie-Pierre Koenig, commander of the French forces of the interior. All that coverage of de Gaulle and the Free French had paid off. That day, she wrote home to say she “went up” to see Pete Quesada, the commanding officer of the Tactical Air Command, and that they called Lyman to join them for dinner. Lyman had grown a mustache that she mistook for dirt on his face but otherwise looked tanned and well. “We had fun,” she wrote, “just the three of us, and the mess wasn’t hard to take …”

Helen told in interviews of how, when on the continent, she would often attach herself to OSS teams “because even in Normandy and Brittany, they would go into Gestapo headquarters and pick up stuff.” In 1945, when the military authorized a group of reporters to cross the Rhine on a glider, Helen was on the list to go. That is, until Lyman, “of all the interfering characters, somehow got wind of it and saw that my name was taken off. Now you could say that was brotherly-sisterly or male-female, but I must say later I was awfully glad because they all got shot down.”

On the night of August 24, Helen rode behind General Jacques-Philippe Leclerc’s tanks as they entered Paris to liberate the city. The following day, she left the luncheon table of Ernest Hemingway, holding court at the newly liberated Ritz, to cover the parade, which culminated at Notre Dame with a memorial service reserved for the families of those who had died in the Resistance. There, inside the cathedral, she was alone among reporters who witnessed the failed assassination attempt on Generals de Gaulle, Koenig, and Leclerc. Years later, for an oral historian, Helen recalled how she had positioned herself standing up and hanging off a grille fence for a better view, when somehow the teeming crowd shoved her into the cathedral, just as the lights went out and the organ music stopped. “Suddenly there was some shooting and a man near me was hit,” she said. “They were shooting from the clerestory balcony. A Dominican monk appeared at the altar. Well, I am not a Catholic, but I knew that monks are not cathedral priests, and he led them in the Magnificat . Then they turned around and marched out.” The day after that, she reopened the Paris office of the Daily News as its new chef de bureau .


Lyman, for his war service, received the U.S. Legion of Merit, the Bronze Star medal, the European Theater Ribbon and five battle stars, and both the French and Belgian Croix de Guerre. For Helen, her U.S. Medal of Freedom covered the period from June 6, 1944, to May 8, 1945: D-Day to V-E Day. She also earned a European Theater Ribbon. From France, she won the Médaille de la Reconnaissance in 1945 and the country’s highest award, the Légion d’Honneur, in 1947—along with three men and Janet Flanner, the Paris-based New Yorker writer known in print as Genêt.

Why, of all the women who distinguished themselves as World War II correspondents, was Helen the only one singled out for the U.S. military’s highest recognition? It could be that her consistent access to solid, privileged information and her “I-was-there” reporting are the only explanations. All of the Medal of Freedom honorees were cited “for exceptionally meritorious achievement, which aided the United States in the prosecution of the war against the enemy in continental Europe.” Helen’s citation went on to describe her courage, how she “never hesitated to face danger in the pursuance of her profession,” and how her “objective interpretation of military operations and particularly of the renaissance of occupied France not only contributed to understanding of the problem in the mind of the American public but also went far to promote good Allied relations, thereby meriting the praise and recognition of the United States.” How so exactly? Is there more to what Helen did than her citation describes?

Once asked by an oral historian about the awarding of the French Légion d’Honneur, Helen remembered the luncheon at the Quai d’Orsay, hosted by Suzanne Borel Bidault, the wife of Foreign Minister Georges Bidault, a figure in the Resistance, and the first French woman to become a diplomat. But Helen said she did not know why she had received it. “I suppose because I had been reporting on the Free French and de Gaulle was president at that point,” she said. “I was known as a strong advocate of France during the war and was in France then, had been there as a correspondent; they gave a number of them to correspondents,” she said. “It was sort of handed out, you know. An awful lot of French have them.” She said she suspected that her friend the baron Louis de Cabrol might have initiated the Médaille de la Reconnaissance because she “plucked him out of a British hospital” and got him into an American one, where a surgeon “saved his knees so that he was able to walk and ride horseback thereafter. I don’t know. I never knew, and, as a matter of fact, I don’t recall it being presented. Maybe it was.”

Lyman, after the war, returned briefly to his postcollege job in magazine publishing but then joined the CIA at its formation in 1947. In time, he would become the agency’s inspector general and after that, executive director, its number-three position, despite the polio that left him paralyzed from the waist down in 1952. When he left the agency at age 48 in 1964, he received the President’s Award for Distinguished Federal Civilian Service and the CIA’s Distinguished Intelligence Medal, an honor that had been conferred at that point no more than a dozen times. The letters of nomination, although considerably more restrained in style than the praise lavished on his sister in the Chicago Daily News, are comparable as indications of the respect and admiration he commanded. “Of all the recipients of this medal known to me,” one nominator wrote, “none has given to the CIA and the intelligence community the dedicated, selfless devotion to duty over such a long period of time as Kirk.”

The Daily News foreign service started to decline in the period after Colonel Knox’s death on April 28, 1944, so Helen and others from their Europe-based team went to the New York Post, starting late in 1946. During the transition, Helen traveled to Moscow with her good friend and the new U.S. ambassador to the Soviet Union, Walter Bedell Smith, and his wife. A long memo followed but nothing publishable, since any reporting could be tied too closely to the new ambassador. Two years later, in 1950, Smith became Lyman’s boss as director of the CIA, succeeded by Allen Dulles.

For Helen, the New York Post of 1947 could not have been a worse fit; the lighter assignments favored by editors annoyed her, as did the way they handled and displayed what she wrote. And yet, despite her stunning résumé, repeated efforts to land another newspaper job failed. No one seemed to have a place for Helen Kirkpatrick. She went to work for the U.S. State Department, first at Voice of America, then as communications director in France for the Economic Cooperation Administration, which administered the Marshall Plan. In Paris again, she worked first for her old friend David Bruce, and then for Barry Bingham when Bruce became the U.S. ambassador to France.

Back in Washington, as a since-declassified document of April 17, 1951, records, then–CIA director Allen Dulles had lunch with “Miss Helen Kirkpatrick, State Department DD/P and Mr. Lyman Kirkpatrick.” Days later, she had a new post in Washington as public affairs adviser to the U.S. assistant secretary of state for European Affairs, a job that involved frequent travel abroad.

Helen left the State Department in 1953—her appointments were always temporary, although she tried several times for civil service status—to become assistant to the president of Smith College and soon the second wife of Robbins Milbank, a widower and Smith College trustee. She continued to lecture and appear on panels about the war, staying in touch with her legions of contacts, just as she had been doing since her first of many U.S. speaking tours in 1937.

Lyman, after leaving Washington, accepted an academic appointment at Brown and wrote two books about the CIA, examining his own experiences and the agency’s structure, strengths, and weaknesses. He died at 78 in 1995, two years before Helen, who lived to 88.

A genealogical search has led me to several living Kirkpatrick descendants. Among them, John Pitner, Helen’s great-nephew and one of Lyman’s grandsons, has the greatest sense of the family’s history and lore. He remembers well his grandfather’s study with its many photographs of Lyman posed with the famous, especially the one with General Patton. Although Lyman never shared his career experiences with Pitner directly, the grandson sensed his grandfather’s august personal history all the same. Lyman’s demeanor, Pitner said, was “stern,” almost the opposite of “Aunt Pat.” He recalled a trip to Jackson Hole she organized with him and another grandnephew, both preteens, when she was about 77. Both boys were in the back seat as she careened into the passing lane of the main two-lane road into town and whizzed past an interminable line of stalled traffic as the oncoming cars came perilously close. “You’re not going to make it,” the fly-fishing guide in the front passenger seat said. “Yes, I am,” she replied.

From his grandfather’s books, Pitner knew well the paramount importance Lyman placed on “keeping secrets secret. I’m certain [Helen] would have been of the same mind,” he said. “Also, her character was such that she didn’t brag or otherwise bring up her numerous exploits and connections unless specifically asked.” His mother, one of Lyman’s four children, had the same sense. Was his Aunt Pat a spy? She never spoke of the matter, and neither Pitner nor his mother has any knowledge about the question one way or the other. However, he said, neither of them would be surprised to learn that she was.

“I do see the potential for an interesting Hollywood story,” Pitner mused. “Given the lack of factual proof (thus far), it might include ‘based on a true story’ in the opening credits. At any rate, maybe you can send me an extra ticket to the premiere!”

The Mozilla Cycle, Part III: Mozilla Dies in Ignominy

Hacker News
taggart-tech.com
2025-11-22 20:21:56
Comments...
Original Article

I owe Mozilla a thank-you. Really, I do. Maybe an Edible Arrangement? People like those. Some lil pineapples cut into stars on sticks and chocolate strawberries might brighten their day. For the note, I'm thinking something like:

Thank you for proving me exactly right.

XOXO MT

Eight months ago, in the fallout of Mozilla's fumbling of a Privacy Policy update, I wrote :

Mozilla is pursuing its primary objective, which is the survival of Mozilla. Its mission statement is more than broad enough to accommodate that, and Firefox is not a real priority. The community should accept that and stop waiting for Mozilla to be the hero they deserve.

Regrettably, I was unable to take my own advice on the last part. So here we are yet again, marveling at Mozilla's dedication toward eroding decades of good will in the community they purportedly serve. To quote one of my sacred texts , it's a focus and intensity normally seen only in successes.

Back in the present, we have Mozilla doubling tripling n thing down on this direction. First, with their announcement of "AI Window," a new feature (used very loosely) coming to Firefox which seems to emulate the user experience offered by AI browsers like Perplexity's Comet or OpenAI's Atlas. In other words, instead of performing search from the address bar and interacting with websites like browsers have done since they were invented, your first interaction will be with a language model prompt, which then mediates your experience of the web.

Not to gloat, but I told you so .

The response from the Firefox community has not just been overwhelmingly negative, it is universally negative as far as I can tell. At least among users willing to post on Mozilla's forums about the issue, which is absolutely a biased sample set. I have received some comments separately in support of Firefox, but they are countable and the vast, vast minority. Mozilla's core audience hates this move. At the very least, they would want all the AI components of Firefox to be opt-in, a choice that Firefox has been unwilling to make so far, instead enabling these new features by default.

What does Mozilla do? Temper the plan? Ease up on the forced features?

Nah, they do what any good corporate PR person would tell you to do when facing public backlash: post through it .

This post is a summary of Mozilla's new Strategic Plan, which is viewable in full here . I read it through a few times, and my brain nearly ripped in half from the cognitive dissonance involved. But I think it's worth examining Mozilla's claims carefully. They are:

  1. AI (by which they mean generative AI) is a transformative technology that will fundamentally alter how we interact with machines and the web.
  2. The current landscape is dangerous and controlled by big tech and "closed source" models.
  3. Mozilla should therefore pivot to develop and support "open source" AI implementations the same way they advocated for open web standards.

The strategy details the "what" and "how" of Mozilla's transformation in this direction. We're going to touch on some of those points, but let's begin with these big claims, affording Mozilla maximum benefit of the doubt.

Is generative AI a transformative technology? The Corpos sure seem to want it to be, although its actual usage seems mostly to be chatbot-related. All other attempts to use this trick in other realms have failed rather miserably. Microsoft, for example, wants you to talk to your computer instead of using a mouse and keyboard like a dinosaur . The results, unfortunately, are much worse than the Jurassic version of computer interaction. The pattern holds true across the board. Google's AI Overview continues to be an inferior provider of information than solid web search results. Also, as it turns out, people learn less from LLM output.

Even the AI browsers Mozilla wants to emulate have significant issues, vulnerable to old web vulnerabilities and new attacks against the models themselves.

Generative AI is transforming something, but I don't think it's the web, and I don't think it's for the better.

Which means their second claim is definitely true! The current landscape is dangerous, as I've been decrying for years.

But because Mozilla is convinced that generative AI is a force for good (with no evidence to back that claim up), they conclude that their mission must be to create "open source" alternatives to commercial ("big tech") offerings.

If you truly believe generative AI is a net good but with potential for significant harm, there are arguments to be made for ethical implementations. This same instinct is what propelled researchers from OpenAI to split and found Anthropic .

This, however, is an article of faith . It cannot be argued rationally because no empirical evidence exists to support it. The entirety of the belief is predicated on future potential—and it always will be, right up until the harms are so inescapably clear that even the most ardent of believers suffer because of them. Even then, not all of the Flock will lose faith. And as we now see, Mozilla leadership are not just the Flock, but Disciples.

Mozilla has had a conversion experience, while its core audience has not. This results in a schism of purpose.

Post by @mttaggart@infosec.exchange

View on Mastodon

The Actual Strategy

Digging into the plan itself, Mozilla's ambitions are remarkable. Mozilla has, as the plan notes, always measured itself against a "double bottom line" of mission and market success. However, it seems these two criteria are now separately defined by: "a. AI that advances the Manifesto; and b. diversifying revenue away from search."

Their specific goals include:

  • All Mozilla orgs have a flagship AI product by 2028
  • 10% year-over-year community growth
  • 20% year-over-year growth in non-search revenue
  • 3 Mozilla orgs have more than $25M in revenue (currently: 1)
  • 10% year-over-year investment portfolio returns

I am dumbfounded by these goals. I can't even be snarky about them. They seem so disconnected from reality that I can't imagine how a Board arrived at them.

Let's go through them one at a time.

"Flagship" AI Products

What "orgs" are we talking about here? Historically there have been two Mozilla organizations : the Mozilla Foundation, which is the not-for-profit to which you donate to preserve the open web); and the Mozilla Corporation, which develops Firefox, makes deals with search engines, and creates other revenue-generating projects like, uh, Pocket .

There now exist three other for-profit subsidiaries of the Foundation, although these are not mentioned in the official listings. The first is MZLA Technologies Corporation, which is responsible for Thunderbird . I can't tell what else they do, if anything.

Another is Mozilla.ai , which has a much clearer purpose. This company produces AI products and services. So of these organizations, 3 of them need to have flagship AI products by 2028. What could that possibly look like?

There is also Mozilla Ventures , which is literally just a venture capital firm throwing money at AI projects that align with Mozilla's Manifesto.

Mozilla.ai has the easiest and clearest road, as their Agent Platform is already in early access and is absolutely a commercial product. MZLA has...Thunderbird? So AI-powered Thunderbird? That's what I can figure, although Thunderbird is not exactly known as an AI platform. Their new paid Thunderbird Pro service currently makes no mention of AI integration. That's a headscratcher. Mozilla VenturesAnd finally, we have the Corporation and Firefox.

Let's be as clear as we can possibly be. Mozilla is an AI company , and Firefox will be a flagship AI product according to this strategic plan. This is the focus for Mozilla, which means users of Firefox will only get more and more AI shoved down their throat, and likely fewer ways to avoid it.

I don't have access to Mozilla's "community" numbers, however they choose to define them. But if the reaction to recent changes is any indication, expecting growth of any kind isn't just optimistic, it's delusional. They are betraying the principles of their core use base in favor of their new god. That behavior is usually not rewarded by users or customers.

Financials

While I don't have access to community numbers, the Mozilla Foundation must disclose its financials, so that I can review —and you can too, if you're broken like me. I don't want to think about how much time I've spent reading Mozilla's 990s.

Looking at their revenue change from 2022 to 2023, we see a drop of 3% in royalties (search deals) and almost 15% in subscriptions and advertising. Let's also note that by these counts, royalties account for about 76% of Mozilla's annual revenue. That's by my own calculation on these disclosure, but Mozilla themselves cite 85% as share of revenue from search alone.

2022's numbers show a similar drop (down 3% from 2021) in royalties, but a 25% (!!) jump in subscriptions and ad revenue. My guess is ads, since I don't think the VPN service is raking it in, nor do I think a bunch of people suddenly signed up for Pocket before it died. This would certainly explain why last year, Mozilla went hard on their "privacy-honoring" advertising acquisition.

Zooming back out: the current business model is not delivering growth. So the pivot to AI is a bet that investment is out there for alternative sources of AI technology. It's also, tacitly, a bet that the AI bubble will last long enough to get competing products off the ground and attract investment before it's too late.

I...wouldn't be so sure.

As far as their investment portfolio's performance, Mozilla has changed their strategy significantly in the last 3 years, resulting in significant increases in dividend and realized gains in investments. What's the change in strategy?

At the end of 2022, Mozilla changed our strategy for managing our financial reserves. In prior years we took a purely defensive approach, investing solely in highly liquid fixed-income securities. Our revised approach is focused on delivering a total return to Mozilla after inflation, while maintaining sufficient liquid reserves to weather economic pressures and seize growth opportunities.

Translation: we invested more in stocks and less in bonds, T-notes, and CDs. What specifically they've invested in is unclear, but you can probably guess it rhymes with Blavidia. I'm sure they have a diversified portfolio, but if you believe (as I do) that the market is heading for a massive correction, this goal is unattainable and a dangerous target for a strategic plan.

Lastly, on revenue. Mozilla plainly has to diversify away from search, because search is dying. Google itself is trying to kill it, in favor of AI. If this succeeds, Firefox's primary revenue stream is drying up, and they know it. More than anything else, this is the reason for the pivot. As I've said before, the will to survive takes precedence over principle when choosing a path forward for any organization, even a not-for-profit with a stated mission.

These goals are not so much reasonable expectations as existential mandates. Either Mozilla approaches these targets in 3 years, or they may be staring death in the face.

Flawed Hypotheses

This strategy hinges on three stated hypotheses from Mozilla:

  1. A generational shift in human computer interaction is widening the gap between Mozilla’s products and trustworthy, user-centered experiences.
  2. A vibrant, successful and decentralized open source AI ecosystem is essential if we want independent tech players to thrive — and for innovation to come from everywhere.
  3. The growing need for sovereign, public interest AI which will only be met by governments and public interest tech players pooling resources and banding together.

That's...okay. We'll take it from the top.

Would you say generative AI is a "generational shift in human computer interaction?" The Corpos want it to be, but so far this hasn't taken place. Declaring it thus is the wish becoming the father of the thought. Maybe someday a functional language model will govern our interaction with computing machines, but that is nowhere near the case now, and the fundamental flaws in the technology preclude it from being so in the foreseeable future.

Would you say generative AI is "trustworthy" or "user-centered?" The people who implicitly trust generative AI are suffering from psychosis. It's a pathology. The model creators themselves tell you not to trust them! What are we doing here?

Y'know what is trustworthy? A goddamned URL bar that takes me to the website I want to go to. A search engine that shows me sources, ideally curated for quality.

Okay so hypothesis 1 doesn't pass the sniff test. On to number 2.

Can someone please explain to me what the hell "open source AI" is? Mozilla's helpful Strategy Wiki lists Mozilla's own products and investments under this category. Remember that for LLMs, you have two major "source" components: the dataset on which a model was trained, and the resulting vectors/weights file that comprises the model. Among them is HuggingFace , which is probably best understood as GitHub for AI. HuggingFace hosts both models and datasets used in ML/AI applications. It's about as close to open source AI as I can imagine.

Some of those HuggingFace datasets are really useful. Like, for example, the OCRed version of the Epstein Files . That's rad as hell, and I'm glad there's a place to share those things. It's even a goldmine for researchers like me, since datasets containing model jailbreaking prompts are available.

But let's be clear about Mozilla's value proposition of the "transformative" generative AI. These are not small models we're talking about; these are large language models that were trained on massive corpora of text. Those corpora are the "source." We know that the training data for frontier models comes from copyrighted material and material scraped without consent. We also know that code generated from scraped sources may well violate the licenses of that source code in reproduction.

In other words, there can never be an open source large language model when the sources are themselves violate of content usage agreements. For all the talk about "ethical" AI, Mozilla fails to address this original sin of the technology.

I'm sorry, I should say "nearly fails." In the "Threats" section of their SWOT analysis of their own strategy, they identify "Open models disappear" as a threat:

Big tech / China stop releasing open models. No public open source frontier models emerge. Mozilla’s strategy is obsolete / outflanked.

Okay so by "open source AI" you actually mean Qwen/Deepseek/Llama. Cool. Cool cool cool. These are open weight only , so the premise of open data goes out the window. And this threat gives away the fact that Mozilla can only succeed on the backs of frontier models. There is no real plan to "democratize" LLMs, nor can there be for the scale required.

This entire exercise is a farce. Yet again, Mozilla pursues a parasitical relationship with the corpos. It worked last time, right??

Hypothesis 3: the growing need for "sovereign" AI. We've already established that there is no large language model possible without corpo scale and investment, except perhaps with government support. So is that what Mozilla wants? State-sponsored LLMs? This hypothesis points in that direction, with Mozilla as the "public interest tech player" catching a percentage somewhere in the middle. Being a government intermediary is also probably not a safe position for anyone at this juncture, much less a tech company.

But also, what "need" are we talking about here? Why is there a need for any of this at all?

Again we encounter the fundamental schism of purpose between Mozilla trying to survive, and the mission its core audience believes in. You could imagine a Mozilla that decided, "Actually, the web was better without this dreck in it, and the experience of the web is not improved by moving users closer to it." You could imagine an organization that doubled down on true privacy, and a human-centered web. We'll never know what kind of funding streams such an organization could build, because Mozilla has chosen the machines over people. They have chosen quick revenue over long-term sustainability.

It's finally time you and I take the advice I offered before: let Mozilla die. It no longer serves its stated purpose.

We Can’t Bridge the U.S.-Mexico Wage Gap

Portside
portside.org
2025-11-22 20:14:14
We Can’t Bridge the U.S.-Mexico Wage Gap Dave Sat, 11/22/2025 - 15:14 ...
Original Article

Members of the independent union SINTTIA rallied at a May 1 demonstration. | SINTTIA

The Mexican government is failing to prosecute violent retaliation and threats against workers who organize, says a new report , putting Mexico out of compliance with its trade agreement with the U.S. and Canada.

The report lists nine separate organizing campaigns in which threats were made against workers: in each instance, the authors found “little evidence of investigation or prosecution by the authorities.”

The report was authored by the Independent Mexico Labor Expert Board (IMLEB), set up by the U.S. Congress under the United States-Mexico-Canada Agreement and tasked with reviewing the implementation of Mexico’s labor reforms.

The USMCA, signed into law in 2020, was modeled on and replaced the North American Free Trade Agreement, which had gutted wages and pitted workers against each other while employers made out like bandits. In the first decade after NAFTA went into effect in 1994, 850,000 U.S. jobs were lost to outsourcing. Mexico, meanwhile, has become one of the world’s top vehicle-producing countries, but its auto workers are among the lowest-paid.

But in an important improvement over NAFTA, the USMCA mandated labor law reforms in Mexico that independent union activists had fought for for decades, sparking hope that the new agreement would help workers in the country raise standards.

The reforms established labor courts, supplanting arbitration boards that had been notorious for colluding with employers and sham unions.

Votes were mandated on all existing collective bargaining agreements, in an effort to help workers rid themselves of pro-employer “protection contracts” that lock in low wages and lock out genuine unions. Mexico’s Secretary of Labor estimated that these accounted for 80 to 85 percent of all union contracts prior to the reform.

It also introduced a first-of-its-kind rapid-response mechanism, which allows unions and workers to bring complaints against employers who violate Mexican workers’ right to organize. If a facility is found to be violating workers’ rights, it faces sanctions and may ultimately lose access to the U.S. market.

The mechanism has become a critical tool to enforce labor rights. It has benefited an estimated 42,000 workers, delivered nearly $6 million in back pay and other benefits, and given independent unions recourse against intimidation by unscrupulous employers and company unions.


FIGHTING DESPITE TERROR

These were critical improvements, but they have not led to the explosion in independent union organizing that many union activists had hoped for. Protection unions remain in control of most contracts.

The contract legitimation process was fraught with issues: incumbent unions ran the votes with minimal, if any, government oversight, and instances of coercion, bribery, and the spreading of lies that workers would lose benefits if they voted no were reported across the country. In the four-year period ending in July 2023, during which all existing contracts had to be put up for legitimation votes, workers approved 30,536 contracts and only rejected 663. Given these issues, the United Auto Workers is now calling for all contracts to be put up for legitimation votes again.

The USMCA has also not cut down on the wage gap between U.S. workers and their Mexican peers. On average , a U.S. auto worker makes about $35.30 per hour. Though their work is largely the same, Mexican auto workers make about $5.70, after decades of government collusion in wage suppression.

To this day, even workers in democratic, independent unions find themselves constrained at the bargaining table, since judges can rule a strike illegal if they deem labor’s demands to be unreasonable. In practice, the IMLEB report found, the line for many is capped at “inflation plus 1 percent.” Retaliation against workers seeking to organize remains common. There have been reported instances of violence or threats against workers in nearly every rapid response mechanism case, the report finds, including at large multinational employers like General Motors and French multinational Saint-Gobain.

Workers must face down threats not only from their employers, but also often from pro-boss protection unions that exist only to stave off true union representation.

In 2022, on the eve of a union election at a General Motors plant in Silao, Guanajuato, three people in a pickup truck with the license plates removed showed up to threaten the leader of the independent union SINTTIA at her home. Workers were also offered bribes to vote for a pro-employer union.

Despite these tactics, the Silao GM workers voted by a landslide to affiliate with SINTTIA, and have gone on to win record-setting contracts in the industry. But their case is an outlier. More often than not, intimidation chills an organizing campaign—whether the threat is job loss or physical violence.

Mexico is also failing to impose sanctions when workers’ labor rights are violated more widely, the report found. Though the text of the USMCA expressly requires that the Federal Center for Conciliation and Labor Registry have the authority to impose sanctions, the center itself has argued that it is unclear if it has the authority to do so under Mexican law.

Employers flagrantly breach their obligations with few consequences. Auto parts supplier VU Manufacturing was twice investigated for alleged labor violations at its Coahuila facility.

Rather than comply with the terms of a mediation plan agreed to by the U.S. and Mexican governments, the company simply shut down its facility. Union supporters have since been blacklisted.

“The problem is not the law,” said Willebaldo Gómez Zuppa, an advisor to SINTTIA. “The problem is its enforcement.”

A HIGH-STAKES REVIEW

The USMCA doesn't expire until 2036, but it is up for review in July 2026, when the U.S., Mexico, and Canada will consider extending it longer. If they don’t, it will be subject to annual reviews by the three countries for the remaining 10 years.

Trump, who negotiated the USMCA during his first term and once called it “the best agreement we’ve ever made,” has floated the idea of supplanting the agreement with bilateral deals. Tariffs imposed by his administration, under the guise of a national security exemption, have angered Mexican and Canadian leaders and put manufacturers on edge.

Unions are seeking to influence the upcoming negotiations to strengthen workers’ rights across the continent.

A group of Mexican independent unions, including the unions at Audi and Volkswagen, and the UAW each issued their own recommendations in October. Both groups advocate for a continental wage floor in the auto industry—also a recommendation made in the IMLEB report. The Mexican unions advocate for a starting floor of $16 (U.S.) per hour, phased in over five years. Currently , the USMCA requires that 40 to 45 percent of the content of a vehicle be made by workers earning this amount to qualify for preferential tariff treatment.

While the UAW does not specify a specific minimum wage, its letter notes that addressing wage suppression in Mexico would disincentivize offshoring and create “billions of dollars in new working-class purchasing power,” which could help jump-start demand for vehicles in Mexico.

The unions and IMLEB also recommend improvements to the rapid-response mechanism, which has been the most effective means for Mexican workers to enforce their rights. These include expanding it to cover blacklisting and bargaining in bad faith, and to cover agricultural workers and workers in the gig economy.

The UAW and the Mexican unions argue that the mechanism should extend to workers in the U.S. and Canada, too. An expansion would mean that a company found to be violating U.S. workers’ right to organize—for instance, Mercedes in Alabama —could lose access to the Canadian and Mexican markets.

A TRI-NATIONAL COUNCIL

In order to jointly formulate language and recommendations, the UAW is calling for the convening of a tri-national labor council of unions in strategic sectors. This will require speedy action from unions in all three countries, and a willingness to come together despite disagreements over issues like tariffs, which the UAW has backed..

Mexican workers still have a long road ahead to enforce their rights and build genuine unions. “Legal reforms alone are inadequate to produce a democratic transformation,” the IMLEB report reads. “It will take a concerted effort by democratic unions and progressive political leaders in all of the USMCA countries… to accomplish these changes.” Workers across North America have a lot at stake.

Labor Notes is a media and organizing project that has been the voice of union activists who want to put the movement back in the labor movement since 1979.

Through our magazine, website, books, conferences, and workshops, we promote organizing, aggressive strategies to take on employers, labor-community solidarity, and unions that are run by their members.

The Censorship Network: Regulation and Repression in Germany Today

Hacker News
liber-net.org
2025-11-22 20:11:47
Comments...
Original Article

Hier alles auch auf deutsch

Over the past several years, investigative reporting has uncovered a range of coordinated efforts to suppress online speech in Western countries. Often described as the Censorship-Industrial Complex , these networks of information suppression (mostly operating under the guise of countering “disinformation” or “hate speech”) have been particularly widespread in the United States, United Kingdom and the European Union. In the EU, Germany plays a central role in this system, with numerous governmental and private actors monitoring online speech and urging ever-greater levels of content suppression.

Germany’s reputation as a hub of censorship has grown significantly in recent years. In early 2025, a US 60 Minutes investigation drew international attention for clips of dawn apartment raids by armed police on people who had posted offensive memes. In other clips, state prosecutors chuckled at the seizure of citizens’ devices and emphasized the seriousness of the offense of “insulting” a politician.

We applied the methodology liber-net had used previously in our collaboration with journalist MattTaibbi on the Twitter Files, in mapping the US-focused Censorship-Industrial Complex. With German specialists, researchers and advisers, we documented this system at a scale far beyond what we anticipated. It includes:

  • a map of the leading content controls organizations in Germany, inspired by the original Censorship-Industrial Complex map (see above, or download the full-resolution image )
  • a report exploring the origins, politics and scope of German content controls
  • an organizations database including profiles of more than 330 government agencies, NGOs, academic centers, think tanks, foundations, and networks involved
  • a grants database of more than 420 content controls awards and grants
  • a series of infographics that visualize the above data

This reflects more than half a year of research and mapping. For more details of how we approached this research, read our methodology .

It is tempting to describe the more than 330 organizations as censorship advocates or even censors, but the reality is more complex. Some clearly are overtly censorious, such as the government-appointed “trusted flagger” organization HateAid . Similarly, the unironic Machine Against the Rage initiative provokes a “you can’t be serious?” response, but serious they are indeed. At the same time, our database includes more moderate initiatives, such as local governments offering small grants for counter-hate speech education as a way to combat increasing social fragmentation.

We’ve included them all in the database and ranked them from one to five flags – with five flags indicating the worst offenders – to provide a full picture of this massive, tangled sector that maintains unusually close relationships with the government whilst presenting themselves as independent.

There is a real problem in online discourse but we argue that heavy-handed approaches are liable to political weaponization and threaten free expression. We hope this material helps journalists, policymakers and advocates see the scale of current speech suppression, defend free expression and develop methods for dealing with beyond-the-pale content that resist political weaponization.

Das Zensurnetzwerk: Regulierung und Repression im heutigen Deutschland

In den letzten Jahren haben investigative Berichte eine Reihe koordinierter Bemühungen zur Unterdrückung von Online-Meinungsäußerungen in westlichen Ländern aufgedeckt. Diese Netzwerke zur Unterdrückung von Informationen, die oft als „ Zensur-Industriekomplex ” bezeichnet werden und meist unter dem Deckmantel der Bekämpfung von „Desinformation” oder „Hassrede” operieren, sind insbesondere in den Vereinigten Staaten, im Vereinigten Königreich und in der Europäischen Union weit verbreitet. In der EU spielt Deutschland eine zentrale Rolle in diesem System: Zahlreiche staatliche und private Akteure überwachen Online-Äußerungen und fordern immer strengere Maßnahmen zur Unterdrückung von Inhalten. Deutschlands Ruf als Zentrum der Zensur hat in den letzten Jahren erheblich zugenommen. Anfang 2025 erregte eine Reportage der US-Sendung „60 Minutes” mit Clips von Razzien bewaffneter Polizisten in den Wohnungen von Personen, die beleidigende Memes gepostet hatten, internationale Aufmerksamkeit. In anderen Clips lachten Staatsanwälte über die Beschlagnahmung der Geräte von Bürgerinnen und Bürgern und betonten die Schwere des Vergehens, einen Politiker „beleidigt” zu haben.

Wir haben die von liber-net in der Zusammenarbeit mit dem Journalisten Matt Taibbi bei den „Twitter Files” verwendete Methodik angewendet, um den auf die USA fokussierten Zensur-Industriekomplex abzubilden. Zusammen mit deutschen Spezialisten, Forschern und Beratern haben wir dieses System dokumentiert, dessen Umfang unsere Erwartungen bei Weitem übertrifft. Die Dokumentation umfasst:

  • eine Karte der führenden Organisationen für Inhaltskontrolle in Deutschland, die von der ursprünglichen Karte des Zensur-Industriekomplex inspiriert wurde (siehe oben oder lade das Bild in voller Auflösung herunter ) .
  • einen Bericht , der die Ursprünge, den politischen HIntergrund und den Umfang der deutschen Kontrolle von Online-Inhalten untersucht.
  • eine Datenbank mit Profilen von mehr als 330 beteiligten Regierungsbehörden, NGOs, akademischen Zentren, Thinktanks, Stiftungen und Netzwerken.
  • eine Datenbank mit Fördermitteln, die mehr als 420 Zuschüsse und Fördermittel für Inhaltskontrolle umfasst,
  • eine Reihe von Infografiken , die die oben genannten Daten visualisieren.

Dies ist das Ergebnis von mehr als einem halben Jahr Recherche und Kartierung. Weitere Einzelheiten zu unserer Vorgehensweise bei dieser Recherche finden Sie in unserer Methodik .

Es ist verlockend, die mehr als 330 Organisationen als Befürworter von Zensur oder sogar als Zensoren zu bezeichnen, aber die Realität ist komplexer. Einige sind eindeutig offen zensierend, wie beispielsweise die von der Regierung ernannte Organisation „HateAid” , die als Trusted Flagger fungiert. Ähnlich reagiert man auf die unironische Initiative „Machine Against the Rage” mit „Das kann doch nicht Ihr Ernst sein?”, aber ernst ist es ihnen tatsächlich. Gleichzeitig umfasst unsere Datenbank auch moderatere Initiativen, wie beispielsweise lokale Behörden, die kleine Zuschüsse für Aufklärungsmaßnahmen gegen Hassreden anbieten, um der zunehmenden gesellschaftlichen Fragmentierung entgegenzuwirken.

Wir haben sie alle in die Datenbank aufgenommen und mit einer bis fünf Flaggen bewertet – wobei fünf Flaggen die schlimmsten Verstöße anzeigen –, um ein vollständiges Bild dieses riesigen, verworrenen Sektors zu vermitteln, der ungewöhnlich enge Beziehungen zur Regierung unterhält und sich gleichzeitig als unabhängig präsentiert.

Es gibt ein echtes Problem im Online-Diskurs, aber wir sind der Meinung, dass hart durchgreifende Maßnahmen leicht zu politischen Zwecken missbraucht werden können und die freie Meinungsäußerung gefährden. Wir hoffen, dass dieses Material Journalisten, politischen Entscheidungsträgern und Aktivisten hilft, das Ausmaß der derzeitigen Unterdrückung der Meinungsfreiheit zu erkennen, die freie Meinungsäußerung zu verteidigen und Methoden zu entwickeln, um mit unzumutbaren Inhalten umzugehen, die sich nicht für politische Zwecke missbrauchen lassen.

Data General History by Foster

Hacker News
www.teamfoster.com
2025-11-22 20:09:48
Comments...
Original Article

The Fair Bastards

It was the best of times, it was the worst of times

It's tough to describe my three years at Data General.  On the one hand, DG and its founder Edson de Castro are responsible for almost all of my business success.  It just would not have happened if Ed had not brought me east in 1976.

On the other hand DG did some really weird things.  Some things were kookie and funny, but others were wrong and mean-spirited.  As CEO, Ed is ultimately responsible for many of the things I experienced at DG -- both good and bad.  But it's not easy to be critical.  Forty years later I have come to know Ed in a different light -- far removed from the pressure and tension of business.  He is a good person with many positive qualities.  So, given the caveat that I consider my old boss to be a good friend, I will try my best to accurately describe my three important yet crazy years at his company.  There's no way I could describe my life in the computer industry without including Data General, and there's no way to write about DG without mentioning the bad along with the good.

Culture Shock

I loved the Data General, but sometimes I hated it.  Not hate -- that's too strong -- but I really disliked some of the things than went on.  And I was confused.  I had been suddenly thrown in to the real world, after seven years of insulation while at Hewlett Packard, the giant love-in.

I loved the freedom.  I loved the fact that my boss left me alone — as long as I stayed within budget I could do almost anything.  But there was a lingering feeling of distrust.  I felt sometimes that I was not to be trusted — it was like some of us had ulterior motives.  And the customers, the folks who buttered our bread — they were sometimes treated strangely, if not harshly.  DG’s culture was an anathema to me — in many ways it didn’t pass the common sense test.

The company was a puzzle.  It broke all the rules and yet was extremely successful.  It had the reputation of being the "bad boy" of the computer industry -- heck, of any industry.  And it enjoyed this reputation.  DG became a public company less than two years after it was founded and shattered records in making it to the Fortune 500.  Too bad Harvard never wrote a case study.  It would have been groundbreaking!  But if they did Harvard would have had to admit that EVERYTHING they taught about how to run a business could be wrong…

I was given very few orders from my boss.  Just get the job done.  I loved reporting directly to the president of the company —  Edson de Castro, The Captain.  The founder of what by 1976 had become one of the most exciting and successful computer companies in the world.  Although I had a great job at Hewlett Packard I was way down the totem pole at that huge company.  At DG I was close to the top.  Close to being a star!!

At HP you felt secure, no matter how bad you screwed up.  It would have taken an act of bloody murder to get fired.  The contrast with DG was extreme. Employees were kept on edge.  Your job was simple:  help the company make a profit — that’s why you’re here. We were all given a lot of rope.  But if you screwed up just take that rope and hang yourself before someone does it to you.  It was clear that everyone was expendable.

Stockholders were put on a pedestal.  Keep them happy.  How?  Profit.  Everyone at DG knew the company was driven by profit — at all costs.  This was new and different to me — at HP people hardly knew how to spell profit.  DG was totally counter to the “HP  way” — they were as different from Hewlett Packard as a cow is to an orange.

Ok, I loved the place and I disliked a few things.  So what?  The net result was very, very positive.  In fact, it was life changing.  Ed brought me from Silicon Valley to sweet New England, out of la-la land and into the real world.  DG helped to make me tough.  I was a cream puff at HP —  I never could have started Stratus straight out of that company.  Joining Ed’s company put me on the path to becoming an entrepreneur.

My first day at DG was right after Halloween, 1976.  I remember it well.  Marian and I had gone to a costume party at our friend’s houseboat at Jack London Square in Oakland.  The party was fun as usual but I knew this was my last day in California — the next morning I would be leaving my family behind, getting on a plane for Massachusetts.  I hated the thought of leaving them for the next two months.  The plan was to let our three small kids finish their semester at school before the whole family moved out in January.

All five of us were natives of California.  Nobody left that enormous state back then — everyone was trying to get to the Left Coast.  Our family and friends thought we were crazy.

My title at DG was Director of Software Development, reporting directly to Ed.  All of his other direct reports had the Vice President title.  Oh well, maybe one day I would be a VP…

Right away I discovered that software was not at the top of the food chain.  My people didn’t work in headquarters with everyone else.  Instead they were exiled a few miles down the road to an abandoned shopping center.  They shared a building with the cable-cutting operation.  The programmers created software while listening to the constant CHUNK, CLUNK, CLANK of the cable-cutting machines.  I learned that the previous year there was talk of moving software all the way up to Maine.  A crazy idea — luckily it fell through.

For you non-computer people you need to understand that separating the software people from the engineers who design the hardware was very wrong.  Software is the heart of a computer.  A computer is useless without the basic stuff that my people developed:  the operating system, programming languages, data management software, communications, etc.   But DG didn’t see it that way.  Its roots were hardware.  Software was a necessary evil, created by hippy-freaks.

But in spite of the less-than-satisfactory working conditions the people were great.  DG had attracted a bunch of top notch software engineers -- equal to if not better than the ones at HP.   I was used to the shiny new buildings that HP had built in the middle of some Santa Clara Valley orchards.  I took me a while to figure out that if the programmers loved the company and the people that they worked with, the building didn't matter --even if it was an old supermarket.

DG was 8 years old when I joined.  It had spun out of Digital Equipment Corporation in 1968.  Ed was a young engineer at DEC and tried to convince its founder Ken Olsen to drop the 12-bit line in favor of 16-bits.  In fact, he had a design for the new computer.  For some reason Olsen wasn't interested.  So Ed and two others took the design and started Data General.  Olsen didn’t sue — it wasn’t his style.  But years later he was quoted in Fortune Magazine saying “What they did to us was so bad we’re still upset about it.” I’m mad as hell but I’m not going to do anything about it.  Not surprisingly, Ed was always a little paranoid about anyone spinning out of DG and starting another computer company.

Ed is on the left. Herb is the good looking guy on the right :)

Ed is on the left. Herb is the good looking guy on the right :)

At its beginning in ’68 DG was essentially selling "naked" boxes with minimal software to highly skilled engineers.  But by the time I joined a lot of their new business was in the commercial world, where their customers depended heavily on software that my folks developed.  But DG still had the box mentality. Software was not part of their culture. This sometimes created big problems as DG entered the commercial marketplace.

In the year that I joined, 1976, Data General was flying high — one of the hottest companies in America and the clear number two in minicomputers, right behind DEC.  Excellent financials. I didn’t know much about them while at HP — we didn’t cross paths much.  I do remember being at a trade show in Las Vegas where DG posters hawking their products could be seen everywhere — at the airport, taxicabs, buses, all over the convention hall.  It was very unusual for a tech company to be that much in your face. At another trade show in New York I saw a big crowd around the DG booth.  I elbowed my way to the front and there was a belly dancer with a micro-Nova chip in her navel.  A very bold thing for the stodgy computer industry.  DG’s booth was the hit of the convention.

During my first week my head was spinning.  What’s going on?  I felt like Alice in Wonderland — everything was upside down and backwards. Or maybe I was the weird one and what I was seeing was normal....

One of my guys told me that DG was about to ship a newly developed disk drive, but my programmers had never seen one — they had never been able to check out their “drivers”, the software that controlled it.  There was a great chance that it wouldn’t work — but when I called Paul Stein, the manufacturing VP, and asked him to stop shipments he told me “Forget it.  The disks are going out the door.  I need them to make my profit goal for this quarter.”  Apparently Stein felt profit trumped shipping things that worked.  Once it’s off the loading dock and in the truck we could log the revenue and satisfy the stockholders.  And Paul would make his goal.  Fix it later.

One of my people had his car squashed in the parking lot.  DG was expanding the building and during construction a front-end loader rolled over the poor guy’s car.  The kid always came to work at 6am so his car was parked in the front row.  DG refused to cover his deductible.  Jim Campbell, our VP of Personnel told me, “sorry Bill, it’s against policy. It's between him and his insurance company…”  I organized a fund and we all quickly chipped in the $500 that the kid needed.  This was really silly -- DG missed a golden opportunity to creat tremendous good will with their people.

At Hewlett Packard it seemed like we were always writing things down -- producing tons of plans, most worthless.  A waste of time.  DG had the opposite mentally --  don't put too much stuff in writing.  It might get into the wrong hands.  Our arch-enemy DEC might steal our secrets.  At Data General secrecy was a highly valued commodity.

Scary security guy

Scary security guy

One morning I was accosted by a couple of burly security guards as I mistakenly tried to enter through a side door at headquarters in Southboro.  Remember the movie The Firm?  The chief of security was played by Wilford Brimley. Big, tough, no nonsense, walkie talkie glued to his hand.  Looking back that’s what one of the guys looked like — a spitting image of Wilford Brimley.  He demanded my ID.  I was held there while he called it in, making sure I was who I said I was and not a spy from DEC.

While in the cafeteria line I watched Herb Richman, a co-founder and VP of Sales, take a bite out of a customer’s sandwich and then put it back on his tray.  Cracked me up.

Herb was a character -- a salesman's salesman.  Fun loving but tough.  He had created one of the best sales forces in the industry.  A few years later when Bob Freiburghouse and I had teamed up to start Stratus Bob said "let's hire DG salesmen.  They are the best in the industry."  He knew -- Bob worked with  DEC, Prime, and Wang as well as DG and he saw the differences.

I watched Herb’s Mercedes being towed away from visitor parking in front of our building one morning.  Ed had his co-founder’s car towed because no employee was allowed to park there, not even Herb.  I saw this towing thing happen several times over my three years.  I wondered if it was really a set-up between Herb and Ed — to show everyone that at DG nobody got special treatment.  But a good friend who knows both of them very well said he is certain it was real.  Herb was constantly trying to pull something over on Ed but he could never get away with it — Ed was just too smart.  And just because Herb helped found the company he wasn't ever going to get any special treatment.

I learned right away there was almost no management structure for my 140-person software group. Programmers pretty much roamed around, doing whatever they wanted, with almost no accountability.  For example, there were three projects working on three different versions of a Basic Interpreter.  We only needed one.  Meanwhile other software products that we needed badly weren’t being done.  Management by Brownian Motion.  This is why they hired me -- Ed knew that the software department was out of control.

My first evening on the job Ed hosted a dinner party at the local Sheraton so that I could meet the “managers”.  It turned out he and the personnel  guy had to guess at who to invite because the software group had no formal structure.  Several people were invited who definitely were not managers and some real managers were passed over.   This pissed off a lot of people -- one of my first jobs was to calm them down.

One morning Jacob Frank, the company’s chief lawyer walked in to my office with a document.

“Bill, you need to sign this.”

“What is it?”

“Your employment agreement.  Everyone who reports to Ed has to sign.”  I quickly read it.  The document said that everything I did or even thought about while working at Data General belonged to them. EVERYTHING, whether it had to do with computers or not, and for ONE YEAR after I left!!  They owned me and my thoughts now and into the future!!!  I’d already burned all my bridges to HP.  There was no turning back.  I had to sign. What I didn't realize then was that non-compete agreements were not uncommon at the higher levels of businesses.  In my naivete I figured they were pulling a fast one on me, because nothing like this existed at HP.

This little piece of paper would haunt me for years to come.  It made me feel like my skinny neck was glued to the chopping block with an ax hovering just above, ready to slice.   The agreement almost prevented me from quitting and pursuing my Big Idea.  Later, just as Stratus’ venture capital deal was about to close, our investors discovered this agreement and nearly backed out.  In 1981 I lied to Harvard when they wrote their case study about Stratus because even though I had left DG more than two years earlier my ex-employer and this agreement still worried me.

In the middle of the night during my first week I was jolted awake with an epiphany!!  I know what’s going on — this isn’t real — it’s a test!  Kind of like Candid Camera — someone created a totally crazy scenario.   If I survive I'll become President of Data General!!!  That must be what’s going on — it’s just a big, elaborate, test…  If I pass I will run the entire company!!

When I came back down to earth depression set in big time.  Had I made the biggest mistake of my life?  HP would not take me back.  We had sold our house in California.  We were committed.  There was no turning back.

I needed a break.  A diversion.  I needed to relax.  I decided to go to the movies. There was something called The Marathon Man playing nearby. It had a good cast: Dustin Hoffman, Laurence Olivier, Roy Scheider.   How could I go wrong?

Is it safe?

Is it safe?

Let me tell you, if you ever want to chill out with a relaxing movie, never, EVER watch The Marathon Man.  It is one of the tensest movies imaginable, especially if you're already frazzled like I was.  When Laurence Olivier used a dentist drill to dig into a nerve in Dustin Hoffman’s tooth, softly saying “is it safe?” while Hoffman screamed bloody murder, I bolted from the theater.  I couldn't stand it. The worst possible movie that I could have picked at that time.

My first priority was to put together some kind of structure for the 140 people who worked for me.  For the first month I spent most of my time organizing the group.  Getting to know everyone.  Searching for some leaders.  I found a few good ones within software and managed to convince a couple of others from different parts of the company to join me.  Finally I was ready to present my new organization to Ed’s staff.  It was pretty standard for software development in a computer company: operating systems, languages, communications, user manuals, and FHP — DG’s new super secret computer system.

After the meeting Ed pulled me aside.  “Don’t mention FHP around Herb.”  The soft spoken guy almost raised his voice!

“How come?  He’s on your staff and he’s a founder of the company.”

“Herb will leak it.  He’ll sell it.  Don’t talk about FHP in front of Herb.”

Hmmm.  The most important project DG was working on — the future of the company.  And Herb, one of the founders, was supposed to be kept in the dark.  I began to realize Ed had a fairly unusual relationship with the sales guy that helped him start DG.

In the early 80's, after I had left DG, group of sales guys started a band to chronicle the culture of this wacky company.  Check out the song by The Talkingpropellerheads about Ed and Herb, with Dan Fennelly leading the group.  Ed's wife Eileen was an occasional member of the band -- that shows how tolerant Ed was of much of the fun stuff that went on.  The guy that ultimately replaced Ed killed the band....

Edson was an enigma, just like his company.  He was (is) a brilliant person.  He flies his own jet and could probably take the whole thing apart and put it back together again, blindfolded.  One day I saw him pouring over the schematics for some complex piece of electronics.  He had just added some new avionics to his Beechcraft Baron and wanted to completely understand the design.

He was a shy person.  One-on-one conversations could be awkward.  Oftentimes in his office when we were discussing something and then the conversation finished he would just stop talking and look down at his Wall Street Journal, which was always open on his desk.  No “I’ll see you later” or “have a good day” or any conventional closure. Just silence.  I would stumble backwards out of his office, trying to avoid falling on my ass.

Maybe it was the work environment that made him reticent to make small talk with his employees.  Maybe he didn't believe in it.  I know from experience it is very hard to be good friends your people, and then have to take harsh action like demoting or firing.  I think in hindsight Ed kept his distance from his people for good reason.

Ed is very soft spoken.  I’ve never heard him raise his voice, no matter how angry.  He could whisper “that’s bullshit” in the softest possible voice but somehow the words penetrated down to your bones.  But put him in front of a large group and he was dynamite.  At annual meetings or sales events — anything with a big crowd, he was great.  Always in command of the facts, a superb public speaker.

There were two classes of people at DG:  the Insiders and the Outsiders.  Ed, Herb, and Jim Campbell were the insiders.  Jim was the VP of Personnel, and an old high school friend of Herb’s .  Everyone else felt like an Outsider. We were not part of the inner circle, and were always a little on edge.

It was very odd that Herb, a co-founder, had very little authority.  On paper he ran sales and marketing but in practice it seemed like he couldn’t sign for a cup of coffee.  Many mornings I would see him camped out in front of Ed’s office, waiting for the magic time of 10:30, waiting for The Captain to arrive, waiting to get authorization for whatever.  Something must have happened before my time to cause Ed to strip Herb of all power.

The World’s Best Computer

Back to FHP —  Fountainhead Project.  DG’s new, SUPER SECRET computer.  So secret they rented rooms miles away from headquarters in the Fountainhead Apartments so that the hardware and software guys who worked on it wouldn’t mingle with the rest of the company.  Keep them away from mainstream DG.  Keep them away from Herb.  Keep them away from spies.

A fellow by the name of Bill S. ran my part of the project — the software part.  The hardest part.  At first glance Bill seemed like a capable enough person.  Smart, witty, seemed to say all the right things.  I asked him to tell me about FHP.  Describe the product, the computer’s architecture.  Bill said “I could do it but the best person would be Jerry.  Talk to him.”

I found Jerry.  He suggested I talk to Lem — “Lem is a great communicator and has a super presentation on FHP.”  I tracked down Lem.

“Bill, the world’s best describer of FHP is George.  While I could tell you about it, George would do a much better job.”

I began to smell a rat.  This project has been underway for more than a year and I was getting the runaround. George finally sat me down in front of a black board and starting drawing.   He didn’t have anything in writing (big surprise) — he just put up a bunch of things on the board.  The “exciting” thing was what they called a “soft” architecture.  This meant that the instruction set was not fixed — the machine would swap instructions in and out depending on what language the application was written in.  One set for Cobol, another for Fortran, etc. There was also a plan to imbed much of the OS in microcode.  The rest of FHP was pretty standard:  multi-tasking, virtual memory, maybe multi-processing.

Variable instruction set?  Burroughs tried that and it bombed.  Way too complex, and switching between instruction sets was very slow.

Embed the OS in microcode?  That sounded really tough, really risky.  I asked George what the goals were for FHP?  What were the design objectives?  “No goals in particular.  Just build the world’s best computer.”

“When will we be shipping?”

“Haven’t figured that out yet — probably in two or three years”

I left the building even more depressed that ever.  Comically, I couldn’t help but notice that FHP had taken over several ground floor apartments at Fountainhead.  Everything George drew on the blackboard was facing the parking lot.  Anyone could easily look through the windows to see DG’s plans.

Now, this gets to my biggest failing at DG.  I should have tried hard to kill the project right then and there. There was no frigging way that FHP as described was ever going to see the light of day.  Telling a bunch of engineers to “build the world’s best computer” was crazy.  Computer geeks need limits, boundaries, deadlines.  Otherwise the project gets hopelessly complex and nothing gets out the door.  Keep it simple, stupid.

I didn’t have the power to kill FHP but at least I didn’t just sit on my thumbs.  I immediately wrote Ed a memo — about 15 pages.  I put my heart and soul in to it.  It meant so much to me that I still have a copy, four decades later.  The first 10 pages were pretty general stuff under the heading “A newcomer’s View of DG.” For example, we needed to always test software on new hardware before it shipped.

The last five pages focused totally on FHP.  HP taught me that long, complex computer projects always fail.  I told Ed we should drop the “soft” architecture — it was way way too complex and risky.  Instead, there should be one small, simple instruction set.  FHP should be a 32-bit mid-range product, not a big, expensive main-frame.  We needed a replacement for our 16-bit Eclipse.

Sadly, my memo had no effect.  Nothing changed.  FHP continued down a path towards oblivion.  An enormous waste of money.  Not being able to either kill or drastically change FHP was my biggest failing at DG.  I suppose I could have been more forceful, but I wasn't.  My bad.

Gonzo

DG hired several new VP’s between 1975 and 1976.  Business was growing fast and they needed to beef up management.  While HP nearly always promoted from within, Data General looked outside to fill most important jobs.  I benefited from both sides of that equation.  At HP I was promoted to run the computer development group while I was still very wet behind the ears.  And DG rarely looked inside to fill big positions.  I don't know how many candidates they went through for my job but I know I wasn’t their first choice.

DG hired Dick Weber from Honeywell to run U.S. Sales.  Back in the ’70’s Honeywell was one of many companies in the computer business — one of many that are now defunct.  I really liked Dick.  He was smart, very capable, a real straight arrow.  I went on numerous sales calls with him, always had a good time and learned a lot about selling.  I was an engineering geek and figured it couldn't hurt to learn how to sell.

One day Herb stuck his head into my office.  “Hey Bill, how’s it going?”

“What’s up Herb?”

“Ed is firing Weber right now.  I know you like Dick and I wanted to make sure you don’t freak out.”

This was crazy.  Dick was one of the good guys.  And US sales were booming.

“Why is Dick being fired?”

“He doesn’t fit in.  We don’t like him.  But I want you to know that you do fit in.  We do like you.”

Gee Herb, thanks.  It's nice to know that I fit in.  My thoughts -- I didn't have the guts to say it.  I never learned exactly why Weber was canned.  But this episode really put me on edge for the rest of my time at DG.

Three years later I tried to recruit Dick to join me as a co-founder of Stratus — to run sales and marketing. Dick wasn’t going for it.  He was a big company guy — he had returned to Honeywell.  No interest in hanging out with a bunch of engineers in a dumpy old building.  Oh well.  He would have been good.  But we lucked out and eventually got a much better guy from Honeywell — a little man in a green rumpled suit who years later went on to run and grow Cisco, one of the computer industry's iconic companies.

Having an office next to Herb’s was always a ball of laughs.  Once when I was interviewing a techie from MIT Herb suddenly walked in with the latest issue of Playboy.  Sticking the centerfold in my face he said, “Hey Bill, what do you think of those???”  The MIT geek turned beat red and mumbled something about going up the road to DEC.

Customer Dissatisfaction

I was still brand new at DG when Ed asked me to come with him to Chicago the next day.  “We’re going to visit Hyster.  They’re really pissed off.  Tom is coming with us.”  That would be Tom Cook, our VP of Customer Service. Tom had recently joined from IBM.  He was a another really good guy.  Really knew his stuff.  If a customer visit involved Tom they must have been very upset.

I knew of Hyster, the fork lift company.  At the tail end of my HP days we were fiercely  competing with DG for the Hyster account.  DG won and we were devastated.

We walked into the Hyster board room.  The chairman and about 10 of his VP’s were there.  The chairman proceeded to tell us how crappy our product was.  The computers kept crashing — and a some of the promised software hadn’t been delivered.  “What are you going to do about it?”

Ed spoke softly:  “I know that you’re upset.  But you have to realize these are computers.  There are always problems.  Especially with the software.  That’s why I brought Foster.  He will get that stuff working.  But meanwhile, you’re not living up to your volume purchase agreement.  We gave you a big discount, and we expect you to take all the Novas that you agreed to buy.”  My jaw dropped.  Tom Cook’s jaw dropped.  We could’t believe what our boss just said.

The Hyster guys couldn’t believe it either.  “You mean you want us to keep taking that crap even though it doesn't work??  Really??  Please leave the room, we need to talk in private.”

In the hallway outside I wanted to ring Ed’s neck.  I wanted to shout “YOU JUST PISSED OFF A CUSTOMER!!!  NOT JUST ANY CUSTOMER, BUT AN EXTREMELY IMPORTANT ONE!!!”  But I didn’t say say a thing.  I was still in shock, and besides, I wanted to keep my job.

Ed spoke:  “I really love to pull their chains.”  He was having fun!!  Ed thought that pissing off a customer was cool!  Tom and I were flabbergasted.  It wasn’t because we were from IBM and HP — customer friendly companies. NO company in its right mind would do what Ed just did!

Ed knew that Hyster was pretty much locked in to our stuff — they had invested lots of money in software that would only run on DG computers.  So maybe that’s why he didn’t care.  No matter, it still didn’t make any sense. We eventually fixed the problems and Hyster was satisfied.  But the next time they needed to develop a new computer system do you think they chose DG?

I’ve seen Ed many times since we both left DG.  I’ve had plenty of opportunities to ask him about that crazy Hyster trip, but I never have.  I guess I’m still a little intimidated by him.  Maybe that’s natural — your old boss is still your boss, even 40 years later….

Ed normally started work late, around 10:30.  But there was one time that I remember him coming in early.  One morning sitting at my desk I was startled to see Ed walk by.  It was only 8:30!  What’s going on! Then I remembered the strange flag flying on the pole next to Old Glory.  Turns out it was the North Carolina state flag.  Their governor was visiting.  Ed came in early to greet him!!  Ed was in the process of sticking it to Massachusetts and its liberal, anti-business attitude, and was starting a division in North Carolina. Those guys were tickled pink to be taking some jobs away from Massachusetts.

Ed treated himself like everyone else —no special perks for The Captain.  By the time he got to work each morning our big parking lot was filled — Ed parked in the back about a mile away…

One cool thing about DG was stock options.  We didn’t have them at HP.  I had never heard of them.  When I hired on I was given a few thousand options.  Here’s an example of how they work.  Let’s say you are granted 100 options today,  at today’s stock price.  But the kicker is that you can buy these shares at some point down the road —maybe as long as five years from now — but, at today’s price.  So, if the stock is trading for 10 bucks today, but 5 years from now it’s at $50, you can “exercise” the option of buying shares at the $10 price!!  Then you can sell the shares for $50.  A cool 40 bucks per share profit.  A really smart concept — everyone wins, no one loses.  It motivates people to get the stock price up.  Shareholders like that.  But if the stock price stays the same or even goes down nothing happens —  no one gets hurt.  You would not pay $10 for a stock that you could buy on the market for $5.

It’s amazing how the press always screws up the reporting of options.  As an example, suppose when I was CEO of Stratus I was granted by the board 100 options at today’s price of $10 each.  The press would say, “Bill Foster, CEO of Stratus, was just given $1,000 in Stratus stock.”  That’s total bs.  I wasn’t given anything of any value — when first granted options are worthless — they only have value if the stock price goes up.  I don’t know if members of the press are just stupid or it’s their liberal bent that makes them want to trash management.  Options are consistently misreported by the press.

Soon after I was hired Ed told me, “Bill, I could give you your options now.  But I think our stock is overpriced. If you want you can wait to see if it goes lower — then you can have your option.  Your call, just tell me when you want the option.”  That was really cool.  And smart.  As far as I know it didn’t break any rules.

Another thing that was totally new to me was DG’s Million Dollar Club.  They held their first one soon after I joined, in the spring of 1977 — in Bermuda at the Sonesta Resort.  Anyone who sold a million bucks of computers got to go.  It turns out that these types of sales clubs were pretty common in the computer industry.  Almost everyone had them — except stodgy old HP.

The 60’s and 70's were the pinnacle years for the computer industry.  Business was booming.  Gross margins for hardware were 80% or higher.  That means that if a company had $100 million in annual sales it would have $80 million left over after the computers were built and shipped!!  $80,000,000 to cover engineering costs, sales, marketing, and overhead.  And there should have been a bunch left over for profit. Because of these huge margins computer companies had plenty to spend on ridiculous trips and prizes for their pampered sales people.

Typical computer salesman

Typical computer salesman

At DG’s first club there were about 20 salesmen — they were all men. Ed invited me and a few other “executives” to come along, so that the sales "winners" could rub shoulders with us and feel even more special. This was great — I loved it, I had never been to Bermuda, it was a free vacation — what’s not to like?  The only negative was we had to share rooms.  My roommate was Paul Stein, the manufacturing VP.  Paul had recently come over from Burroughs.  He was an ok guy but not exactly a ball of fun — pretty much all business all the time. Plus he snored to high heaven, but maybe I did too, who knows??  Who am I to complain?  In the early days of Stratus we doubled up all the time.  At the Red Roof no less!!

The highlight of the event was Herb’s closing speech to the troops.  Herb was funny, witty, amusing — as usual.  Lot’s of laughs.  It turns out at the same time we were there IBM was having it’s Golden Circle event at the South Hampton Princes just a mile or so away.  They had taken over the ENTIRE large hotel.  Wives were included.  Henry Kissinger was the guest speaker.  Harry Belafonte for entertainment.  We at little Data General had Herb for both.  But he was great.  Herb ended his talk to the sales team with this:  “I’ve been telling you guys all along to screw IBM.  But if you can’t do that then go down the road and screw their wives!!!!” Typical Herb.  He broke the place up.

Conventional wisdom is that the engineers are the really smart ones in computer companies.  But I’ve decided it’s really the guys in sales.  They somehow convince everyone that to do their jobs right they needed special incentives:  Winner’s Circles,  Million Dollar Clubs,  Golden Circles,  free trips, free stuff, awards, plaques, pats on the back.  And huge commissions, especially if they beat their quotas.  What do engineers get for doing their jobs? Or folks in manufacturing?  Or accounting?  YOU GET TO KEEP GETTING YOUR PAYCHECK!!!  Come on, engineers, dummy up!  The sales guys are laughing all the way to the bank.  And to Bermuda.

In early 1977 we moved into our new corporate headquarters in Westboro.  The new digs were a big improvement, and included an executive row.  All the VP’s and Ed sat in a row of big offices at the front of the building.  Behind a glass door.  No one could accidentally walk past an important person’s office — it took a special, concerted effort to enter executive row.

I was the only non-VP there — I was only a Director.  When I went to see the facilities guy about my new office he looked at me like I had two heads.  “What makes you think you’re getting an office up there? You’re not a VP!”

“Ed told me I could have one.”

“Really?  Are you sure??  I’ll call him.”  For the first few days at DG I thought I was a big deal, but quickly I learned I was just another pogue.  This was another example of pushing me back down to earth.  The facilities guy, Dom, spoke to Ed in a hushed tone and then turned to me.  “Ok, I guess you’re right.  I’ll get you an office up there.”  Gee, thanks -- thanks for admitting that I DIDN'T LIE TO YOU!!!  Oh, the ignominy of it all!! More proof that I was of no importance!

At least with this office I wouldn't look like a second class citizen in front of my troops by not being on Executive Row with the Big Boys.  And our large offices were really cool!  Dom assigned me to an office right next to Herb Richman.  This had to me one of the most entertaining spots at headquarters.

Now that we were in the new building my guys no longer had to work in a run-down shopping center along with the cable cutting operation.  But they were located about a mile from my office — at least it seemed that far. Software was placed downstairs at the back of our enormous building.  They were as far from me as they possibly could be.   But I would not give up my office in executive row.  I felt important up there.  I did, however, sneak in a secret desk right in the middle of software so that it was easier to  hang with my people.

Foolish Programmers

One morning Jim Campbell rushed into my office, grabbed the Data General Mini News out of my in-box, and as he left he said “Bill, you need to get control of your people!”  What’s going on?  I hadn’t yet read the latest issue of our company newsletter.

I should have guessed.  It was April Fool’s Day, and some of “my people” had come out with their own edition of the Mini News.  Campbell was scurrying around trying to grab them all up.  I called down to Matt Blanton, one of my guys, and told him to hide his copy.  I still have that newsletter.

It was a was pretty benign joke — not too horrible, in my opinion.  Jim should have just left it alone. Maybe he was worried about the announcement of a $5,000 cash bonus for referring new hires. There was no such program.

Or the article about the town of Westboro voting to move their town line to the west so that DG would no longer be part of the town — this was actually pretty funny.  The article mentioned that Westboro didn’t want to process all sewage that we were producing, and that Southboro, on the other side, had already moved their town line further east so that DG headquarters would no longer be in any township in Massachusetts.  A man without a country...  This article was immediately followed with the headline “Need Fertilizer?”  DG suddenly had a surplus of unprocessed organic fertilizer which anyone could pick up for free.  To top it off it was being packaged by Servamation, the people who ran our cafeteria.

Another article entitled “Muskrat Love” mentioned that we were being sued by the Captain and Tennille for $1 million because of the constant sounds of muskrats in our phone system.  At the end of the article there was this:  “Caution!!!! Turn down your telephone buzzer — muskrat mating season is about to begin!!!”

Another headline:  “VP Takes Leave”  “Herb Richman, comedian and part-time DG Vice President, will take a two-year leave of absence beginning next week to join the Kansas City Bombers roller derby team.”

The newsletter ended by reminding everyone that since April 2 was John Galt’s birthday it was a paid DG holiday and everyone could sleep in.  There were some pretty funny folks at Data General.  And of course since Jim was trying to destroy the fake newsletter everyone devoured every word.

More Humor

The new headquarters building finally placed software next to hardware, which was a good thing.  Software and hardware are integral to a computer system.  It was kind of hard for the software guys to do their job when the guys who developed the hardware were in the next town.

But being co-located did cause some problems.  Just like at HP, the DG software guys were a little weird — a little off-kilter.  The hardware guys were pretty normal, as far as geeks go.  Late one evening I got a call from Tom West, DG’s top hardware guy.  He sounded angry.  “Bill, I was walking through software on my way to the parking lot and I got hit in the head with a frisbee!! What the F!!!”  I went down to investigate.  Turns out there was a good reason: The guy who threw the frisbee was riding a unicycle through the halls.  The next day I told Tom I would not let them toss the frisbees "whilst ridding unicycles."  He wasn’t amused.

Washing Machine or Disk Drive?

Washing Machine or Disk Drive?

Back in those days some computer disk drives looked like washing machines. About the same size, with a removable stack of 15-inch disks inside. The idea was that when you filled up the stack you could remove it and drop in a new one.  The disks had a big, maybe 5 inch hole in the middle.  One of our guys thought it made a perfect hat.  Dan would walk around the building with the disk on his head, with his ample thick, curly hair poking through.  No one gave it a second thought.

These washing machine drives had a quality problem — the lids on top kept randomly popping open causing the computer to crash.  One day I walked into the lab and saw a heavy construction brick on the top of the disk drive holding the lid down.  The brick was painted in the official DG blue color, with a DG logo pasted on the side.  The construction brick looked like an official part of the DG product line.

Matt Blanton, one of the guys that I recruited to help run software, called me early one morning.   “b-b-Bill, I’ve decided to f-fire Dan but he’s here in my office with a t-t-tape recorder.”  Matt, the nicest guy in the world, was a little upset.  He was thrown off kilter by the tape recorder, and this briefly affected his speech.

I told him “don’t worry about it, just fire Dan if that’s what you want to do.  If he decides to sue us we have an army of lawyers — with plenty of experience.”

I had started to get used to this firing thing.  At HP we never fired anyone.  It would have been impossible to fire someone and besides,  that would be against The HP Way.  I’m sure Matt had a good reason for getting rid of Dan, but I didn’t ask — I trusted Matt’s judgement.  Dan probably screwed up one too many projects.

In the summer of 1977 I fired Bill S., the leader of the super secret FHP project.  He was screwing up everything, missing every schedule, way over budget.  The more I got to know him the more I learned that he was just a giant b.s. artist.  (Hence, his initials?)  Almost everything he said was crap.  Sounded good at first, but was always crap. The final straw was a trip he took to a trade show.  Bill had been twisting my arm and insisting it was vital for him to go, to  see what the competition was doing.  He never showed up at the show, and spent the entire time visiting old friends in San Francisco.  I was looking for a reason to get rid of Bill and finally had one.

It all happened quickly.  At the end of the day I called him in to my office  —  I told Bill he was fired.  He didn’t protest — he had sensed for awhile that he was on thin ice.  Security escorted him out the side door, straight to his car.  He was gone.  Everyone in software pretty much knew that when Bill S. didn’t return to his desk he was gonzo.

The very next week I needed to talk to one of my guys, Jit Saxena— one of our best people.  It was late in the day. When we finished our conversation Jit figured rather than making the long trek back through software he would just go out the side door straight to his car — he never returned to his desk.  Everyone figured I fired him.  When he told me the next morning about all the calls he got that night consoling him for the loss of his job, we had a big laugh.  A big, nervous laugh, because at DG this could happen to anyone.  At any time.

Best Mission Statement Ever

In 1978 Fortune magazine wrote an article about DEC and DG.  Mostly about the animosity between the companies.  It talked about how Ken Olsen thought the Nova rightfully belonged to him because Ed designed it while at DEC.  But the highlight of the story was Herb Richman’s quote when he was asked about DG’s tough reputation.  Herb said, “Sure we’re bastards.  But we’re fair bastards.” The best possible elevator pitch to describe Data General!

At a staff meeting it was mentioned that Apple promised to take all of their employees to Hawaii if they made their sales goal.  Apple was new, booming, and still a private company.  I blurted out “we should have a similar program. A really big sales goal — a real stretch.  If we make it we could charter a bunch of 747’s and take everyone to Hawaii.  We could drag banners behind the jets that say “THE FAIR BASTARDS.”  I thought this was pretty funny. Nobody else in the room laughed.  I guess I was starting to feel a little more secure to even come out with such blasphemy.

WSJ November 1975

WSJ November 1975

Fast forward a year later.  One day in a staff meeting there was talk about a DG competitor in New Jersey that was making knock-off Novas — Digital Computer Controls.  DG sued DCC the previous year for stealing their design.  Now we were going to buy the company and give its CEO really tough if not impossible sales goals, so that he would never get his earn-out.

WSJ January 1975

WSJ January 1975

This reminded me of an old story that had circulated the industry.  It was rumored a few years earlier that DG had burned down a west coast company because they were duplicating DG parts.  Keronix filed suit, accusing Data General, Ed de Castro, and Fred Adler of having their building set on fire and while they were at it wiretapping their phones.  I was vaguely aware of this caper but didn’t pay any attention — I was still at HP.  Keronix was unable to prove their allegation and eventually dropped the lawsuit. But maybe they did deserve to be torched.  Keronix had stupidly copied DG’s designs down to the last detail, including duplicating mistakes that were on the circuit boards.

32-bits Or Else

Ed wanted to do something in North Carolina.  He wanted to send a message to The Commonwealth of Massachusetts and our governor Mike Dukakis that the state’s attitude towards business sucked.  It was decided to move super secret FHP there.  After we secured a building I began bi-weekly trips to Research Triangle Park near Raleigh.  A pretty neat place, with the town of Chapel Hill nearby.  Except for being away from family I liked going down there.

I got to meet the famous Fred Brooks who taught at the nearby University of North Carolina.  I set up a meeting to find out if he would look in to FHP to help sort things out.  He had about 20 little clocks in his office — alarm clock size.  He was constantly punching them, stopping one and starting another.  “Fred, what are you doing with all these clock?”  Turns out he had become a fanatic regarding time management.  He wanted to spend his time in the most efficient manner, and at the end of each week he would look at the clocks and figure out how much time he spent on various thought processes and activities.

My meeting with Fred was just a year after he had published “The Mythical Man-Month,”  a book about his experiences as project manager of OS 360, the operating system for IBM’s iconic 360 line of computers. This was a hopelessly complex software project that included hundreds of programmers spread around the world.  Fred had some great one-liners, like “adding manpower to a late software project only makes it later.” Maybe he figured that by managing his time more carefully he would avoid repeating the mistakes of IBM’s largest ever software project.  In any case, after a brief description of FHP Fred said “No thanks.  Not the slightest bit of interest.  And Bill, good luck with that one….”

About a year after the move to North Carolina Ed put me in charge of all of FHP — the software and the hardware. I now owned the whole enchilada!  Carl Carman, my counterpart on the hardware side, was happy, even eager, to rid himself of any connection to the project.  FHP was still going nowhere and was hopelessly complex.  My excuse for still holding back and not working harder to kill it was that Ed had promoted me to VP.  Looks like I was willing to do anything to get that that title, including run a project that I didn't think had a chance of seeing the light of day.

Many smart engineers at headquarters saw what I saw — the odds of FHP ever getting finished were  close to zero. DG desperately needed a 32-bit computer, and all the eggs were in the FHP basket.  Until Tom West woke up.  Tom, the guy that my freaks beaned with a frisbee, was one of the sharpest people in the company.  He and Steve Wallach, another extremely smart guy, decided to do a 32-bit Eclipse.  In other words, lower the risk of getting to 32-bits by leveraging off the existing product line.

A writer named Tracy Kidder wrote a pretty good book about this project  — it won a Pulitzer Prize.   Tom West did the hardware part and my guys in Massachusetts did the software.  As is always the case, the software was the hardest part.  Morphing a 16-bit operating system and it’s programming languages over to 32-bits is very complex.

Now there were competing projects to get to 32-bits — the relatively low risk Eagle (code name for the 32-bit Eclipse) and FHP, the pipe dream.  Kidder’s book  (Soul of A New Machine) did a reasonable job of describing the process, but he got a couple of things wrong.   He didn’t give Tom and Steve and the project team nearly enough credit for saving DG’s buttocks.  Eagle was an underground project.  It was done in spite of management — it did not start as a sanctioned project.  If Tom and Steve hadn’t done Eagle DG would have been screwed royally.  Also, Kidder didn’t really mention how unusual DG was as a company.  He talked a little about the weirdness, but the reader might have thought all computer companies were like DG.  No way, it was in a class by itself.

DG had a really clever and aggressive PR machine. They were brash, in your face, cocky.  At the trade shows DG posters were everywhere.  Gimmicks like the microchip in the bellybutton got a lot of free publicity.  When my guys came out with a new Fortran compiler PR came up with an unusual way of touting it. This compiler produced really good code — very efficient.  But in churning out the machine instructions our Fortran compiler was very slow.  It was a real pig.  So, our PR guys came up with a full page ad with a pig right in your face. You couldn’t scan through Computerworld without stopping and reading ad. They did a lot of similar things — ad’s that you just couldn't ignore.

The best ad that DG unfortunately never ran had to do with Big Blue.  In 1977 IBM finally decided to get into the minicomputer business with the Series/1.  This market was growing too fast for them to ignore any longer.  Our PR guys came up with a great full-page ad that was supposed to run in the Wall Street Journal:

SOME SAY IBM’S ENTRY LEGITIMIZES THE

MINICOMPUTER INDUSTRY

THE BASTARDS SAY WELCOME!

Data General

It was perfect!! Really characterized Data General — the bastards, no fear.  But sadly it was never run.  I suppose it was Ed that pulled the plug on it.  Too bad, the ad would have generated all kinds of additional free publicity.

Tandem:  Killing Me Softly

By the spring of 1978 I was in a real funk.  I loved my job but I was restless.  Tandem was riding high — they were the talk of the computing world.  I was totally jealous of my old friends.  These guys were now rich and famous!!  If I had tried harder in 1974 I could have been a founder of Tandem!!  I screwed up royally!!

Ok, so what do I do?  Start a computer company?? Is that even possible?  Well, my friends showed me that it was — I was as good as the Tandem guys and look what they had achieved! And look at DG — a highly successful company run by some smart guys with quirky ideas that broke all the rules.   Really, if the DG and Tandem folks could do it, why not me?

I am a note taker.  And a note saver.  I still have notes from staff meetings at HP in 1972!!  The notes for “Nimbus” begin in March of 1978.  (For some reason even back then I glommed on to the idea of naming my company after a type of cloud.)

I would need money, and an idea.  The idea department was pretty empty back then.  I would start a company to go after DEC, HP, DG, Prime, and Tandem.  It would be a “low cost, 32-bit, virtual memory machine with a fast commercial instruction set.”  Pretty weak.  Nothing really new.  No breakthroughs.  So, most of the notes focus on staffing, schedules, and financing.

I knew this whole idea was pretty lame from the get-go.  But by June of ’78 my focus began to  change.  I would go after Tandem.  Strictly Tandem and the market they created — “non-stop” computers.  The only improvements over Tandem that I could come up with were: “less application work to provide non-stop”, and “repair the system on the fly.”  Plus, I now was a considering a 48 bit word.  But none of that was very exciting.  There still wasn’t the Technical Contribution -- the big idea that Dave Packard constantly preached about.

On the money side I got a list of the current venture capital companies, both East and West coast.  And I figured that some of my University of Santa Clara business school professors might have an idea for funding — I was going to look them up.

In June of 1978 I flew my beat-up ten year old Cessna down to New Jersey to attend the National Computer Conference in New York City.  I went right to the Tandem booth and got a nice demo from Dennis McEvoy.  Their operating system was basically HP’s MPE, their language was HP’s SPL.  I knew that stuff inside and out. This was the first time I touched their hardware -- the computer looked like a tank.  Very rugged.

DG was doing nothing about this new market even though Tandem was getting all the headlines and a lot of new business.  As far as I could tell HP, IBM, DEC, and all the others weren’t doing anything either.

This was back in the days before apps.  You didn’t download free or nearly free stuff from the app store.  Back then the customer wrote his own application software, or hired someone to do it.  These applications only ran on one type of a computer.  A DEC app would not run on DG or HP, etc.  One of the first to break this rule was Gene Amdahl.  His company made clones of the IBM 360 — his machines could run IBM software.  Naturally IBM hated this and did all they could to make Amdahl’s life a living hell.

I figured that for DG or anyone else to go after Tandem they would have to come up with a product that was incompatible with their current line.  The customer base would hate that and maybe jump ship — particularly DG’s customers who didn’t have a strong amount of loyalty.  So it was unlikely that Tandem’s first threat would come from an older company.  (I was wrong about this.  It turned out there was a way to make a non-stop computer that was ran old software.  But I hadn’t figured that out yet.)

We had a great family vacation planned for the summer of ’78.  We went to a little island in the Bahamas — Harbor Island.  Two weeks.  I was determined to spend much of that time thinking. My real problem, the thing that was ultimately holding me back, was Dave Packard.  His big booming voice constantly rattled around in my tiny brain: “YOU MUST MAKE A TECHNICAL CONTRIBUTION!!”

Oh crap, really?  Can’t I just try to improve on Tandem? Isn’t 32-bits or maybe 48 enough?

“NO!! IF YOU DON’T COME UP WITH SOMETHING REALLY DIFFERENT YOU WILL FAIL!!!”

Double crap.  What can I do that is new and different and better?

For two weeks, whenever I wasn’t playing with the family or doing other fun stuff, I thought,  churned,  mulled. When the vacation was over I had nothing.  Nada.  Zip-zero.  Big time depression set in.  It wasn’t going to happen.  Not in this lifetime.  I was never going to start company.  Tandem would continue to prosper and I would be stuck in this place, in charge of a project that was going to fail. I gave up.

Nothing much exciting happened until the following summer.  FHP continued to go nowhere, but surprisingly no one seemed to care.  Ed never put pressure on me — never said “Foster, get that thing on track or you’re fired!”  I’m really surprised he never leaned on me hard.  By the time I left DG the project was at least 4 years old -- I'm not sure exactly when it started.  Even at friendly old HP something would have happened by then.  At minimum the project manager would have been replaced, or the entire thing would have been canceled.  At DG nothing.  Nada.  Just keep pouring good money down a rat hole.  I have no idea when they finally pulled the plug on FHP -- I was long gone.

Eagle, the 32-bit Eclipse, was making good progress.  My software team did some amazing things to morph the old 16-bit software into something that would work.  Tom West and his boys were strutting around, naturally happy that they were saving DG’s butt.  Every time FHP missed another milestone West strutted a little higher.

Getting David Packard’s Monkey Of My Back

In June of ’79  I decided to try once again.  Try for an idea.  I need something, anything….  Ok,  Tandem is the only fault tolerant company.  They have a good product, but what is wrong with it?  What don’t people like about Tandem?  Maybe that’s how to attack the problem.

The most obvious problem was that their systems were hard to program.  Basically, a Tandem system consisted of two or more computers connected together with a high speed bus.  To achieve reliability the computers would “checkpoint” each other now and then.  Computer A would work on a problem for a while, then when instructed by the application program it would send information over to Computer B.  Likewise, B would send it’s stuff to A now and then.  That way if either computer failed the other would always know what it’s partner had been doing, and carry on.

It was a good idea.  But it was complex.  Their operating system, the morphed MPE taken from HP, needed extensive modifications to make checkpointing work.  But the worst thing was that the customer’s  application had to actually perform the checkpointing commands — the app had to be designed for a feature that only existed on Tandem.  These commands had really friendly sounding names, such as CHECKOPEN, CHECKMONITOR, CHECKSWITCH, GETSYNCHINFO, etc, etc.  It was a royal pain for a programmer to work all of those commands into the application.  And it meant that you couldn’t take an app from an IBM 360 or some other machine and move it over — it had to be extensively modified, or worse, rewritten.

I knew the trends.  Software was becoming a problem.  Software budgets were getting large.  Businesses were beginning to spend more on software than hardware.  A paradigm shift was happening.  Historically the cost of hardware had far outweighed software.  If I could come up with a scheme (“gimmick” as my friend John Couch would later call it) that eliminated all that extra software — well, that would be a breakthrough.

But how could you make a non-stop computer look like a normal one?  Ok!!  How about two computers?? Two computers running the same program???  After all, the cost of logic, the jelly beans that computers are made of, was dropping very fast.  Why not just have two computers work on the same problem at once?  It sounded wasteful, but it made things so simple.  Maybe this is the beginning of an idea…..

The principle of doing non-stop in hardware, which on the surface seems pretty obvious, was the basis for my Big Idea.  Tandem did their’s with software — my scheme would use hardware.  It was just a glimmer of an idea and needed a lot of work.  But it was a start.  This very simple but major difference finally got Dave Packard off my back.  Free at last!!!

Ideas were coming fast and furious.  I figured I better start writing things down.  I had a composition notebook that was always with me at DG as I went to meetings and such.  It was small and easy to carry around.  It allowed me to keep a record — so I that could look back and see how badly we were missing schedules, etc.  So, for the dream company I called  "Nimbus" I simply flipped this book upside down and backwards and starting writing on the back side of the pages.  The front side was Data General.  The back side was Nimbus.  DG, Nimbus. Old world, new world.  Old shit, new shit.  Boredom, excitement.   It was easy to flip from one world to the next.

This was actually a pretty dumb move considering that I worked for The Bastards, the toughest computer company ever.  If my notes were ever discovered they would string me up.  People would come to work one morning and there would be Bill, hanging by his neck on the flag pole, between Old Glory and the North Carolina state flag.

But never mind.  I’m a risk taker.  I ride motorcycles and fly airplanes.  As a teenager I climbed Half Dome and sat on the ledge at the top with my legs dangling over, 3000 feet straight down.  Dumb.  And later that summer my buddies and I climbed the Golden Gate Bridge, from the base of the tower on up.  Really, really dumb!!  If I could do that dumb stuff then I could take the risk of walking around DG with these notes.

On the way to work each morning for the next two weeks I began pulling over at a secluded spot in the town of Ashland — home of the electric clock.  (There are many iconic places where historic things happened in good old New England.)  I found a spot deep in the woods where I could park and no one would see me.  All the ideas that had come to me in the middle of the night I wrote down before I lost them.

Holy Grail

Holy Grail

I’m glad that I kept that notebook.  It is extremely revealing — my very early thoughts about something that would eventually become Stratus. The first entry was June 10, 1979.   The last was on June 24.  The most productive two weeks of my business life!  The first entry was a list 20 or so people that I would try to recruit as partners.  All from HP — I was going back to California.

The list included Bob Miyakusu, John Welch, Len Shar, Gary Smith, John Sell, John Couch, Phil Sakakihara,  Terry Opdendyk, and a few others.  All really great guys, people who I liked and respected immensely.  I would look them up as soon as I traveled west.  Little did I know then that only a few would have any interest.  By the summer of 1979 most of the best engineers had moved on — had left HP for Tandem or Apple or some other Silicon Valley startup.

On the next morning, on June 11, I pulled over to my spot in Ashland and wrote down in my little notebook the advantages of having two computers work on the same problem simultaneously:

1.  No application impact!!!

2.  Takes advantage of the most rapidly decreasing components:  memory, logic!

3.  Simplicity.

4.  Can make use of software not designed for fault tolerance!!

This last point was somewhat groundbreaking.  Doing non-stop with hardware solved one huge problem that prevented current companies from going after Tandem:  backwards compatibility.  If done right, you could strap any two machines together —  DEC, or HP, or (gulp) Data General.  But they would look like one computer to the software!  The non-stop offering would be totally compatible with the old product lines. Customers would not revolt!

For the next 12 days I worked on refining this concept.  Improving it.  Asking questions, such as how often do I compare the two computers?  Every minute or so?  Every few instructions?  Every instruction?  I wrote down “run them in lockstep” but since I was not a hardware guy I didn’t know how to do this, or even if it was possible.  I even considered strapping a couple of IBM System 38’s together.  This way Nimbus wouldn’t have to invent a bunch of new hardware or software.

The other big question was:  how do you know when a computer fails?  What if it doesn’t just crash?  What if it keeps running but just produces the wrong answer?  What if one says 2 plus 3 equals 5 and the other says it equals 6??  Who do you believe??  Well, how about a third computer to break the tie?

The concept of three computers voting was not new — it had been written about in various publications for years. The idea was simple:  have three computers work on the same problem, then vote on the answer.  If there’s a disagreement majority wins.  I had remembered a mention of voting in the Journal of the ACM. (Association for Computing Machinery, a geek organization I have belonged to for almost 50 years!)  I looked through some old issues and sure enough, there it was! The ACM referenced an IBM article going back to 1962 that described voting. They mentioned that the famous Hungarian mathematician John Von Neumann had describe it a couple of decades earlier.  I was also aware of a Silicon Valley startup called Magnuson that attempted a voting product but failed.  So this was all public domain stuff —  I was just re-hashing a very old concept.

I continued to toy with the idea, and started to figure out how many people this would take.  I had ten people for software, 6 for hardware, and 3 for documentation.  I would need 19 people for the first machine.

Notebook entry for June 21, 1979.  Idea # 7.  “Get Fortran, PL/1 from Freiburghouse.”

That would be Bob Freiburghouse, proprietor of his own small software company, Translation Systems.  At DG we licensed Bob’s PL/1 complier — that’s how I knew him.  This seemingly minor entry, almost a footnote, proved to be huge.  Throw out everything else in my notebook except do it in hardware and talk to Bob .  99% of the value of what I had done so far was in those two decisions.

My last entry was June 24, just two weeks after I started taking notes.  I had finally settled on  voting — that was the way to go.  Still, it seemed really expensive — maybe fault tolerance should just be an option? Maybe the customer could buy the extra processors only if non-stop was really, really important.

Looking back, I was wrong about voting.  Hardware was still too expensive in 1979 to justify three conventional computers working on the same problem, except perhaps for a very tiny and uninteresting market.  My Big Idea would have failed. But well after I left DG a technology breakthrough occurred — a powerful computer-on-a-chip became available from Motorola.  This made the voting viable.

Need a miracle to solve voter redundancy problem

Need a miracle to solve voter redundancy problem

I did have one annoying little note on this last page:  “no redundancy” with an arrow pointing to the voter. This plan had one big flaw:  what if the voter fails?  The voter was an obvious weak link.  The whole idea of a non-stop computer is that no single component failure could bring the entire system down. How about two voters?  But what if the voters disagree — who do you believe?    IBM had a solution in their 1962 paper but it was hopelessly complex and extremely expensive. Ever see the cartoon of the mad scientist at the blackboard?  A very complex equation, and he is asked to explain the little note in the middle:  “And then a miracle occurs.”  That was my thought — don’t worry about it now, somehow a miracle will occur.

Finally I was excited about something.  But wait!  I’ve got that DG employment agreement — every synapse in my brain belongs to them — or at least that was their claim.  I knew there was no invention here — that Data General could never claim they owned the idea.  But the only way to move ahead with a clear conscience was to quit and strike out on my own.  I decided to put everything on hold until I was free and clear of DG.  Hmm…  Three small kids.  Limited resources.  No outside source of income.  Maybe I should slow down, think about this for a little bit.  One thing for sure — no more work on Nimbus.  Once again, everything ground to a halt.  Chicken.

A few days later I was sent to Europe to talk to our people about 32-bits and what we were doing about it.  Everyone knew we were late to the market and employees were becoming concerned, so it was decided that we should open up a little about our plans.  In the middle of my first night in England I suddenly popped awake.  There was a voice in my head. “Just do it.  Go home, quit DG, and just try, for crying out loud!  You have the beginnings of an idea.  You’re excited about it.  JUST DO IT!!!”

An ugly little red devil had appeared on my shoulder. “COWARD.  DO IT!!”

And then an angel popped up on the other side.  “It’s too risky.  You have a wife and three little kids to worry about.  How could you think of leaving a great job?” (sob, sob)

“OH SHUT UP!! HE’S GOT NOTHING TO LOSE.  THE WORST THING THAT HAPPENS IS HE FAILS!  SO WHAT?  JUST DO IT, CHICKEN!!”

The devil won out.  I waited for morning to arrive in Boston and called Marian.  When I told her I was going to quit start a computer company she said “Ok, whatever, see you later.”  She wasn’t too excited —  she had heard this from me before, with no action.

Miss Bottom Line 1979

Miss Bottom Line 1979

Everything happened quickly after that.  I got home, typed up my resignation letter.  Marian asked how long we could live on our savings.   About a year.  “Ok, I’ll keep track of the cash and when we’re a month away from being broke I’ll tell you to get a job.”  No complaints — just the facts.  Marian was and is a bottom line person.  “The bottom line is: when we’re almost broke you go get a job.  Up until then I am totally with you.”  Great!  We’re about to spend our life savings, all the money we had for the kid’s college fund — and she never complained.  Total support from day one!  Could you imagine how impossible this would have been if she had said, “OMG we have three small kids and you’re going to waste our life savings!  How could you consider doing this?!!”

Next morning I walked into Ed’s office.  “Ed, I’m resigning — here’s my letter giving two weeks notice.”

“Why?”

“I’m going to start a computer company.”

“Where?”

“Back home.  The West Coast.  I’m going back to California.”

“All right.”

That was it.  He was friendly, almost supportive.  He didn't kick me out like Hewlett Packard had done!!  Herb was right -- DG is fair!!!

Electronic News 9/3/79

Electronic News 9/3/79

I went back to my office and sat.  I didn’t tell anybody.  I figured Ed would.  A week went by.  Nothing.  Ten days, nothing.  Nobody seemed to know.  It was business as usual.  On my final day I waited for Ed to arrive and asked what the plan was.  “Steve Gaal is replacing you.  Good luck.”  Steve was a really good guy — this was a great choice.  But more importantly, Ed wished me good luck!!  Fantastic!  Maybe he’ll leave me alone…

Well, he didn’t.  The first lawsuit threat came just a few months later.  The second was just after we closed our initial financing.  The final threat came in 1981.

Call Me Flounder

I was totally down in the dumps in the fall of ’79.  It had been three months since I left DG and everything was going wrong.  I had bombed in California.  I had no partners.  I was a total failure at raising money — none of the venture people said no, but nobody said yes.  They were stringing me along and Marian and I were closely watching our remaining cash.  I avoided everyone — friends, family.  I didn’t want people knowing how bad things were.

Fred Adler was the initial investor in Data General.  He was responsible for raising $800k in two segments — that’s all they needed to get to a public offering.  I had met Fred a few times while working for Ed. Apparently he was keeping tabs on my failing effort.  I got a call out of the blue.  “Bill, I understand you’re having trouble raising money.  Send me a copy of your business plan and if it looks good I might invest.”

Once again the ugly devil popped up on my shoulder. “Don’t do it!!!  He can’t be trusted!!”

The cute little angel appeared on the other side and meekly said,   “But you’re not getting anywhere.  You’re eating through all of your savings.  What about your poor little wife and kids?” (sob, sob)

“It doesn’t matter.  DON’T DO IT!!!!”

I tend trust people.  That’s where I start.  But sometimes it backfires.  Remember that classic scene in Animal House when the frat-brats trashed Flounder’s car?  “Flounder, don't look so depressed.  You f...ed up. YOU TRUSTED US!!!” Just call me Flounder.  I knew it was dangerous to send Adler my plan, but I was desperate.  The angel won out.   I mailed my plan off to his office in New York.  I screwed up.

Several weeks later I was chopping wood in our back yard.  We had a ton of oak trees and my plan was save some cash and heat our house that winter with wood.  Our daughter Robin called out, “Dad, phone call.  His name is Mr. Kaplan.”   Wow!  Carl Kaplan.  One of Adler’s lawyers.  I had met him a few times in New York and liked him — he seemed like a really good guy, a real straight arrow.  My heart jumped.  This might be it!!  They’re going to invest!  My dream might actually come true!

It didn’t take long for my hopes to be crushed.

“Hi Carl, how’s it going?”

He got right to the point.  “Bill, thanks for the business plan.  Looks like a great idea.  But don’t get too excited.  We’re going to sue you.”

“WHAT!!!!”

“We’re going to sue you for Breach of Fiduciary Responsibility.”

Wow!  Talk about going from a high to a low in about a nanosecond!!  I was totally bummed.

“What in the hell is this fiduciary crap?”

“We’re going to say that you’re supporting yourself by selling names of employees to headhunters.  You’re selling inside information.”

“But it’s a lie — total bullshit.  And it’s absurd — no one will believe you.”

“That doesn't matter.  Just suing you will scare off all the venture guys.  So give it up — unless you want a lawsuit on your hands.  You’re never going to get any money.”

After Kaplan hung up big time depression set in.  I figured this was just about the worst day in my life.  But as happens often, it was a turning point.  For the positive.  The shackles were off.  I had nothing to lose.  This threat gave me the freedom to move ahead.

These guys really screwed up!  They had my plan and did nothing with it except threaten me.  They had the plan for a company to go after Tandem — the hottest and most exciting computer company in 1979. They had a chance to invest and make a lot of money but instead they chose to intimidate.

The next threat came on May 12, 1980 one week to the day after we put $1.7 million in the bank.  (I know the exact date from my trusty notebook.)   Our lead investor Joe Gal got a call from Fred Adler informing him that once again they were about to sue.  We had only enjoyed our newfound fortune for one measly week and DG was threatening me for a second time!!   Joe suggested I call Ed to try to work something out.  On May 14 I called Ed.  He was cordial but insisted that as an ex-officer of DG I was using proprietary information — the names of employees.  At that time we had hired only 4 people from his company.  I told Ed I didn’t know any of them — Gardner Hendrie had come up with the names.  He should sue Gardner.  (Sorry for throwing you under the bus, Gardner….)

I then said that we had made offers to 2 or 3 others and I didn’t know yet if they had accepted, but after them we would stop, for a year.  Ed agreed that for one year he would back off.  Then he mentioned that Adler had been wanting to sue me for months, but Ed had held him back.  I believed him.  I never thought throwing around lawsuits was in Ed’s DNA.  He could achieve all the success he wanted just by using his brain.

The final threat came one year later, again for hiring just a couple of their people.  From that point on they left us alone.  While Ed never did sue me or Stratus, it was unfortunate that all of this nonsense did prevent a bunch of good people from joining us.  These folks would have been in on the ground floor of an exciting startup but were held back by their employer.

In 1981 Harvard wrote a case study about Stratus — the trials and tribulations of a would-be entrepreneur who didn’t have a clue about how to raise money.  I lied to the case writer  — told him that I had no clue about who I was going to compete with until after I left DG.  I was still afraid of them, even in 1981.  For the next 10 years when the case was taught at Harvard and later at Stanford I would start by explaining to the class that in fact I was not totally brain-dead — I had decided before leaving my job that I was going after Tandem.

When my father died years ago I was going through his things and found a letter from me that he had saved.  It was dated May 30, 1980 — three weeks after we got our first round of money.  I had no idea yet whether or not we would ever be successful.  Here’s a quote:  “There is still a chance that DG will sue us. They know they can’t win.  Their objective would be to make life difficult for us by wasting time and money with lawyers.  It’s a ridiculous way to run a company, and in the long run I think their policies will backfire.”

Did their policies backfire?  You bet they did!  Before the first threat I had no intention of going after Data General people.  In fact, I was doing all I could to stay away from them.  I tried getting something going in California — as far away from DG as possible.  When that failed I had no choice but to stay in New England. Still I avoided DG people.  The company scared me.  I would have to find engineers from DEC or Prime, but there was one giant problem:  Other than my neighbor who ran sales for Prime Computer I didn’t know a single soul at either company.  I was pretty much at the end of my rope.

Then came the first threat — the ridiculous trumped-up charge that I was selling names to headhunters.  Here I was, with no money and no partners.  What were they afraid of?  Why did they even bother with me?  During the fall of 1979 when that threat came I was dead in the water.

Carl Kaplan’s call only inspired me.  I had nothing to lose!!  The gloves were off!  We still had six months of cash left — I wasn’t close to giving up.  I eventually called Gardner Hendrie, one of DG’s best hardware people.  And as they say, “the rest is history”.  Gardner became a co-founder and was a critical link in our eventual success.  We got our first round of financing because of his contacts.  And, most importantly, he and his team came up with a fantastically elegant but simple solution to the non-stop voter problem.  If Adler had just left well enough alone I never would have gone after Gardner, and Stratus never would have happened.

My three years at DG were life-changing.  The move east got me out of HP and taught me about the real world.  In doing so I joined one of the most unusual companies ever.  I learned a lot and became tough(er). They made me a survivor.  They forced me to think about profit and cash flow — things that were never emphasized at HP.

I valued the lessons from DG.  I told myself that if I ever got a company going I would try to take the best from both HP and DG.  HP’s attitude toward people — trust them,  treat them with respect.  Emphasize quality in everything.  And do whatever it takes to keep customers happy.

And from DG it was their emphasis on profits and cash.  Doing what it takes to survive.  In a startup cash is king. The chance of survival is small.  DG taught me to watch my bottom line.

Many of the original minicomputer companies are now a part of Hewlett Packard, partly because of HP’s own acquisitions and later from their merger with Compaq.  What is left of Tandem is buried somewhere deep in the bowels of HP.  DEC, the first minicomputer company, is deep within those same bowels.  Data General was acquired by EMC which is about to be acquired by Dell.  All the original minicomputer companies are gone -- except, that is, for Stratus, which still survives as an independent privately held company.

I’ve been asked what I think about Ed — the threats and the efforts to kill my venture.  I’ve always liked Ed.  I liked working for him.  I like him as a person.  We’ve enjoyed some good times together long after we both left Data General.  He is a great guy and has done many good things for others.  What about when I was being harassed?  I wasn’t too fond of Ed then.  But I can't imagine Stratus ever existing if Ed had not brought me to Massachusetts.  How could I ever hold a grudge against him?  After all, I'm describing things that happened nearly 4 decades ago!  It was always just business with Ed —none of the threats were ever personal.  He was taught somewhere that harassing now and then was the way to run a company.  He was given some bad advice.

Would I have run Data General differently?  Sure.  Would the company have had a different culture? Absolutely! Would DG have been more successful?  Who knows?  Probably not.  I think working hard for customer loyalty and showing trust towards people are simple common sense attributes of any company.  But during his time maybe Ed’s style was needed for survival. Computer companies, both big and small,  were dropping like flies during the 70's and 80's, but under Ed’s leadership Data General thrived.  Who am I to judge?  Data General, the company that Ed founded, was and is an iconic force in computer history. .

During the late ’80’s DG began to fall on hard times.  Ed left the company — I don’t know if he was forced out or quit, but I imagine the board wanted him to go.  You have to change leadership to have any chance of real change in a company.

They brought in a fellow with a finance background named Ron Skates.  I didn’t know Ron well but I did run into him now and then at various events around Boston.  One time he pulled me aside.  “Bill, I’ve been running DG for three years now and it is so frustrating.  Ed is long gone but I can’t change the culture.  No matter how hard I try we're still a bunch of bastards.”

A company begins to develop its personality on day one.  The early people set the company’s style, nurture its culture.  But once there is some momentum the company’s personality cannot be changed, even as the people come and go.  The founders better get it right from the start.

Are Zohran Mamdani and Katie Wilson Democratic Socialists or FDR Democrats?

Portside
portside.org
2025-11-22 20:09:14
Are Zohran Mamdani and Katie Wilson Democratic Socialists or FDR Democrats? Dave Sat, 11/22/2025 - 15:09 ...
Original Article

Between now and next year’s midterm elections, the “S” word, and even the “C” word, are going to get a workout. President Trump and his allies have called New York’s socialist mayor-elect Zohran Mamdani a Communist, a Marxist, a terrorist, and even a jihadist. They’re warning that the U.S. is experiencing a wave of “socialism,” a term that they hope still carries its hoary Cold War connotations. They hope to make Mamdani the face of the Democratic Party, a tactic intended to discredit its candidates in swing races.

During the Red Scare hysteria of the 1950s, American socialism fell on hard times. Few Americans distinguished between the European social-welfare systems and the communism of the Soviet Union or China. Across the nation, universities, labor unions, public schools, movie studios and other major institutions purged themselves of their left-wingers. Even many liberals were afraid to speak out for fear of being called a Communist and losing their jobs.

Through the Obama administration, the use of the term “socialism” as a kind of political epithet was on full display, with the president’s opponents — the Republican Party, the Tea Party, the right-wing blogosphere, the Chamber of Commerce, and conservative media figures such as Glenn Beck, Ann Coulter, Sean Hannity, and Rush Limbaugh — labeling anything he proposed, including his modest health-care reform law, “socialism.”

Over the last decade, however, something has started to shift. Republicans have not stopped red-baiting, and they will continue to shout “socialism” as they attempt to defeat Democrats next year and in 2028. But the political climate has dramatically changed. Americans, particularly those under 50, are more open to candidates who call themselves socialists, so long as they have practical ideas for solving their problems. They are reassessing their understanding of socialism, and its place within American identity.

This is fitting: For more than a century, socialism has been integral to American progressivism, championing early many of the reforms that would eventually come into vogue on the center left.

We are seeing that dynamic play out again today.

From the margins to the mainstream

Mamdani is a member of Democratic Socialists of America, but most of the one million New Yorkers who voted for him would likely not describe themselves as “socialists.” The same is true of those who voted for Katie Wilson, the socialist community organizer who will become Seattle’s new mayor, or for the other dozen socialists who were elected to office for the first time in November. There are few self-identified socialists among the voters who supported the more than 250 people now serving in office who are DSA members, were endorsed by local DSA chapters, or, like Sen. Bernie Sanders of Vermont, call themselves democratic socialists but have never joined the organization.

There are now at least 135 DSAers and DSA-affiliated city council members, 64 state legislators, 21 school board members, 6 mayors, and three members of Congress. In November, voters in Atlanta; Detroit; Tucson; Greenbelt, Maryland; Troy and Poughkeepsie, New York; Hamden, New London and New Britain, Connecticut; and Amherst, Massachusetts elected democratic socialists on their city councils. Minneapolis added one new democratic socialist to its city council, bringing the total to five. Ithaca, New York, added two, bringing the total to three. Five DSAers serve on the Chicago city council and four serve on its counterpart in Portland, Oregon. In Los Angeles, four of the 15 city council members are affiliated with DSA, and two others are currently running for council seats that will be decided next year. Voters have elected eight DSAers to the New York state legislature and three to Pennsylvania’s. Last year, voters in Eau Claire, Wisconsin elected socialist Christian Phelps, a freelance journalist and organizer for Wisconsin Public Education Network, to the state assembly, replacing a Republican.

To put that all in perspective, we haven’t seen so many socialist office holders since, roughly, 1912. That year, Eugene V. Debs — the Socialist Party’s presidential candidate — won more than 900,000 votes, 6 percent of the total (and fewer than the votes Mamdani got for mayor). Debs might have garnered even more, but two other candidates — Democrat Woodrow Wilson and Progressive Party candidate (and former president) Theodore Roosevelt — stole some of the Socialists’ thunder, winning the support of workers, women and consumers with promises of such progressive reforms as women’s suffrage, child labor laws and workers’ right to organize unions — policies the socialists helped to mainstream.

Debs lost, but that year 1,200 members of the Socialist Party held public office, from school boards to Congress, including 79 mayors in cities such as Milwaukee, Buffalo, Minneapolis, Reading and Schenectady. In office, they pushed for the expansion of parks, libraries, playgrounds and other services, including public ownership of utilities and transportation facilities, free meals for poor schoolchildren, a living wage for workers, and a friendlier attitude toward unions, especially during strikes.

This has been the pattern with American socialists. For more than a century, their role has been to move so-called “radical” ideas — proposals to make society more humane, more livable, and more fair, and to give everyday people a stronger voice in their democracy and workplaces — from the margins to the mainstream.

In 1916, Congressman Victor Berger, a Milwaukee socialist, sponsored the first bill to create “old age pensions.” The bill didn’t get very far, but two decades later, in the midst of the Depression, President Franklin D. Roosevelt persuaded Congress to enact Social Security. Conservative critics and big business groups denounced it as un-American, even Communistic. But today, most Americans, even conservatives, believe that Social Security is a good idea. What had once seemed radical has become common sense. In fact, much of FDR’s other New Deal legislation — the minimum wage, workers’ right to form unions and public works programs to create jobs for the unemployed — was first espoused by American socialists.

A crisis of capitalism

The DSA was founded in 1982, as the Cold War waned, after a merger of two small leftist organizations. Before Sanders ran for president in 2016, its membership hovered around 8,000. Within a year after Sanders’ first presidential campaign and then Rep. Alexandria Ocasio Cortez’ victory in 2018, the number jumped to about 32,000. Mamdani’s campaign inspired a new wave of membership, which now exceeds 80,000 nationwide. The median age has declined from 68 in 2017 to about 33 today, according to DSA.
DSA members, however, represent just a tiny sliver of the Americans who offer support for socialism at the ballot box. According to a recent Gallup poll, 39% of Americans (including 66% of Democrats) have a positive attitude toward socialism. Moreover, the proportion with a favorable view of capitalism has plunged from 60% in 2021 to 54% today.

But even that figure underestimates that potential appeal of socialist-like ideas among Americans. Polls show that a vast majority of Americans share progressive views, even if they don’t describe themselves as socialists. For example:

  • 71% consider wealth inequality a serious national issue.
  • 69% of Americans think the American economy is rigged to advantage the rich and powerful.
  • 82% view the influence of money in politics as a threat to American democracy.
  • 71% of Americans (including 53% of Republicans) think that billionaires aren’t taxed enough.
  • 63% of Americans — including 43% of Republicans — say tax rates on large businesses and corporations should be raised.
  • 82% say the cost of prescription drugs is unreasonable, and say profits made by pharmaceutical companies are a “major factor” in the high price of prescription drugs. Surprisingly, 89% of Republicans share this view, compared with 78% of independents and 84% of Democrats.
  • 62% of Americans — including 90% of Democrats, 65% of independents, and 32% of Republicans — think it is the responsibility of the federal government to make sure all Americans have health care coverage.
  • 59% support a single-payer or Medicare for All system (27% oppose the idea and 14% had no opinion).
  • 68% of Americans support labor unions, a significant increase since the 1960s.
  • 83% support raising the federal minimum wage from the current $7.25 an hour to $12, while 64% of voters (including 45% of Republicans) think it should be increased to $17 an hour.
  • 82% (including 76% of Republicans) support a federal paid family and medical leave program.
  • 73% of Americans support government-funded universal child care.
  • These views align with the positions of DSA office-holders and other progressive officials. In fact, nothing about Mamdani’s agenda for New York, or Wilson’s for Seattle, is particularly radical. For example, many cities around the world, and quite a few in the United States, provide free public transit. Hundreds of cities already have some form of rent control, including New York. Government-run grocery stores — a Mamdani proposal on which right-wing voices fixated — are successful around the world, operated by governments left, right and center, and, in fact, already exist in New York .
  • If Mamdani, Wilson, and other democratic socialists have any model at all, it is not the authoritarian governments of Russia, China or Cuba, but the social democracies of Scandinavia. These countries have greater equality and a higher standard of living for working families, stronger unions, and a much wider social safety net. They also have free (or almost free) universities, universal health insurance, and lower poverty.
  • Like social democrats and democratic socialists, most Americans agree that private businesses should be subject to rules that require them to act responsibly. Banks shouldn’t engage in reckless predatory lending. Energy corporations shouldn’t endanger the planet and public health by emitting too much pollution. Companies should be required to guarantee that consumer products (like cars and toys) are safe and that companies pay decent wages and provide safe workplaces.
  • They want to reduce the political influence of the super rich and big corporations, increase taxes on the wealthy to help pay for expanded public services like child care, public transit, and higher education, and reduce barriers to voting. They support a higher minimum wage, paid sick days and paid vacations, safer workplaces, and unions. Socialists emphasize government enterprise, but even most Americans favor government-run police departments, fire departments, national parks, municipally-owned utilities, local transit systems, and public state universities and community colleges.
  • It was Mamdani’s and Wilson’s focus on affordability and democracy that catapulted them into the winner’s circle. The lesson for Democrats is not that all or even most of their candidates should run as democratic socialists, but that they should embrace policy ideas that help Americans who are worried about making ends meet, concerned about the widening wealth and income divide, and upset that our democracy is being gutted by billionaires. Indeed, the arrogance and corruption of Trump and Elon Musk — and their obvious indifference and cruelty toward Americans just trying to get by — were a major factor in recruiting Americans to vote for Mamdani, Wilson, and other socialist candidates, as well as Buffalo’s newly-elected progressive mayor Sean Ryan and more mainstream Democrats like newly-elected governors Mikie Sherrill in New Jersey and Abigail Spanberger in Virginia.

A very American tradition

Throughout American history, some of the nation’s most influential activists and thinkers, such as Debs, John Dewey, Helen Keller, W.E.B. DuBois, Albert Einstein, A. Philip Randolph, Walter Reuther, Gloria Steinem, and Martin Luther King Jr. embraced democratic socialism.
King believed that America needed a “radical redistribution of economic and political power.” He told his staff, “maybe America must move toward a democratic socialism.”

Conservative Americans love to recite the “Pledge of Allegiance” and sing “America the Beautiful. Few probably know that Francis Bellamy, a socialist Baptist minister, wrote “The Pledge” and that a socialist poet, Katherine Lee Bates, penned “America the Beautiful” in the 1890s.

In the early 1900s, socialists led the movements for women’s suffrage, child labor laws, consumer protection laws and the progressive income tax. Socialists were in the forefront of the civil rights movement from the founding of the NAACP in 1909 through the Voting Rights Act of 1965. Socialists have long pushed for a universal health insurance plan, which helped create the momentum for stepping-stone measures such as Medicare and Medicaid in the 1960s and Obamacare today.

Mamdani, Wilson, and other elected socialists are well-acquainted with the history of American socialism, including the opportunities and pitfalls they faced when governing cities.

In the early 1900s, Milwaukee was the center of American socialism. Dominated by the brewery industry, the city was home to many Polish, German, and other immigrant workers who made up the movement’s rank and file. In 1910 Milwaukee voters elected Emil Seidel, a former pattern maker, as their mayor, gave socialists a majority of seats on the city council and the county board and selected socialists for the school board and as city treasurer, city attorney, comptroller and two civil judgeships.

In office, the socialists expanded Milwaukee’s parks and library system and improved the public schools. They granted municipal employees an eight-hour day. They adopted tough factory and building regulations. They reined in police brutality against striking workers and improved working conditions for rank-and-file cops. Milwaukee’s socialists improved the harbor, built municipal housing and sponsored public markets. The socialists had their own local newspaper and sponsored carnivals, picnics, singing societies and even Sunday schools. Under pressure from city officials, the local railway and electricity companies — which operated with municipal licenses — reduced their rates.

Grateful for these programs, voters kept socialists in office. They elected Daniel Hoan as mayor from 1916 to 1940. In those years, Milwaukee was frequently cited for its clean, efficient management practices, and its leaders boastfully called themselves “sewer socialists.”

In April 1936, Time Magazine put Hoan on its cover. The story called Hoan a “reform mayor” who “represents a party which has only some 3,000 paid-up members in a city of 578,000 people. The city’s bankers, utilities and big real estate owners are his “his sworn enemies,” Time noted, and the local press is “solidly against him.” Despite this, Milwaukee voters kept Hoan in office for 24 years. The reason, Time explained, is that Hoan “remains one of the nation’s ablest public servants, and under him Milwaukee has become perhaps the best-governed city in the U.S.”

New York City, too, has a history of adopting policies labeled as “socialist.” Fiorello La Guardia, during his three terms as New York’s mayor (1933–45), was in particular denounced as a leftist by his critics. Though he ran an honest, efficient, and progressive administration that helped lift the spirit and improve the conditions of New York’s polyglot working class, business groups constantly attacked him as an impractical radical. When La Guardia wanted the city to purchase snow-removal equipment in advance of winter storms, Comptroller Charles Craig said it was “the wildest kind of radical, socialistic” idea. La Guardia once told the New York Times, “When anyone raises a question about the existing order, he is called either a reformer or a radical. It has been my lot to be called the latter. Why? Only because I have consistently objected to things which I believe unjust and dangerous.”

Socialists and the Democratic Party

In 1932, in the depths of the Depression, Norman Thomas, a Protestant minister, ran for president on a Socialist Party platform that called for old-age pensions, public works projects, a more progressive income tax, unemployment insurance, relief for farmers, subsidized housing for working families, a shorter work week and the nationalization of banks and basic industries. Thomas figured that in such desperate times, his message would appeal to voters. But many voters who may have agreed with Thomas’ views did not want to “waste” their vote on a socialist who had no chance to win and who might even take enough votes away from the Democratic candidate, Franklin D. Roosevelt, to keep Republican Herbert Hoover in office. Thomas did not expect to win, but he was disappointed that while FDR garnered 22.8 million votes (57 percent), he had to settle for 884,781 (2 percent). When friends expressed delight that FDR was carrying out some of the Socialist platform, Thomas responded that it was being carried out “on a stretcher.” He viewed the New Deal as patching, rather than fixing, a broken system.

Following the success of his popular muckraking book, “The Jungle,” about the horrors of Chicago’s slums and factories, journalist Upton Sinclair moved to California and ran on the Socialist Party ticket for the House of Representatives (1920), the U.S. Senate (1922) and California governor (1926 and 1930), winning few votes. In 1934, Sinclair figured he might have more influence running for office as a Democrat. He wrote a 64-page pamphlet outlining his economic plan — “I, Governor of California and How I Ended Poverty” — and entered the California Democratic gubernatorial primary.

Much to Sinclair’s surprise, his pamphlet became a bestseller across California. His campaign turned into a popular grassroots movement. Thousands of people volunteered for his campaign, organizing End Poverty in California (EPIC) clubs across the state. The campaign’s weekly newspaper, the EPIC News, reached a circulation of nearly one million by primary day in August 1934. The campaign allowed Sinclair to present his socialist ideas as common-sense solutions to California’s harsh economic conditions.

Sinclair shocked California’s political establishment (and himself) by winning the Democratic primary. Fearing a Sinclair victory, California’s powerful business groups joined forces and mobilized an expensive and effective dirty-tricks campaign against him. On Election Day, Sinclair got 37 percent of the vote — twice the total for any Democrat in the state’s history.

Sinclair did not win, but his ideas pushed the New Deal to the left. After the Democrats won a landslide midterm election in Congress that year, FDR launched the so-called Second New Deal, including Social Security, major public works programs and the National Labor Relations Act, which gave workers the right to unionize.

The Sinclair campaign taught socialists a valuable lesson: Rather than run candidates on a third-party ticket, which would make them a fringe political movement, they would not only continue to participate in the key movements for a more robust politics and humane country, but also work within the Democratic Party to push it to embrace more radical ideas.

The new Gilded Age

As the Democratic Party finds its way, largely out of power amid Trump’s authoritarian onslaught, oligarchs’ looting of government and a precarious, worsening economy, New York voters welcomed Mamdani’s laser-like focus on making the city more affordable, including free buses, universal pre-K and child care, city-owned grocery stores, and rent freezes.

New Yorkers also embraced his attacks on the city’s super-wealthy, including the outsized influence of the Wall Street and real estate industries. Voters favored Mamdani’s plan to increase taxes on the city’s super-rich to help pay for improving and expanding public services. This includes an additional tax bracket for New York City residents with income over $1 million, which would be taxed at 5.9%, and raising the state’s highest corporate tax rate from 7.25% to 11.5%.

Mamdani frightened many of the city’s, and the nation’s, billionaires, who were used to getting their way in politics. Within the last few weeks before the November 4 election, a few dozen billionaires donated over $8 million to political action groups opposed to Mamdani or in support of his chief rival, ex-governor Andrew Cuomo. That last-minute war chest had little impact on the outcome. Mamdani (who beat Cuomo in the Democratic primary in June) garnered over half the vote in a three-way contest with Cuomo (running as an independent) and Republican candidate Curtis Sliwa.

At the root of Americans’ growing criticism of capitalism and support for socialist-like policies is the widening disparity of wealth and income. The country’s 1,135 billionaires are today worth $5.7 trillion, more than the combined wealth of the bottom half of U.S. households. Elon Musk, Jeff Bezos, and Mark Zuckerberg alone accounted for nearly $1 trillion in wealth. In 1965, CEOs were paid 21 times as much as the typical worker; it has since skyrocketed to 285 times. This year Starbucks is set to make over $25 billion in profits. Brian Niccol, its CEO, made $95.8 million last year, while the typical barista earned less than $15,000. Starbuck’s CEO-to-median-worker pay ratio is 6,666-to-1.

More than one-third of Americans say they can’t afford an unexpected expense of $400, with close to half saying they have less in emergency savings than they did a year ago. Rents are skyrocketing, making it difficult to save to buy a home. The median age of new homebuyers is now 40 — the highest it has ever been. Meanwhile, in this new Gilded Age, billionaires like Trump and Musk flaunt their wealth, engage in the corrupt practices of crony capitalism, and reveal their disdain for democracy and ordinary Americans. It is not lost on voters that these are facts of the sort that America’s most well-known democratic socialist politician, Bernie Sanders, injected into the electoral conversation — messaging that has helped socialism’s rehabilitation after decades of red-baiting.

Mamdani’s come-from-far-behind victory was made possible by an army of over 100,000 volunteers. Some were veterans of earlier electoral fights; some had been active in union, community organizing, and other battles. For others, the Mamdani crusade was their first political involvement. The same was true of the more than five million Americans who participated in October’s “No Kings” marches and rallies across the country, the largest one-day protest in the nation’s history, many of whom are now part of the resistance to Trump’s ICE and Border Patrol invasions of U.S. cities. The key organizers of these efforts are progressives and leftists, but most of the participants are simply people who are fed up with economic hard times, fading dreams, and anger at the arrogance of wealthy plutocrats.

These dynamics have helped put socialists into office, but winning office is just a first step. For those who have had to actually govern cities, Mamdani has perhaps the hardest task. He must run an administration that focuses on civic housekeeping — getting the trash collected, fixing the potholes, making sure the parks and playgrounds are maintained, and guaranteeing that the schools and hospitals have the resources they need and the police respond quickly to 911 calls. To make that happen, he has to work with Gov. Kathy Hochul and the state legislature, because the city lacks the authority and money to adopt many of his ideas, which the real estate and finance lobby groups will try to obstruct. Can the state’s liberals and progressives, including DSA, push Hochul and state legislators to embrace Mamdani’s agenda? For the next three years, he’ll also have to deal with a hostile president, determined to undermine cities run by Democrats. If the Democrats take back the House next year, it could help to blunt many of Trump’s policies — including attacks on the things that many New Yorkers and Americans rely on, such as health insurance, food stamps and rent subsidies. If a Democrat wins the White House in 2028, New York, and Seattle, could serve as laboratories for a progressive agenda on jobs, environment, and housing.

In his November 4 victory speech, Mamdani said: “The sun may have set over our city this evening, but as Eugene Debs once said: ‘I can see the dawn of a better day for humanity.’” If they want his administration to succeed, Mamdani’s supporters need to recognize the constraints he’s facing, give him leeway to forge compromises to get things done, and mobilize when business lobbyists try to block his proposals in the legislature. A better day is possible, but it won’t happen overnight — in New York, Seattle or across the country.

Peter Dreier is professor of politics and urban policy at Occidental College. His books include "Baseball Rebels: The Players, People, and Social Movements That Shook Up the Game and Changed America," "We Own the Future: Democratic Socialism, American Style," "The 100 Greatest Americans of the 20th Century: A Social Justice Hall of Fame," "Place Matters: Metropolitics for the 21st Century," and "The Next Los Angeles: The Struggle for a Livable City." From 1984-1992 he served as a deputy to Boston Mayor Ray Flynn.

Markdown Is Holding You Back

Hacker News
newsletter.bphogan.com
2025-11-22 20:03:14
Comments...
Original Article

I've used many content formats over the years, and while I love Markdown, I run into its limitations daily when I work on larger documentation projects.

In this issue, you'll look at Markdown and explore why it might not be the best fit for technical content, and what else might work instead.

Markdown Lacks the Structure You Need

Markdown is everywhere. It's human-readable, approachable, and has just enough syntax to make docs look good in GitHub or a static site. That ease of use is why it's become the default choice for developer documentation. I'm using Markdown right now to write this newsletter issue. I love it.

But Markdown's biggest advantage is its biggest drawback: it doesn't describe the content like other formats can.

Think about how your content gets consumed. Your content isn't just for human readers. Machines use it too. Your content gets indexed by search engines, and parsed by LLMs, and those things parse the well-formed HTML your systems publish. Markdown's basic syntax only emits a small subset of the available semantic tags HTML allows.

IDE integrations can use your docs, too. And AI agents rely on structure to answer developer questions. If you're only feeding them plain-text Markdown documents to reduce the number of tokens you send, you're not providing as much context as you could.

Worse, when you want to reuse your content or syndicate content into another system, you quickly find out that Markdown is more of the lowest common denominator than a source of truth, as not all Markdown flavors are the same.

There are other options you can use that give you more control. But first, let's look deeper into why you should move away from Markdown for serious work.

Markdown is "implicit typing" for content

If you're a developer, you know all about type systems in programming languages. Some languages use Implicit typing, in which the compiler or interpreter infers the data type from the value. These languages give you flexibility, but no guarantees. That's why many developers prefer languages that use explicit typing, where you predefine data types when writing the code. In those languages, the compiler doesn't just build your code; it guarantees specific rules are followed. That's the main reason for the rise of TypeScript over JavaScript: compile-time guarantees.

Markdown is implicit typing . It lets you write quickly, but without constraints or guarantees. There's no schema. No way to enforce consistency. A heading in one file might be a concept , in another it might be a step , and there's no machine-readable distinction between the two.

To complicate things further, there are multiple flavors of Markdown, each with its own features and markup. Here are just a few:

You think you're writing "Markdown," but what works in one tool may not render in another. Some Markdown processors allow footnotes, Others ignore soft line breaks. And some even require different formatting for code blocks. Inconsistency makes Markdown a shaky foundation for anything beyond the most basic document.

And then there's MDX , which people often use to extend Markdown to support things it doesn't:

Here's a typical MDX snippet:

# Install

<Command>npm install my-library</Command>

That <Command> tag isn't Markdown at all; it's a React component. Instead of using a code block, the author chose to create a special component to standardize how all commands would display in the documentation.

It works beautifully on their site because their publishing system knows what <Command> means. But if they try to syndicate this content to another system, it breaks because that system also needs to implement that component. And even if it was supported elsewhere, there's no guarantee that the component is implemented the same way.

MDX shows that even in Markdown-centric ecosystems, people instinctively add more expressive markup. They know plain Markdown isn't enough. They're reinventing semantic markup, but in a way that's custom, brittle, and not portable.

Why semantic markup matters

Semantic markup describes what content is , not just how it should look . It's the difference between saying "here's a bullet with some text" and "here's a step in a procedure." To a human, those may look the same on a page. To a machine or to a publishing pipeline, they are entirely different.

Web developers already went through all this with HTML. Prior to HTML5, you had <div> as a logical container. But HTML5 introduced <section> , <article> , <aside> , and many other elements that described the content.

Semantic markup matters for two important and related reasons:

  • Transformation and reuse . With semantic markup, you can publish the same content to HTML, PDF, ePub, or even plain Markdown. With Markdown as your source, you can't easily go to another format. You can't turn a bullet into a <step> or a paragraph into a <para> without guessing. You can't add context if it wasn't there to begin with, but you can strip out what you don't need when you transform the document, and you can choose how to present each thing in a consistent way.
  • Machine consumption . LLMs and agents can make better use of content that carries structure. A step marked as a <step> is unambiguous. A bullet point might be a step, or a note, or just a list item. The machine has to guess. This is why XML was a preferred mechanism for web services for a long time, and why JSON Schema exists.

Let's explore four formats that give you more control over structure than plain Markdown.

reStructuredText

reStructuredText is a plain-text markup language from the Python/Docutils ecosystem that supports directives, roles, and structural semantics. It is the foundational format used by Sphinx for generating documentation.

Installation
============

.. code-block:: bash

   npm install my-library

.. note::  
   This library requires Node.JS ≥ 22.

See also :ref:`usage-guide`.

Here you see a code-block directive, an admonition ( note ), and an explicit cross-reference via :ref: . You'll find support for images, figures, topics, sidebars, pull quotes, epigraphs, and citations as well.

All of those encode semantics, not just presentation.

AsciiDoc

AsciiDoc aims to be human-readable but semantically expressive. It has attributes, conditional content, include mechanisms, and more.

Here's an example of AsciiDoc:

= Installation
:revnumber: 1.2
:platform: linux
:prev_section: introduction
:next_section: create-project

[source,bash]
----
npm install my-library
----

NOTE: This library requires Node.JS ≥ 22.

See <<usage,Usage Guide>> for examples.

AsciiDoc has native support for document front-matter. Attributes like :revnumber: or :platform: let you parameterize content.

<<usage,Usage Guide>> is a cross-reference syntax.

Like reStructuredText, AsciiDoc supports admonitions like NOTE and WARNING so you don't have to build your own custom renderer. It also has support for sidebars, and you can add line highlighting and callouts to your code blocks without additional extensions.

And if you're writing technical documentation, there's explicit support for marking up UI elements and keyboard shortcuts.

Using AsciiDoctor , you can transform AsciiDoc into other formats, including HTML, PDF, ePub, and DocBook, which you'll look at next.

DocBook (XML)

DocBook is an XML-based document model explicitly designed for technical publishing. It expresses hierarchical and semantic structure in tags and attributes, enabling industrial-grade transformations.

Here's an example

<article id="install-library">
  <title>Installation</title>
  <command>npm install my-library</command>
  <note>This library requires Node.JS &gt;= 22</note>
  <xref linkend="usage-chapter">Usage Guide</xref>
</article>

Every tag is meaningful: <command> vs <para> , <note> vs <xref> . You'll find predefined tags for function names, variables, application names, keyboard shortcuts, UI elements, and much more. Being able to mark up the specific product names and terminology you use makes it so much easier to create glossaries and indexes. And Docbook has tags for defining index terms, too.

DocBook's rich ecosystem of XSLT stylesheets supports transforming to HTML, PDF, man pages, and even Markdown. Using DocBook ensures structure and validation at scale, as long as you use the tags it provides.

Then there's DITA.

DITA (Darwin Information Typing Architecture)

DITA is a standard for writing, managing, and publishing content. It's a topic-based XML architecture with built-in reuse, specialization, and modular content design. It's an open standard, and it's widely used in enterprises for multi-channel, structured content that needs standardization and reuse.

Here's an example:

<task id="install">
  <title>Installation</title>
  <steps>
    <step><cmd>npm install my-library</cmd></step>
  </steps>
  <prolog>
    <note>This library requires Node.js &gt;= 22</note>
  </prolog>
</task>

DITA defines types like <task> and <step> , which cleanly map to procedural structure. You can compose topics, reuse via content references (conrefs), and specialize as your domain evolves.

One of the more interesting features DITA provides is the ability to filter content and create multiple versions from a single document.

The DITA Open Toolkit and many enterprise tools handle rendering, transformation, and reuse pipelines.

Ew. XML.

Yes, XML. The syntax is more verbose than Markdown. Tooling is less ubiquitous than Markdown. Migration requires effort, and your team may resist the learning curve. For small docs, Markdown's features are often enough.

But if you're already bolting semantics onto Markdown with MDX or plugins or custom scripts, you're paying that complexity cost anyway, and you don't get the benefits of standardization or portability. You're building a fragile, custom semantic layer instead of adopting one that already works.

So where does that leave you?

If you're writing a quick README or a short-lived doc, Markdown is fine. It's fast, approachable, and does the job. If you're building a developer documentation site that needs some structure, reStructuredText or AsciiDoc are better choices. They balance expressiveness with usability. And if you're managing a large doc set that needs syndication, reuse, and multi-channel publishing, DocBook and DITA give you the semantics and tooling to make that process more manageable.

The key is to start with the richest format you can manage and export downward. Markdown makes a great output for developers. It's approachable and familiar. But be careful not to lock yourself into it as your source of truth, because you can't add context back as easily as you can strip it out.

Things To Explore

  • I have a new book out. Check out Write Better with Vale . This book walks you through implementing Vale, the prose linter, on your next writing project to create consistent, quality content.
  • Tidewave.ai is a full-stack coding agent from the creators of the Elixir programming language. It supports Ruby on Rails, Phoenix, and React applications and has a free tier. You'll need an API key for OpenAI, Anthropic, or GitHub Copilot to use it.
  • Google's Chrome for Developers blog has a post on creating accessible carousels . It's worth the read if you have to implement one of these on your site.

Parting Thoughts

Before the next issue, here are a couple of things you should try to get some hands-on experience with a different format.

As always, thanks for reading. Share this issue with someone who you think would find this helpful.

I'd love to talk with you about this issue on BlueSky , Mastodon , Twitter , or LinkedIn . Let's connect!

Please support this newsletter and my work by encouraging others to subscribe and by buying a friend a copy of Write Better with Vale , tmux 3 , Exercises for Programmers , Small, Sharp Software Tools , or any of my other books .

Quebec’s New Health Care Law

Portside
portside.org
2025-11-22 19:54:03
Quebec’s New Health Care Law Dave Sat, 11/22/2025 - 14:54 ...
Original Article

François Legault, leader of Quebec’s Coalition Avenir Québec government, comes by his reputation as an authoritarian honestly. Over the past seven years, he has repeatedly used extraordinary parliamentary tools to consolidate power and sidestep democratic scrutiny. The premier has invoked the notwithstanding clause to shield two major laws from Charter challenges—effectively placing them beyond judicial review—and has relied on closure to ram through contentious legislation by cutting off debate. Most recently, the CAQ used closure again to fast-track Bill 2, a move that underscores Legault’s preference for control over consultation.

The CAQ argues its new legislation is about wresting control from a fragmented, ineffective system to ensure equitable access for all Quebecers. It tries to do this by completely overhauling the way doctors are paid, tying compensation to performance metrics that doctors call unrealistic, especially after the government’s own $1.5 billion in health care cuts led to the very shortages and delays it now claims to want to fix. These cuts have severely compromised the system’s efficiency and have left staff with fewer resources and restricted access to medical care—problems well beyond doctors’ control. Even more alarming, physicians warn, the new law threatens their constitutional right to dissent, imposing fines of up to $20,000 per day for public protest.

In 2018, one of the CAQ’s main electoral promises was that all Quebecers would have access to a doctor. Seven years later, roughly 1.5 million Quebecers still remain without a family physician . Premier Legault now believes he can get every Quebecer access to a health care professional by 2026 (coincidentally a provincial election year), even though access to a general practitioner remains difficult, wait lists to see a specialist have almost doubled in the past five years, and surgery wait lists remain high, with nearly 30 percent of operating rooms in the province’s public network closed in 2024 due to staffing shortages.

Doctors say Bill 2’s oppressive requirements penalize them for the governments’ own failings. “They tabled a bill that affects our remuneration without even consulting us while we’re in negotiations,” Véronique Godbout said back in June , when I interviewed her for Cult MTL . A practising orthopedic surgeon and the president of the Quebec Orthopedic Association, she calls the government’s actions “disrespectful and denigrating.”

She’s not the only doctor who’s furious. Andrée Vincent, the lead physician at a Montreal-based network of family doctors called Groupe de médecine de famille (GMF), is a franco-Albertan who fell in love with Quebec during her medical residency at McGill and decided to stay. Now she’s considering a move back west. She says the government is pushing through legislation built on false premises. “The facts were wrong right from the start,” she says. “The bill works under the assumption that there are enough doctors and enough resources, but there aren’t. We already can’t meet the demand, and many of us are already working way more than forty hours per week. It’s still not enough.”

Quebec health minister Christian Dubé insists the reforms would improve access to health care by incentivizing doctors to take on more patients. Francine Goldberg, owner of a multi-disciplinary GMF clinic, says the move will accomplish the opposite. “Minister Dubé does not have a good grasp of what’s happening on the ground and on the front lines.” Vincent rejects accusations of entitlement levelled by Dubé and Legault. “They accuse us of being lazy and not wanting to work,” she says. In reality, explains Goldberg, doctors feel trapped by a system that is stripping away their professional autonomy. She points to the specific benchmarks set by the new centralized agency, Santé Québec—such as the number of patients treated, appointment availability, follow-up rates, and efficiency measures—which she says override doctors’ ability to decide how best to treat patients and manage their workload.

The result, she says, will be predictable. They will simply walk away. “Clinics are going to lose doctors; therefore, patients are going to lose access to specialists, and family doctors can’t keep up with everything. I’m going to spend days vetting angry phone calls from sick patients who can’t access health care, and I won’t be able to do one single thing to help them.”

The backlash has been unprecedented. Two Quebec federations representing the province’s medical specialists and family doctors have filed a lawsuit to challenge the legislation . One-fifth of doctors at Montreal’s Queen Elizabeth Family Medicine Group plan to quit Quebec, according to the Montreal Gazette . Three chief physicians with western Quebec’s health authority in Outaouais have already resigned .

Many doctors staged a clinical work stoppage last month, further reducing access for patients, while students at all four medical schools have voted to strike . The Montreal Gazette reports that nearly 400 doctors have already applied for licences in other provinces, with Ontario and New Brunswick seeing a significant uptick in applications.

The CAQ’s own ranks haven’t been spared in the public fallout. After Social Services Minister Lionel Carmant’s daughter, a specialist in maternal-fetal medicine, published an open letter in Le Devoir , saying she might move elsewhere in Canada if the government does not “allow her to practise freely,” Carmant resigned, explaining he was “choosing his own family.” He now sits as an independent.

Days later, CAQ member of the national assembly Isabelle Poulet would be ousted from the party caucus by Legault after she openly criticized Bill 2 and was looking to cross the floor and join the Quebec Liberal Party.

While internal conflict erupts within the CAQ, other provinces are actively trying to poach Quebec doctors. Ontario premier Doug Ford even encouraged the exodus of Quebec physicians, jokingly inviting those interested to call “1-800-Doug-Ford” and reach out to him on his cellphone during a press conference. “We’ll have you working real quick,” he said. “Come by. We’ll roll out the red carpet for you.” The move angered Legault, who called Ford’s statement “ totally unacceptable .”

A province with a rapidly aging population and a worsening doctor shortage can ill afford to lose more physicians. The new legislation could also deter medical students, residents, and early-career doctors from choosing Quebec as a place to train or practise, as the province’s reputation suffers from what critics call overreach and control. Doctors already resent being micromanaged, especially under such strained conditions and mounting administrative burdens that have left many feeling stretched to the limit.

Many questions also remain about the legislation. Performance indicators are determined at the government’s discretion and can vary by region. There is, as yet, no transparent, standardized system for how these targets are calculated or communicated. “Who decides if a doctor is doing enough? Bureaucrats who’ve never treated a patient?” says Goldberg. “The metrics are not defined. Would you sign a contract where the clauses are not defined? That’s what they’re doing to doctors.”

Ultimately, Goldberg says, the bill penalizes Quebecers the most. “Highly coveted professionals can—push comes to shove—leave,” she says. “They can leave and hang their shingles elsewhere. The government is punishing the public.”

While Legault continues to play hardball, there are signs he might be bending to public pressure. The CAQ has suspended two key parts of Bill 2 that would have eliminated bonuses for doctors and invited them back to negotiations. It’s a move Vincent isn’t certain will be enough to pacify the medical community. “At this point, it would be a consultation, not a negotiation,” she says. “Because the law has already been written.”

Beneath the anger and disillusionment lies deep concern about the future of Quebec’s health care system and what that could mean for a public already having trouble accessing medical services. “If this were to truly happen,” says Vincent, referring to the law, “I wouldn’t want to get sick here in the next year.”

Introducing F# 10

Lobsters
devblogs.microsoft.com
2025-11-22 19:20:53
Comments...
Original Article

F# 10 is now shipping with .NET 10 and Visual Studio 2026. This version is a refinement release focused on clarity, consistency, and performance, bringing small but meaningful improvements that make your everyday code more legible and robust.

You can get F# 10 today in a few ways:

With F# 10, we’re continuing our journey to make the language simpler and more performant. Key ergonomic improvements include scoped warning suppression , more consistent syntax for computation expressions , and better support for auto property accessors . The release also includes an infrastructure upgrade to speed up compilation and interactive tooling in the form of the new type subsumption cache .

Language Improvements

1. Scoped warning suppression

The first feature I want to introduce for F# 10 is a much-requested one: the ability to suppress warnings in arbitrary code sections. Our compiler now supports the #warnon directive, which pairs with #nowarn to enable or disable warnings within a specific code span.

Let’s take a look at a motivating example:

// We know f is never called with a None.
let f (Some a) =    // creates warning 25, which we want to suppress
    // 2000 loc, where the incomplete match warning is beneficial

Of course, we can add a #nowarn 25 directive right above the function definition, but without a matching #warnon 25 , this will disable the FS0025 warning in the remainder of the file, risking the suppression of legitimate issues elsewhere.

With F# 10, you can delimit the section explicitly, applying the warning suppression to as narrow a scope as a single line:

#nowarn 25
let f (Some x) =    // FS0025 suppressed
#warnon 25
    // FS0025 enabled again

Conversely, if a warning is disabled globally (e.g., via a compiler flag), you can enable it locally with #warnon . This directive will then apply until a matching #nowarn or the end of the file.

Compatibility:

This feature is accompanied by several changes that improve the consistency of #nowarn / #warnon directives, leading to more predictable behavior. However, these are breaking changes that might affect your codebase when you update to F# 10. A detailed list of potential issues can be found in the RFC-1146 , but here are a couple of examples most likely to occur:

  • Multiline and empty warn directives are no longer allowed.
  • Whitespace between # and nowarn is no longer allowed.
  • Triple-quoted, interpolated, or verbatim strings cannot be used for warning numbers.

Script behavior is also affected. In previous versions, a #nowarn directive anywhere in a script applied to the whole compilation. Now, its behavior in scripts matches that in .fs files, applying only until the end of the file or a corresponding #warnon .

2. Access modifiers on auto property accessors

A frequent pattern in object-oriented programming is to create publicly readable but privately mutable state. Before F# 10, achieving this required explicit property syntax with backing fields, which added significant boilerplate:

type Ledger() =
    [<DefaultValue>] val mutable private _Balance: decimal
    member this.Balance with public get() = this._Balance and private set v = this._Balance <- v

With F# 10, you can apply distinct access modifiers to individual property accessors. This allows you to specify different access levels for the getter and setter of a property inline, enabling common patterns like publicly readable but privately mutable state without verbose boilerplate.

Now you can rewrite the above as:

type Ledger() =
    member val Balance = 0m with public get, private set

Worth noting:

You can place an access modifier either before the property name (applying to both accessors) or before individual accessors, but not both simultaneously.

Also, this feature does not extend to signature (.fsi) files. The correct signature for the Ledger example above would be:

type Ledger() =
    member Balance : decimal
    member private Balance : decimal with set

3. ValueOption optional parameters

Optional parameters can now use a struct-based ValueOption<'T> representation. By applying the [<Struct>] attribute to an optional parameter, you can instruct the compiler to use ValueOption<'T> instead of the reference-based option type. This avoids a heap allocation for the option wrapper, which is beneficial in performance-critical code.

Previous versions of F# always used the heap-allocated option type for optional parameters, even when the parameter was absent. For high-throughput scenarios or inner-loop object creation, this imposed unnecessary GC pressure. Developers working on performance-sensitive code had no way to avoid these allocations:

// Prior to F# 10: always uses reference option
type X() =
    static member M(?x : string) =
        match x with
        | Some v -> printfn "Some %s" v
        | None -> printfn "None"

Now you can use the [<Struct>] attribute on an optional parameter to leverage the struct-backed ValueOption and eliminate allocations when the argument is absent:

type X() =
    static member M([<Struct>] ?x : string) =
        match x with
        | ValueSome v -> printfn "ValueSome %s" v
        | ValueNone -> printfn "ValueNone"

Choose this struct-based option for small values or frequently constructed types where allocation pressure matters. Use the default reference-based option when you rely on existing pattern matching helpers, need reference semantics, or when the performance difference is negligible. This feature strengthens parity with other F# language constructs that already support ValueOption .

4. Tail-call support in computation expressions

Computation-expression builders (for example, coroutines or other builders implemented with resumable state machines) can now opt into tail-call optimizations. During desugaring, the compiler recognizes when an expression like return! , yield! or do! appears in a tail position and, when the builder provides special methods, routes those calls to the optimized entry points.

If a builder implements:

  • ReturnFromFinal , the compiler will call it for a tail return! (falling back to ReturnFrom if the final variant is absent).
  • YieldFromFinal , the compiler will call it for a tail yield! (falling back to YieldFrom if absent).
  • For a terminal do! , the compiler will prefer ReturnFromFinal , then YieldFromFinal , before falling back to the normal Bind pathway.

These *Final members are optional and exist purely to enable optimization: they allow builders to short-circuit continuations or otherwise relinquish resources early. Builders that do not provide these members keep their existing semantics unchanged.

Examples:

coroutine {
    yield! subRoutine() // tail position -> YieldFromFinal if available
}
coroutine {
    try
        yield! subRoutine() // not tail -> normal YieldFrom
    finally ()
}

Compatibility:

This change can be breaking if a builder already defines members with these names. In most cases, it is backward-compatible: existing builders continue to work without modification when compiled with F# 10. Older compilers will simply ignore the new *Final methods, so builders that must remain compatible with earlier compiler versions should not assume those methods will be invoked.

5. Typed bindings in computation expressions without parentheses

A long-standing inconsistency in type annotation syntax for computation expression bindings has been resolved. You can now add type annotations on let! , use! , and and! bindings without wrapping the identifier in parentheses.

Prior versions of F# required parentheses for type annotations in computation expression bindings. For example, let! (x: int) = fetchA() was valid, but the more natural let! x: int = fetchA() would cause an error. This forced developers to use visually noisy parentheses even for simple type annotations:

async {
    let! (a: int) = fetchA()
    and! (b: int) = fetchB()
    use! (d: MyDisposable) = acquireAsync()
    return a + b
}

Now you can write type annotations without parentheses, matching the style of ordinary let bindings:

async {
    let! a: int = fetchA()
    and! b: int = fetchB()
    use! d: MyDisposable = acquireAsync()
    return a + b
}

6. Allow _ in use! bindings

Prior versions of F# rejected the discard pattern in use! bindings, even when the resource value itself was never referenced. This inconsistency with regular use bindings forced developers to create throwaway identifiers like __ or _ignored just to satisfy the compiler.

The discard pattern ( _ ) now works in use! bindings within computation expressions. F# 10 allows you to use _ directly when binding asynchronous resources whose values are only needed for lifetime management, without being forced to provide a named identifier.

Now you can use the discard pattern directly, clarifying your intent and matching the behavior of use :

counterDisposable {
    use! _ = new Disposable()
    // logic
}

7. Rejecting pseudo-nested modules in types

Structural validation has been tightened in this release to reject misleading module placement within types. F# 10 now raises an error when a module declaration appears indented at the same structural level inside a type definition, preventing a common source of confusion about module scoping.

Previous versions of F# accepted module declarations indented within type definitions, but these modules were actually created as siblings to the type rather than being nested within it. This indentation pattern frequently misled developers into believing they had created a nested module, resulting in unexpected scoping behavior:

type U =
    | A
    | B
    module M = // Silently created a sibling module, not nested
        let f () = ()

Now, this pattern raises error FS0058, forcing you to clarify your intent with proper module placement:

type U =
    | A
    | B

module M =
    let f () = ()

8. Deprecation warning for omitted seq

A deprecation warning now appears for bare sequence expressions that omit the seq builder. F# 10 warns when you use bare range braces like { 1..10 } or similar forms, encouraging you to use the explicit seq { ... } form for consistency with the broader computation expression model.

Historically, F# allowed a special-case “sequence comprehension lite” syntax where the seq keyword could be omitted. This diverged from how other computation expressions work and created an inconsistency in the language:

{ 1..10 } |> List.ofSeq  // implicit sequence

Now, the compiler warns about this pattern and encourages the explicit form that clarifies semantics:

seq { 1..10 } |> List.ofSeq

This is currently a warning, not an error, giving you time to update your codebase. The explicit seq form improves code clarity and consistency with other computation expressions. Future versions of F# may make this an error, so we recommend adopting the explicit syntax when updating code.

9. Attribute target enforcement

Attribute target validation will be enforced with this release across all language constructs. F# 10 validates that attributes are only applied to their intended targets by checking AttributeTargets across let-bound values, functions, union cases, implicit constructors, structs, and classes.

Previous versions of F# silently allowed attributes to be misapplied to incompatible targets. This caused subtle bugs, such as test attributes being ignored when you forgot () to make a function or analyzer directives failing to take effect, leading to confusing CI discrepancies:

[<Fact>]
let ``this is not a function`` =  // Silently ignored, not a test!
    Assert.True(false)

Now, the compiler enforces attribute targets and raises a warning when attributes are misapplied:

[<Fact>]
//^^^^ - warning FS0842: This attribute cannot be applied to property, field, return value. Valid targets are: method
let ``works correctly`` =
    Assert.True(true)

Compatibility:

This is a breaking change that may reveal previously silent issues in your codebase. The early errors prevent test discovery problems and ensure that attributes like analyzers and decorators take effect as intended.

FSharp.Core enhancement – support for and! in task expressions

This release brings a single improvement to the FSharp.Core library: support for and! in the task computation expression.

Using task is a popular way to work with asynchronous workflows in F#, especially when interop with C# is required. However, until recently there was no concise way to await multiple tasks concurrently in a computation expression.

Perhaps you started with code that awaited computations sequentially:

// Awaiting sequentially
task {
    let! a = fetchA()
    let! b = fetchB()
    return combineAB a b
}

If you then wanted to change it to await them concurrently, you would typically use Task.WhenAll :

// Use explicit Task combinator to await concurrently
task {
    let ta = fetchA()
    let tb = fetchB()
    let! results = Task.WhenAll([| ta; tb |])
    return combineAB ta.Result tb.Result
}

With F# 10, you can instead write a more idiomatic version using and! :

task {
    let! a = fetchA()
    and! b = fetchB()
    return combineAB a b
}

This combines the semantics of the second snippet with the simplicity of the first.

Performance & Tooling

Type subsumption cache

This release also introduces a new type subsumption cache to accelerate type checking and improve IDE responsiveness, especially in projects with complex type hierarchies. In F# 10, the compiler memoizes the results of type relationship checks, reducing redundant computations and improving overall compiler and tooling performance.

Previously, the F# compiler would repeatedly perform expensive subsumption checks when dealing with large type hierarchies, such as those involving numerous numeric primitives or many interface implementations. This could lead to noticeable latency during type inference and IDE operations, particularly in large solutions or during long editing sessions, consuming extra CPU and memory.

Now, the new type subsumption cache stores the results of these checks. When the compiler needs to determine if one type can be used in place of another, it first consults the cache. This memoization avoids re-computing the same type relationships, leading to faster compilation and more responsive IntelliSense.

No code changes are required to benefit from this improvement; the performance gains are automatic.

Better trimming by default

F# 10 removes a long‑standing bit of friction with trimming F# assemblies: you no longer have to hand‑maintain an ILLink.Substitutions.xml file just to strip large F# metadata resource blobs (signature/optimization data) that aren’t needed at runtime in the final application.

When you publish with trimming enabled ( PublishTrimmed=true ), the F# build now auto‑generates a substitutions file that targets the tooling‑only F# resources. The result: smaller output by default, less boilerplate, and one fewer maintenance hazard.

If you need full manual control you can still add your own substitutions file. The auto-generation can be turned off with a property, like so: <DisableILLinkSubstitutions>false</DisableILLinkSubstitutions> .

Parallel compilation in preview

An exciting update for F# users looking to reduce their projects’ compilation times is the progressing stabilization of the parallel compilation features. Starting with .NET 10, three features: graph-based type checking, parallel IL code generation, and parallel optimization are grouped together under the ParallelCompilation project property.

Currently, this setting is turned on by default for projects that are using LangVersion=Preview , and we plan to turn it on for everyone in .NET 11. Be sure to give it a try and see if it speeds up your compilation. However, if you want to opt out but still enjoy other preview features, then set the ParallelCompilation to false.

Typecheck-only mode for scripts

F# is a great scripting language, and we want to ensure that the tooling also supports that use case. That is why we’re extending the --typecheck-only compilation flag to also work for .fsx scripts, where it’s arguably the most beneficial. In F# 9 and earlier, there was no easy way to validate syntax and type correctness of a script without running the code, which would often produce side effects.

With this release, you can simply use the --typecheck-only flag while invoking fsi to typecheck the script without execution. This feature makes it easy to add CI gates that will be able to catch script rot in your codebase.

Known issue caveat: As of publishing this post, there is a known bug that severely limits the benefits of using --typecheck-only with scripts that include others via #load . In that case type checking will conclude prematurely after processing the first loaded source file. However, the fix has already been implemented and will be included in the upcoming patch release 10.0.200.

Thanks and acknowledgements

F# is developed as a collaboration between the .NET Foundation, the F# Software Foundation, their members and other contributors including Microsoft. The F# community is involved at all stages of innovation, design, implementation and delivery and we’re proud to be a contributing part of this community.

In the past year, we merged commits from 25 contributors. We would like to thank all of you, as well as many others who filed issues , raised and voted on language suggestions , and contributed to language design . In particular, we explicitly call out some of the most active community contributors:

  • @majocha for a multitude of contributions, including infrastructure, testing and tooling improvements.
  • @Martin521 most notably for a herculean effort on the scoped #nowarn feature design and implementation.
  • @edgarfgp for his continued efforts in the area of error handling and diagnostics, as well as great improvements to the parser.
  • @auduchinok for his work on improving tooling via fixes to parser recovery and other areas, as well as contributions to testing infrastructure.

Contributor showcase

As is tradition in our release announcements, the contributor showcase section is where we celebrate some of our most impactful open-source community contributors. While our long-time contributor Jakub Majocha received a well-deserved shout-out in a .NET 8 announcement two years ago, his outstanding work over the past year on the F# compiler has earned him one here in the F# release blog post. This time, he is going to share a little about his approach to contributing to the repo:

Looking back at my PR history I see that most of them are fixes to things that I broke 🙂 That’s why as a hobbyist I try to stick to contributions that have low risk of breaking something serious. For example I had a lot of fun making the tests run in parallel. I’m also always looking for low-hanging fruit, there still are quite a few. For example you can revive some unfinished PRs that were already almost done: Type subsumption cache was one and the other was tail-call support in computation expressions – my first ever RFC implemented. I have an interest in tooling and its performance, so another surprising win that I’m very happy with is reducing StackGuard related slowdowns. I had a hunch about StackGuard impact on editor performance, and experimented for some time with various approaches. Adding some instrumentation to collect metrics was the most helpful here and the solution turned out to be very straightforward. I’d like to add huge thanks to the Amplifying F# collective for encouraging my attempts and sponsoring my Copilot subscription.

If you read it carefully, you might have noticed a teaser of an upcoming performance improvement that didn’t make it to this release, but we are very excited to bring to F# 11.

And just like Jakub, we would like to recognize Amplifying F# for their invaluable support of community contributors. Efforts like theirs help make the F# ecosystem thrive.

What’s next?

Work on F# 11 is already under way: more performance wins, language improvements, and tooling upgrades.

Go to our GitHub repo to jump into discussions , help wanted issues or good first issues for new contributors .

Trying out C++26 executors

Lobsters
mropert.github.io
2025-11-22 19:17:15
Comments...
Original Article

on C++ , Game development , Graphics programming

I wanted my program to boot up faster so I tried to multithread it with both executors and the more classic TBB.

Back in 2020 I did a remote talk at CppCon about my work on making Stellaris boot up much faster . I later brought similar improvements to Hearts of Iron IV and Europa Universalis IV (although I’m afraid the latter was never pushed to the live branch, you have to opt-in to a beta version). They key to those improvement was, of course, concurrency: put stuff to load in the background if it’s not needed upfront, and use multiple threads to do the work when possible.

One of my goals this year has been to get into graphics programming, using my spare time between clients to play with OpenGL, raylib, SDL3_GPU and more recently, Vulkan. My latest project is to build a Vulkan renderer step by step, and as it grew so did the startup times. It’s still only a matter of seconds (even in Debug) yet I thought it would be interesting to take advantage of Vulkan’s strong push towards multithreaded graphics to get some prototype going, and use the lessons learned to eventually make a proper efficient asset loader.

And so I got thinking, while I could definitely do it with some combination of C+11 future/promise/async and C++17 std::execution::par_unseq loops, or use TBB (which was my go-to on production games), I figured I should give the upcoming C++26 executors a shot.

The use case

My project currently has only a few assets to load, but they each take some measure of CPU time:

  • The mesh drawing pipeline uses 2 shaders (vertex + fragment) that need to be compiled from GLSL to SPIRV
  • The test mesh itself uses PNG for textures, which need to be decompressed to RGBA before they can uploaded to VRAM

Of course, a production game would usually have a baking process to avoid both of these caveats, making sure to ship pre-compiled SPIRV shader blobs and assets optimized with textures that can be directly uploaded to the GPU, but in my experience during development you definitely want your asset loader to able to handle unoptimized assets and text shaders to be able to iterate quickly.

Originally, the whole process was all serially done on the main thread, but obviously we can improve on that:

  • Shaders can be compiled in parallel, and the pipeline that uses them can itself be created while the assets are being loaded on another thread as long as the shader binding layout used has been initialized.
  • A GLTF asset can be parsed on a dedicated thread, and then we can have a task repacking/uploading vertices, and another that handles the textures upload, and it can itself break down the PNG texture decompression on multiple threads.

Here’s a diagram for reference:

Parallel asset and shader loading

If done correctly, most of those tasks can be done in parallel assuming our machine has enough cores available.

The classic way

To get a frame of reference (and proof that the idea is sound), I ran a quick vcpkg install tbb (you guys are using a package manager, right?) and implemented it in the way I’m familiar with. I picked TBB over std::execution_par_unseq and std::async because I am quite familiar with TBB and I know all jobs are guaranteed to run on the same thread pool (unlike std::async which might create new thread) and also because it provides the very convenient tbb::task_scheduler_observer which can used to ensure the worker threads will properly register in Optick (I had to hack around a bit to make that work with stdexec ).

Here’s a shortened version of what it looks like:

Pipeline create_pipeline( vk::Device device, vk::DescriptorSetLayout layout )
{
    vk::raii::ShaderModule vertex_shader;
    vk::raii::ShaderModule fragment_shader;
    tbb::parallel_invoke(
        [ & ] { vertex_shader = compile_shader( device, "mesh.vs", VK_SHADER_STAGE_VERTEX_BIT ); },
        [ & ] { fragment_shader = compile_shader( device, "mesh.fs", VK_SHADER_STAGE_FRAGMENT_BIT ); }
    );

    /* ... */
    return make_pipeline( device, layout, vertex_shader, fragment_shader );
}

void MeshLoadJob::decompress_textures()
{  
    tbb::parallel_for(
        size_t( 0 ),
        _textures.size(),
        [ & ]( size_t index )
        {
            _textures[ index ].decompress();
        } );
}

Model load_model( vk::Device device, TexManager& tex_manager )
{
    MeshLoadJob job( device, tex_manager, "model.glb" );
    job.parse();
    tbb::parallel_invoke( [ & ]() { job.upload_vertices(); },
                          [ & ]
                          {
                              job.decompress_textures();
                              job.upload_textures();
                          } );
    return job.finalize();
}

void main()
{
    /* Init Vulkan... */

    Pipeline pipeline;
    Model model;
    tbb::parallel_invoke(
        [ & ] { pipeline = create_pipeline( device, layout );	},
        [ & ] { model = load_model( device, tex_manager ); } );

    /* Draw loop... */
}

And here’s a profile capture of the result:

Parallel asset and shader loading with TBB

This is about as good as we could expect, TBB uses 9 work thread (including the main one) to decompress 6 textures, upload vertices and compile 2 shaders. Technically we could further optimize it because the RGB to RGBA conversion done by SDL in upload_image() is still taking about 50ms of mostly CPU crunch but this is good enough to illustrate our gains here. The whole process takes about 200 ms in Release and 900 ms in Debug (mostly bound by how slow the shader compiler is in Debug, but mixing Debug/Release packages is a topic for another day) down from maybe 4-5 seconds when iterating with the Debug build.

Now, let’s see how executors will compare.

The C++26 way

For this experiment, I downloaded NVIDIA’s stdexec which is the reference implementation used for the executor’s paper . I initially used the version from vcpkg , but I realized the package is outdated by over a year so I switched to the main branch as of this post’s date. Then, I focused on solving the asset loading part, thinking that once I got it going for that I could compose it further to add the shader compilation.

Here’s how it looks like:

Model load_model( vk::Device device, TexManager& tex_manager )
{
    MeshLoadJob job( device, tex_manager, "model.glb" );

    using namespace stdexec;
    auto parse_ask = split( scheduler.schedule() | then( [ & ] { job.parse(); } ) );
    sync_wait(
        when_all( parse_ask | then( [ & ] { job.upload_vertices(); } ),
                  parse_ask
                      | let_value(
                          [ & ]
                          {
                              return just()
                                  | bulk( stdexec::par_unseq, job.get_texture_count(),
                                             [ & ]( int i ) { job.decompress_texture( i ); } );
                          } )
                      | then( [ & ] { job.upload_textures(); } ) )
        | then( [ & ] { job.finalize(); } ) );

    return job.get_result();
}

It works and we can describe the whole job in one go, but sadly the practical result didn’t look overwhelming:

Parallel asset loading with stdexec, attempt #1

As you can see, the bulk() part was actually executed in serial, and no amount of stdexec::par_unseq seemed to affect it. This is also quite verbose. The let_value() wrapper is necessary to make sure the count passed to bulk() is only evaluated once the parse() is done and not immediately, else it gets 0 because we haven’t run the parser yet (and so we skip decompression and crash later cause the uncompressed textures aren’t there). Needing to do just() | bulk() and schedule() | then() all the time is also adding extra noise.

Debugging why my bulk() was not actually ran in parallel proved fruitless, lost in thousand character long template argument lists and compile-time evaluation that I couldn’t put any breakpoint on to see where the branches go wrong.

Worse, I only caught the issue because I am always double checking the profiler. I really don’t like the idea that stdexec::par_unseq seems to only be a suggestion, meaning it can result in cases where it seems to work but the performance is actually terrible because everything is run in serial. I’d much prefer a compile error if my task construction somehow breaks a constraint required to parallelize.

After a while I was saved by my ex-colleague Jérémy Demeule who suggested I added a continues_on() call to make sure the executors didn’t lose track of where they’re supposed to execute and go back to the original thread pool scheduler:

let_value(
    [ & ]
    {
        return just()
            | continues_on( scheduler ) // remind the executor where we are...
            | bulk( stdexec::par_unseq, job.get_texture_count(),
                         [ & ]( int i ) { job.decompress_texture( i ); } );
    } )

This way we get proper multithreading:

Parallel asset loading with stdexec, attempt #2

I’m not sure if this is an intended part of the spec or a bug in stdexec implementation but I hope it’s the latter because else I can see tripping a lot of people silently (remember: it doesn’t fail to compile, it just executes on the default serial scheduler).

In the end I elected to simplify the code by first running the parse() part directly, then building a task graph:

Model load_model( vk::Device device, TexManager& tex_manager )
{
    MeshLoadJob job( device, tex_manager, "model.glb" );

    using namespace stdexec;
    job.parse();
    sync_wait( when_all( scheduler.schedule() | then( [ & ] { job.upload_vertices(); } ),
                         scheduler.schedule()
                             | bulk( stdexec::par_unseq, job.get_texture_count(),
                                         [ & ]( int i ) { job.decompress_texture( i ); } )
                             | then( [ & ] { job.upload_textures(); } ) )
               | then( [ & ] { job.finalize(); } ) );

    return job.get_result();
}

Conclusion

While I like the pipe declarative syntax I worry that the potential footguns and extra verbosity will turn off potential users. As with a lot of recent C++ libraries, the library relies on a lot of template/constexpr magic going right, and leaves you in a pretty bad spot when it doesn’t.

The amount of extra just() and continues_on() and then() needing to start a task chain in general feels like a bit too much and could benefit from some trimming/shortcuts. Especially since introducing any form of runtime dependency in the chain already requires some amount of let_value() wrapping. This is probably less of an issue when describing a networking processing chain or a single async disk load, but handling dynamic splits and joins isn’t nearly as nice.

I also would like to see some form wait_steal rather than only sync_wait . Feels like a waste that invoking the whole thing puts the caller to sleep rather than have them contribute to the work. I haven’t tried how it behaves if called from a work thread on the same scheduler, hopefully it doesn’t just keep it idle while it waits, else I could see it starving itself out.

I haven’t mentioned the impact on compile times but according to MSVC’s build insights adding only this one execution added a whopping 5s build time, mostly in template instantiation so even modules won’t save this one.

Finally, I haven’t mentioned that a big feature of C++26 execution is the ability to pass parameters between tasks and automatically handle their lifecycle. It looks quite nice for single/one shot cases, but you’d notice for this case I elected to keep all the shared data as an external struct on the stack captured by reference rather than having to deal with how to handle data passing between and splits and joins.

In the end, I know we’ve been talking about executors for over a decade, and the people working on it have certainly done an extremely smart job that I couldn’t pull off myself. Yet, I wonder: is it the right way to add such a big thing to the standard? Wouldn’t that energy be better spent in making a widely used and adopted library (like Boost in its time), and then standardize it once we had enough real-world experience in live projects? After all that’s how we got unique_ptr , optional , tuple , any , span , variant and even expected .

To quote the project’s page (emphasis mine):

stdexec is experimental in nature and subject to change without warning. The authors and NVIDIA do not guarantee that this code is fit for any purpose whatsoever .

This doesn’t make me feel like something that should to be required in every C++ compiler starting next year. And obviously with that in mind I’ll keep this as an experiment on a branch and stick with TBB until then.

Time in C++: Understanding and the Concept of Clocks

Lobsters
www.sandordargo.com
2025-11-22 19:15:57
Comments...
Original Article

Let’s start a new series!

A few months ago, I began working on tasks related to instrumentation and logging, and I had to deal with timestamps, different clock types, and more. Some parts felt a bit cryptic at first, so I decided to dig deeper into the topic.

As I’ve shared many times during the past eight years, the main goal of this blog is to document what I learn — hence this new series on clocks and time.

In this first part, I’ll set the stage for the whole series, which is aimed at C++ developers who’ve used std::chrono but never really thought deeply about what a clock is.

Many people assume that all clocks just return “the current time” or that it doesn’t matter whether they use system_clock or steady_clock . Such assumptions can lead to subtle bugs.

You’ve probably written something like this a hundred times:

1
auto start = std::chrono::steady_clock::now();

Even if you have, time handling can still surprise you. Ignoring time zones helps, but daylight saving changes can still wreak havoc!

And let’s not forget about testing or choosing the wrong clock type — both can cause headaches.

Since C++11, the <chrono> library has made working with time simpler, but we can only use its features to their full potential if we understand them well .

<chrono> was introduced to fix two major pain points:

  • Lack of type safety : Developers mixed seconds, milliseconds, etc., using plain integers.
  • Lack of composability : It was hard to build higher-level abstractions on top of raw time values.

The <chrono> library was built around 3 pillars:

  • Durations ( std::chorno::duration )
  • Timepoints ( std::chrono::time_point )
  • Clocks ( std::chrono::clock )

Later, C++20 introduced calendar dates and time zones, but we won’t cover those here — this series focuses on clocks.

std::chorno::duration

Durations — unsurprisingly — represent time intervals. They are defined as:

1
2
3
4
template<
    class Rep,
    class Period = std::ratio<1>
> class duration;

Rep is a numeric type representing the tick count, while Period represents how long one tick is, in seconds, expressed as a compile-time rational fraction ( std::ratio ).

Let’s look into some other ratios to understand better their meaning:

  • std::ratio<1, 1'000'000'000> means nano, you can actually replace it with std::nano
  • std::ratio<1, 1'000'000> is the same as std::micro
  • std::ratio<1, 1'000> is the same as std::milli

Now let’s look at some common durations:

  • std::duration<long long, std::nano> (or std::duration<long long, std::ratio<1, 1'000'000'000>> ) represents durations in nanoseconds, it’s aliased as std::chrono::nanoseconds
  • std::duration<long long, std milli> represents durations in milliseconds ( std::chrono::milliseconds )
  • std::duration<long long> represents durations in seconds ( std::chrono::seconds )

As a fraction doesn’t have to be smaller than one, we can represent durations in coarser granularitly.

  • std::chrono::duration<int, std::ratio<60>> is std::chrono::minutes
  • std::chrono::duration<int, std::ratio<3600>> is std::chrono::hours

C++20 even added aliases for days , weeks , months and years .

Also note that the type for Rep changed in the above examples depending on the number of bytes you need to represent durations.

A great thing about std::chrono::duration is that it prevents comparing values with different units by accident:

1
2
3
4
5
6
7
8
9
#include <chrono>

int main() {
    using namespace std::chrono_literals;
    constexpr auto d1 = 500ms;   // std::chrono::milliseconds
    constexpr auto d2 = 2s;     // std::chrono::seconds
    static_assert(d1 < d2);
    return 0;
}

If d1 and d2 were plain integers, 500 < 2 would be false — and you’d have an embarrassing bug.

std::chrono::time_point

A time_point represents a point in time relative to a clock’s epoch (its zero time point).

1
2
3
4
5
6
7
8
9
10
11
#include <chrono>
#include <iostream>

int main() {
    using namespace std::chrono_literals;
    std::chrono::time_point now = std::chrono::system_clock::now();
    std::cout << now << '\n'
}
/*
2025-11-05 04:49:07.802540469
*/

You can subtract two time_points to get a duration, but adding them doesn’t make sense — what would 2025-11-05 04:49:07 + 2025-11-07 21:15:51 even mean?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#include <chrono>
#include <iostream>

int main() {
    using namespace std::chrono_literals;
    std::chrono::time_point now = std::chrono::system_clock::now();
    std::chrono::time_point now2 = std::chrono::system_clock::now();
    auto d = now2 - now;
    auto d2 = now - now2;
    std::cout << d << ' ' << d2 <<  '\n';
    return 0;
}
/*
565ns -565ns
*/

You can also shift a time_point by adding or subtracting a duration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
#include <chrono>
#include <iostream>

int main() {
    using namespace std::chrono_literals;
    std::chrono::time_point now = std::chrono::system_clock::now();
    std::chrono::time_point earlier = now - 10min;
    std::chrono::time_point later = now + 10min;
    std::cout << earlier << ' ' << now << ' ' << later << '\n';
    return 0;
}
/*
2025-11-05 04:49:53.388934987 2025-11-05 04:59:53.388934987 2025-11-05 05:09:53.388934987
*/

But to get a time_point , we first need a clock…

std::chrono::clock

Last but not least, let’s talk about the third main building block of <chrono> : clocks .

Clocks are the source of time_point s. At their core, they need two things:

  • a starting point, a.k.a. an epoch
  • and a tick rate

A typical example is the Unix epoch (1st of January 1970) with one tick per second.

For a type to qualify as a clock ( std::chrono::is_clock_v<T> == true ), it must:

  • have a nested duration type
  • have a nested time_point type (typically std::chrono::time_point<Clock, duration> )
  • provide a static now() function that returns a time_point
  • have a constant is_steady boolean
  • define rep and period types for the underlying representation and tick frequency (just like duration does)

A clock is steady if its guaranteed to be monotonic , in other words, now() can never go backwards as the time moves forward. Given that we’ve just switched back to winter time in many countries, it’s easy to see that not every clock is steady! C++ also requires that steady clocks tick at a constant, uniform rate.

C++11 introduced three standard clocks:

  • system_clock
  • steady_clock
  • high_resolution_clock

Then C++20 brought us 4 more:

  • utc_clock
  • tai_clock
  • gps_clock
  • file_clock

Each clock has its own purpose and behavior — we’ll explore them in the coming articles.

Conclusion

<chrono> brought much-needed clarity, safety, and composability to time handling in C++. Understanding durations, time points, and clocks is the foundation for writing robust and testable time-related code.

In the next articles, we’ll dive deeper into the standard clocks and how to choose the right one for your use case. Stay tuned!

Connect deeper

If you liked this article, please

WhatsApp API flaw let researchers scrape 3.5 billion accounts

Bleeping Computer
www.bleepingcomputer.com
2025-11-22 18:53:21
Researchers compiled a list of 3.5 billion WhatsApp mobile phone numbers and associated personal information by abusing a contact-discovery API that lacked rate limiting. [...]...
Original Article

WhatsApp

Researchers compiled a list of 3.5 billion WhatsApp mobile phone numbers and associated personal information by abusing a contact-discovery API that lacked rate limiting.

The team reported the issue to WhatsApp, and the company has since added rate-limiting protections to prevent similar abuse.

While this study was conducted by researchers who have not released the data, it illustrates a common tactic used by threat actors to scrape user information from publicly exposed and unprotected APIs.

Wiz

Abusing WhatsApp API

The researchers from the University of Vienna and SBA Research used WhatsApp's contact-discovery feature, which lets you submit a phone number to the platform's GetDeviceList API endpoint to determine whether a phone number is associated with an account and what devices were used.

Without strict rate limiting, APIs like this can be abused to perform large-scale enumeration across a platform.

The researchers found this to be the case with WhatsApp, as they were able to send a high volume of queries directly to WhatsApp's servers, checking more than 100 million numbers per hour.

They ran the entire operation from a single university server using just five authenticated sessions, initially expecting to get caught by WhatsApp. However, the platform never blocked the accounts, never throttled their traffic, never restricted their IP address, and never reached out despite all the abusive activity coming from one device.

The researchers then generated a global set of 63 billion potential mobile numbers and tested all of them against the API. Their queries returned 3.5 billion active WhatsApp accounts.

The results also gave a previously unknown snapshot of how WhatsApp is used globally, showing where the platform is most used:

  • India: 749 million
  • Indonesia: 235 million
  • Brazil: 206 million
  • United States: 138 million
  • Russia: 133 million
  • Mexico: 128 million

Millions of active accounts were also identified inside countries where WhatsApp was banned at the time, including China, Iran, North Korea, and Myanmar. In Iran, usage continued to grow as the ban was lifted in December 2024.

In addition to confirming whether a phone number was used on WhatsApp, the researchers used other API endpoints to enumerate additional information about users, including the GetUserInfo , GetPrekeys , and FetchPicture .

Using these additional APIs, the researchers were able to collect profile photos, "about" text, and information about other devices associated with a WhatsApp phone number.

A test of US numbers downloaded 77 million profile photos without any rate limiting, with many showing identifiable faces. If public "about" text was available, it also revealed personal details and links to other social accounts.

Finally, when the researchers compared their findings with the 2021 Facebook phone-number scrape , they found that 58% of the leaked Facebook numbers were still active on WhatsApp in 2025. The researchers explain that large-scale phone number leaks are so damaging because they can remain useful in other malicious behavior for years.

"With 3.5 B records (i.e., active accounts), we analyze a dataset that would, to our knowledge, classify as the largest data leak in history, had it not been collated as part of a responsibly-conducted research study," explains the " Hey there! You are using WhatsApp: Enumerating Three Billion Accounts for Security and Privacy " paper.

"The dataset contains phone numbers, timestamps, about text, profile pictures, and public keys for E2EE encryption, and its release would entail adverse implications to the included users."

Other malicious cases of API abuse

WhatsApp's lack of rate limiting for its APIs is illustrative of a widespread issue on online platforms, where APIs are designed to make it easy to share information and perform tasks, but they also become vectors for large-scale scraping.

In 2021, threat actors exploited a bug in Facebook's "Add Friend" feature that allowed them to upload contact lists from a phone and check whether those contacts were on the platform. However, this API also did not properly rate-limit requests, allowing threat actors to create profiles for 533 million users that included their phone numbers, Facebook IDs, names, and genders.

Meta later confirmed that the data came from automated scraping of an API that lacked proper safeguards, with the Irish Data Protection Commission (DPC) fining Meta €265 million over the leak .

Twitter faced a similar problem when attackers exploited an API vulnerability to match phone numbers and email addresses to 54 million accounts.

Dell disclosed that 49 million customer records were scraped after attackers abused an unprotected API endpoint .

All of these incidents, including WhatsApp's, are caused by APIs that perform account or data lookups without adequate rate limits, making them easy targets for large-scale enumeration.

Wiz

7 Security Best Practices for MCP

As MCP (Model Context Protocol) becomes the standard for connecting LLMs to tools and data, security teams are moving fast to keep these new services safe.

This free cheat sheet outlines 7 best practices you can start using today.

Show HN: Forty.News – Daily news, but on a 40-year delay

Hacker News
forty.news
2025-11-22 18:47:08
Comments...

Host a website from an old phone using postmarketOS

Lobsters
far.computer
2025-11-22 18:42:53
Comments...

Depot (YC W23) Is Hiring a Staff Infrastructure Engineer

Hacker News
www.ycombinator.com
2025-11-22 18:05:47
Comments...
Original Article

Build faster. Waste less time.

Staff Infrastructure Engineer

$100K - $250K US / GB / CA / FR / ES / NL / DE / PT / IT / CH / HU / PL / GR / SE / BE / Remote (US; GB; CA; FR; ES; NL; DE; PT; IT; CH; HU; PL; GR; SE; BE)

Skills

Go, Linux, Docker, Amazon Web Services (AWS)

Connect directly with founders of the best YC-funded startups.

Apply to role ›

About the role

Staff Infrastructure Engineer

At Depot, we are on a mission to redefine software collaboration and accelerate developers everywhere. We are creating a build performance and developer platform unlike any other, combining performance, empathy, and centralized collaboration to enable companies to iterate exponentially faster.

We launch millions of EC2 instances per month and orchestrate half a petabyte of cache data to accelerate CI jobs and local builds. We are looking to hire a Staff Infrastructure Engineer who can build for the next level of scale.

For this role, you should be a seasoned expert with robust systems engineering skills and the ability to engage deeply in technical discussions. As part of a small team, you will work side-by-side with other engineers to test ideas, build proofs of concept, and ultimately ship quality solutions to customers. You will be a key contributor with the ownership and autonomy to see projects through from beginning to end.

Please note: We are an equal opportunity employer and remote-only company. At this time, we can only support hiring within North America and Europe for this role.

Responsibilities

  • Infrastructure Management: Maintain and evolve the AWS infrastructure, ECS clusters, and networking layer using Terraform.
  • Systems Engineering: Dive deep into low-level systems including compute isolation (Firecracker/microVMs), storage optimization (Ceph, LVM), and performance debugging across the stack.
  • Automation & Tooling: Automate critical workflows such as AMI updates, machine binary deployments, and internal tooling to improve developer velocity.
  • Observability: Implement and refine tracing, monitoring, and SLAs for core services like BuildKit builds, Depot Cache, and the registry.
  • Backend Development: Write and maintain Go services responsible for infrastructure provisioning, orchestration, and management.
  • Reliability: Participate in the on-call rotation, leading incident response, and supporting deep technical debugging of production issues.

What we’re looking for

  • 7+ years of experience in infrastructure, systems engineering, or DevOps roles.
  • Proficiency with AWS, Go, and Terraform.
  • Experience with or a desire to learn the technologies that power Depot, including Firecracker/microVMs, Ceph, LVM, OpenTelemetry (OTEL), ClickHouse, and Vitess.
  • Strong debugging skills in distributed systems, networking, and storage, with experience in high-scale production environments.
  • A self-starter who is excited about hard challenges and comfortable working in an environment with ambiguity.
  • The ability to take complex problems and feedback, then distill them down to clear and focused solutions.
  • Capable of asking questions, experimenting, and advocating for what is right.
  • A strong desire for ownership, with the ability to define goals and execute on them from idea to measurement.
  • Strong written and verbal communication skills, and comfort collaborating asynchronously across time zones.

Depot values and culture

Depot is a fully remote and globally distributed team across the US, Europe, and Canada. As a remote startup, there are several key values and expectations:

  • The team will only get more distributed as time goes on. As such, there is always stuff happening across Depot, so the ideal candidate thrives in that type of environment.
  • Work is not family. You are expected to get things done and work hard to meet goals. But you should spend time with your family and friends, finding the balance that accomplishes both.
  • As a small team, the goal is to accomplish massive things. Everyone at Depot is a self-starter, deeply passionate about the problems being solved, and driven to solve them well.
  • Ownership is the key to growth. Part of that growth is making choices, and owning the success, failure, and lessons learned that come with those choices.
  • Decisions are based on data and customer needs. Success comes from being data-driven in decision-making.

About Depot

Depot is a build acceleration and developer productivity platform that saves companies like PostHog, Wistia, Semgrep, and Secoda thousands of hours in build time every week.

We are developers. We started Depot because we were frustrated with the constant pain of slow build performance. We were fed up waiting for builds and annoyed by the lack of tooling and providers that actually made builds performant. So, we went and built the solution we had always wanted.

Slow builds are the dam standing in the way between mediocrity and innovation. They’re wasteful, expensive, and a drain on developer happiness & productivity. They slow down innovation.

Taking a 40-minute build down to a minute, changes everything. We help folks save literal years in build time every single week.

And we’re just getting started. For us, it’s all about iteration speed and keeping developers in their flow state. Our mission is to be relentless in accelerating software development.

Depot

Founded: 2022

Batch: W23

Team Size: 8

Status: Active

Founders

China Reaches Energy Milestone by "Breeding" Uranium from Thorium

Hacker News
humanprogress.org
2025-11-22 17:49:27
Comments...
Original Article

“An experimental reactor developed in the Gobi Desert by the Chinese Academy of Sciences’ Shanghai Institute of Applied Physics has achieved thorium-to-uranium fuel conversion, paving the way for an almost endless supply of nuclear energy.

The achievement makes the 2 megawatt liquid-fuelled thorium-based molten salt reactor (TMSR) the only operating example of the technology in the world to have successfully loaded and used thorium fuel.

According to the academy, the experiment has provided initial proof of the technical feasibility of using thorium resources in molten salt reactor systems and represents a major leap forward for the technology.

It is the first time in the world that scientists have been able to acquire experimental data on thorium operations from inside a molten salt reactor, according to a report by Science and Technology Daily.”

From South China Morning Post .

China reaches energy milestone by "breeding" uranium from thorium

Hacker News
www.scmp.com
2025-11-22 17:49:27
Comments...
Original Article

An experimental reactor developed in the Gobi Desert by the Chinese Academy of Sciences’ Shanghai Institute of Applied Physics has achieved thorium-to-uranium fuel conversion, paving the way for an almost endless supply of nuclear energy.

The achievement makes the 2 megawatt liquid-fuelled thorium-based molten salt reactor (TMSR) the only operating example of the technology in the world to have successfully loaded and used thorium fuel.

According to the academy, the experiment has provided initial proof of the technical feasibility of using thorium resources in molten salt reactor systems and represents a major leap forward for the technology.

It is the first time in the world that scientists have been able to acquire experimental data on thorium operations from inside a molten salt reactor, according to a report by Science and Technology Daily.

The article, published on Saturday, was China’s first official confirmation of its success in the development of TMSR technology, an innovation that is poised to reshape the future of clean sustainable nuclear energy.

02:37

Trump orders US military to resume nuclear weapons tests for first time in 33 years

Trump orders US military to resume nuclear weapons tests for first time in 33 years

Li Qingnuan, Communist Party secretary and deputy director at the Shanghai Institute of Applied Physics, told the newspaper that “since achieving first criticality on October 11, 2023, the thorium molten salt reactor has been steadily generating heat through nuclear fission”.

Gwern's "Stem Humor" Directory

Hacker News
gwern.net
2025-11-22 17:49:05
Comments...
Original Article

Gwern

“Conference Fermi Problems ”, Gwern 2025

Conference Fermi Problems

“Human Cannibalism Alignment Chart ”, Gwern et al 2025

Human Cannibalism Alignment Chart

“The Meta-LW Doomsday Argument ”, Gwern 2025

The Meta-LW Doomsday Argument

“Estimated Cost of DMT Machine Elf Prime Factorization Experiment ”, Gwern 2015

Estimated Cost of DMT Machine Elf Prime Factorization Experiment

“A Christmas Protestation ”, o1-pro et al 2024

A Christmas Protestation

“Second Life Sentences ”, Gwern 2024

Second Life Sentences

“The Carcinisation of Satan ”, Gwern 2024

The Carcinisation of Satan :

“On the Impossibility of Superintelligent Rubik’s Cube Solvers ”, Gwern et al 2023

On the Impossibility of Superintelligent Rubik’s Cube Solvers

“Paperclip Alignment Chart ”, Gwern 2023

Paperclip Alignment Chart

“Rare Greek Variables ”, Gwern 2021

Rare Greek Variables

“Gilles Goullet, Author of the Blindsight ”, Gwern 2010

Gilles Goullet, Author of the Blindsight

The realities of being a pop star

Hacker News
itscharlibb.substack.com
2025-11-22 17:47:23
Comments...
Original Article

Being a pop star has its pros and cons like most jobs in this world but before I state some of them I want to clarify that firstly I don’t view what I do as a ‘job’ and I secondly don’t really view myself as purely a pop star, I’m just using that terminology specifically for this piece of writing. I’ve always been delving into different creative zones adjacent to music. Building out an entire album roll out obviously constitutes immersing yourself in many different mediums but now more than ever I feel a pull away from music and into other realms. Unfortunately this probably means I’d describe myself more as a ‘creative’ (gross) or plain and simply put: an artist. However, for the purposes of this piece I’m focusing purely on the the realities of being a pop star because it was my original dream, because it’s the role in my life I have the most experience navigating and because it’s also the most ridiculous one.

One of the main realities of being a pop star is that at a certain level, it’s really fucking fun. You get to go to great parties in a black SUV and you can smoke cigarettes in the car and scream out of the sunroof and all that cliche shit. At these parties you sometimes get to meet interesting people and those interesting people often actually want to meet you. You get to wear fabulous clothes and shoes and jewelry that sometimes comes with its own security guard who trails you around the party making sure you don’t loose the extortionate earrings sitting on your lobes or let some random person you’ve just met in the bathroom try on the necklace around your neck that is equivalent to the heart of the ocean. You get good free shit like phones and laptops and vinyl and trips and shroom gummies and headphones and clothes and sometimes even an electric bike that will sit in your garage untouched for the best part of 5 years. You get to enter restaurants through the back entrance and give a half smile to the head chef (who probably hates you) and the waiters (who probably hate you too) as they sweat away doing an actual real service industry job while you strut through the kitchen with your 4 best friends who are tagging along for the ride. You get to feel special, but you also have to at points feel embarrassed by how stupid the whole thing is. You also get to hear a lot of incredible music that undoubtably is going to shift culture and public perception months before its released (the time Addison played me Diet Pepsi for the first time while driving around New York after dinner at Casino springs to mind). Sometimes you get to help out your other pop star friends by providing an opinion or lending an ear or a helping make a decision relating to their work which allows you to feel a part of a interconnected community of people you love and respect. You also get to have fans and their dedication to your work makes you feel like they will be there for you until the end of time, even though in reality they won’t. You get to stand on stage and feel like a God. You get to make people cry with happiness, you soundtrack their break ups, their recovery, their crazy nights out, their revenge, their love, their lives. You get to travel the world and see all kinds of different places and you never even have to worry about booking a single element of the travel yourself because you have an amazing tour manager to do that for you. You get to call in sick whenever you want and you never have to worry about bailing on work last minute because you know for certain that there’s another pop star out there who’s actually way more unreliable and flakey than you. Thank God.

You will also end up spending a lot of time inhabiting strange and soulless liminal spaces. Whether its the holding area of the event you’re about to enter, the airport lounge, the visa office, the claustrophobic tour bus, the greenroom with no windows, the underneath of a stage or the set build of a photoshoot or music video you’re on, you are often caught in the in-between. You’re in transit, you’re going somewhere but the journey itself takes up the majority of the experience. When Rachel Sennott came to shoot her scene in our upcoming film The Moment she was overnighted in a van straight from the front row of the Balenciaga show in Paris to the back doors of a warehouse in London’s Docklands. She was bundled up with blankets and pillows and shipped directly to us like a package. The journey took all night but she was only on set for around an hour.

Another thing about being a pop star is that you cannot avoid the fact that some people are simply determined to prove that you are stupid. I’ve always been completely fascinated by this and think it has something to do with self projection. Being a pop star has always been partially about being a fantasy and obviously the fantasy is decided mostly by the consumer. Marketing and strategy and packaging and presentation can do it’s best to guide a viewer to the desired outcome but at the end of the day the consumer gets to decide whether a pop star is a symbol of sex, or anarchy or intelligence or whatever else they wish to see. Sometimes people don’t like to be lumped in with general consensus, they like to go against the grain of public opinion and that’s when a totally opposite defiant stance is born. Instead of “she’s a sex symbol” it becomes “she’s a whore”. Instead of “she’s anarchic” it's “she’s a fucking drug addict”. Instead of “she’s intelligent” it’s “she’s pretentious and said a whole load of nothing” and so on and so on. I find that this is often where the stupidity narrative can be born. I’ve always wondered why someone else’s success triggers such rage and anger in certain people and I think it probably all boils down to the fact that the patriarchal society we unfortunately live in has successfully brainwashed us all. We are still trained to hate women, to hate ourselves and to be angry at women if they step out of the neat little box that public perception has put them in. I think subconsciously people still believe there is only room for women to be a certain type of way and once they claim to be one way they better not DARE grow or change or morph into something else. Also people obviously want the clicks and an opposite stance is more likely to get that. When I joined substack there were a flurry of think pieces and questions as to why. Some people theorized on the desire for long form content and a deeper connection with a fanbase, some people were just purely excited, some people suggested I might be heeding the advice of my record label in attempt to be omnipresent across all platforms, some people were surprised I even had enough brain cells left to write because of all that partying I do (!). The truth is I’ve always loved writing so why the fuck not? I felt generally welcomed to the community but also did see the small wave of people being annoyed I’d broken down the walls of my box they were determined to keep me in, the box, or should I say the brand, of the party girl who smokes cigs, does coke, loves the color green and has the capacity for nothing else. To them I’m a silly little idiot because that’s what they desire me to be. I guess sometimes that’s just part of the deal.

Sometimes being a pop star can be really embarrassing, especially when you’re around old friends of family members who have known you since before you could talk. The discrepancy in lifestyles becomes more and more drastic the more successful and paranoid you become. As a British person the longer you stay in LA the more you lose touch with the realities of certain things, but that’s why being a pop star can also be seriously humbling too, especially when your old friends mock and ridicule you for caring about something absolutely pointless. In ways being a pop star makes me think about the person I used to be compared to the person I am now. How is that person different? Or is she still the same? A couple of weeks ago Yung Lean came for dinner at my house and we were discussing some industry adjacent friends of ours and whether we felt they had changed after their successes in their certain fields. The next day my brain was stewing and so I text him to ask him whether he thought I had changed. I knew he would be honest because he always is and I know he sees through everything, all the persona and all the facade. He is probably one of the wisest people I know. I’m sat there waiting for his response and the three speech bubble dots kept appearing and disappearing on my phone screen which was sending me into a total spiral. When he finally pressed send his message said that he thought I had not changed from the person he knew when we were younger and that he didn’t think I would in the future but also that I definitely do have ‘yes people’ around me that blow smoke up my ass. I said I could see the truth in that but luckily he went on to say that generally speaking I’m too British and self deprecating to actually believe any of the wild compliments the ‘yes people’ might pay me so I was probably safe.

My final thought on being a pop star is that there is a level of expectation for you to be entirely truthful all the time. Over recent years some people seem to have developed a connection between fame and moral responsibility that I’ve never really understood. All my favorite artists are absolutely not role models nor would I want them to be, but maybe that’s just me. I want hedonism, danger and a sense of anti establishment to come along with my artists because when I was younger I wanted to escape through them. I don’t care if they tell the truth or lie or play a character or adopt a persona or fabricate entire scenarios and worlds. To me that’s the point, that’s the drama, that’s the fun, that’s the FANTASY.

I’m ending the essay with a link to one of my favorite Lou Reed interviews . Is it performance? Is it truth? Is it lies? Who fucking cares? In my opinion it’s just funny and cool.

Discussion about this post

Unusual circuits in the Intel 386's standard cell logic

Lobsters
www.righto.com
2025-11-22 17:35:49
Comments...
Original Article

I've been studying the standard cell circuitry in the Intel 386 processor recently. The 386, introduced in 1985, was Intel's most complex processor at the time, containing 285,000 transistors. Intel's existing design techniques couldn't handle this complexity and the chip began to fall behind schedule. To meet the schedule, the 386 team started using a technique called standard cell logic. Instead of laying out each transistor manually, the layout process was performed by a computer.

The idea behind standard cell logic is to create standardized circuits (standard cells) for each type of logic element, such as an inverter, NAND gate, or latch. You feed your circuit description into software that selects the necessary cells, positions these cells into columns, and then routes the wiring between the cells. This "automatic place and route" process creates the chip layout much faster than manual layout. However, switching to standard cells was a risky decision since if the software couldn't create a dense enough layout, the chip couldn't be manufactured. But in the end, the 386 finished ahead of schedule, an almost unheard-of accomplishment. 1

The 386's standard cell circuitry contains a few circuits that I didn't expect. In this blog post, I'll take a quick look at some of these circuits: surprisingly large multiplexers, a transistor that doesn't fit into the standard cell layout, and inverters that turned out not to be inverters. (If you want more background on standard cells in the 386, see my earlier post, " Reverse engineering standard cell logic in the Intel 386 processor ".)

The photo below shows the 386 die with the automatic-place-and-route regions highlighted; I'm focusing on the red region in the lower right. These blocks of logic have cells arranged in rows, giving them a characteristic striped appearance. The dark stripes are the transistors that make up the logic gates, while the lighter regions between the stripes are the "routing channels" that hold the wiring that connects the cells. In comparison, functional blocks such as the datapath on the left and the microcode ROM in the lower right were designed manually to optimize density and performance, giving them a more solid appearance.

The 386 die with the standard-cell regions highlighted.

The 386 die with the standard-cell regions highlighted.

As for other features on the chip, the black circles around the border are bond wire connections that go to the chip's external pins. The chip has two metal layers, a small number by modern standards, but a jump from the single metal layer of earlier processors such as the 286. (Providing two layers of metal made automated routing practical: one layer can hold horizontal wires while the other layer can hold vertical wires.) The metal appears white in larger areas, but purplish where circuitry underneath roughens its surface. The underlying silicon and the polysilicon wiring are obscured by the metal layers.

The giant multiplexers

The standard cell circuitry that I'm examining (red box above) is part of the control logic that selects registers while executing an instruction. You might think that it is easy to select which registers take part in an instruction, but due to the complexity of the x86 architecture, it is more difficult. One problem is that a 32-bit register such as EAX can also be treated as the 16-bit register AX, or two 8-bit registers AH and AL. A second problem is that some instructions include a "direction" bit that switches the source and destination registers. Moreover, sometimes the register is specified by bits in the instruction, but in other cases, the register is specified by the microcode. Due to these factors, selecting the registers for an operation is a complicated process with many cases, using control bits from the instruction, from the microcode, and from other sources.

Three registers need to be selected for an operation—two source registers and a destination register—and there are about 17 cases that need to be handled. Registers are specified with 7-bit control signals that select one of the 30 registers and control which part of the register is accessed. With three control signals, each 7 bits wide, and about 17 cases for each, you can see that the register control logic is large and complicated. (I wrote more about the 386's registers here .)

I'm still reverse engineering the register control logic, so I won't go into details. Instead, I'll discuss how the register control circuit uses multiplexers, implemented with standard cells. A multiplexer is a circuit that combines multiple input signals into a single output by selecting one of the inputs. 2 A multiplexer can be implemented with logic gates, for instance, by ANDing each input with the corresponding control line, and then ORing the results together. However, the 386 uses a different approach—CMOS switches—that avoids a large AND/OR gate.

Schematic of a CMOS switch.

Schematic of a CMOS switch.

The schematic above shows how a CMOS switch is constructed from two MOS transistors. When the two transistors are on, the output is connected to the input, but when the two transistors are off, the output is isolated. An NMOS transistor is turned on when its input is high, but a PMOS transistor is turned on when its input is low . Thus, the switch uses two control inputs, one inverted. The motivation for using two transistors is that an NMOS transistor is better at pulling the output low, while a PMOS transistor is better at pulling the output high, so combining them yields the best performance. 3 Unlike a logic gate, the CMOS switch has no amplification, so a signal is weakened as it passes through the switch. As will be seen below, inverters can be used to amplify the signal.

The image below shows how CMOS switches appear under the microscope. This image is very hard to interpret because the two layers of metal on the 386 are packed together densely, but you can see that some wires run horizontally and others run vertically. The bottom layer of metal (called M1) runs vertically in the routing area, as well as providing internal wiring for a cell. The top layer of metal (M2) runs horizontally; unlike M1, the M2 wires can cross a cell. The large circles are vias that connect the M1 and M2 layers, while the small circles are connections between M1 and polysilicon or M1 and silicon. The central third of the image is a column of standard cells with two CMOS switches outlined in green. The cells are bordered by the vertical ground rail and +5V rail that power the cells. The routing areas are on either side of the cells, holding the wiring that connects the cells.

Two CMOS switches, highlighted in green. The lower switch is flipped vertically compared to the upper switch.

Two CMOS switches, highlighted in green. The lower switch is flipped vertically compared to the upper switch.

Removing the metal layers reveals the underlying silicon with a layer of polysilicon wiring on top. The doped silicon regions show up as dark outlines. I've drawn the polysilicon in green; it forms a transistor (brighter green) when it crosses doped silicon. The metal ground and power lines are shown in blue and red, respectively, with other metal wiring in purple. The black dots are vias between layers. Note how metal wiring (purple) and polysilicon wiring (green) are combined to route signals within the cell. Although this standard cell is complicated, the important thing is that it only needs to be designed once. The standard cells for different functions are all designed to have the same width, so the cells can be arranged in columns, snapped together like Lego bricks.

A diagram showing the silicon for a standard-cell switch. The polysilicon is shown in green. The bottom metal is shown in blue, red, and purple.

A diagram showing the silicon for a standard-cell switch. The polysilicon is shown in green. The bottom metal is shown in blue, red, and purple.

To summarize, this switch circuit allows the input to be connected to the output or disconnected, controlled by the select signal. This switch is more complicated than the earlier schematic because it includes two inverters to amplify the signal. The data input and the two select lines are connected to the polysilicon (green); the cell is designed so these connections can be made on either side. At the top, the input goes through a standard two-transistor inverter. The lower left has two transistors, combining the NMOS half of an inverter with the NMOS half of the switch. A similar circuit on the right combines the PMOS part of an inverter and switch. However, because PMOS transistors are weaker, this part of the circuit is duplicated.

A multiplexer is constructed by combining multiple switches, one for each input. Turning on one switch will select the corresponding input. For instance, a four-to-one multiplexer has four switches, so it can select one of the four inputs.

A four-way multiplexer constructed from CMOS switches and individual transistors.

A four-way multiplexer constructed from CMOS switches and individual transistors.

The schematic above shows a hypothetical multiplexer with four inputs. One optimization is that if an input is always 0, the PMOS transistor can be omitted. Likewise, if an input is always 1, the NMOS transistor can be omitted. One set of select lines is activated at a time to select the corresponding input. The pink circuit selects 1, green selects input A, yellow selects input B, and blue selects 0. The multiplexers in the 386 are similar, but have more inputs.

The diagram below shows how much circuitry is devoted to multiplexers in this block of standard cells. The green, purple, and red cells correspond to the multiplexers driving the three register control outputs. The yellow cells are inverters that generate the inverted control signals for the CMOS switches. This diagram also shows how the automatic layout of cells results in a layout that appears random.

A block of standard-cell logic with multiplexers highlighted. The metal and polysilicon layers were removed for this photo, revealing the silicon transistors.

A block of standard-cell logic with multiplexers highlighted. The metal and polysilicon layers were removed for this photo, revealing the silicon transistors.

The misplaced transistor

The idea of standard-cell logic is that standardized cells are arranged in columns. The space between the cells is the "routing channel", holding the wiring that links the cells. The 386 circuitry follows this layout, except for one single transistor, sitting between two columns of cells.

The "misplaced" transistor, indicated by the arrow. The irregular green regions are oxide that was incompletely removed.

The "misplaced" transistor, indicated by the arrow. The irregular green regions are oxide that was incompletely removed.

I wrote some software tools to help me analyze the standard cells. Unfortunately, my tools assumed that all the cells were in columns, so this one wayward transistor caused me considerable inconvenience.

The transistor turns out to be a PMOS transistor, pulling a signal high as part of a multiplexer. But why is this transistor out of place? My hypothesis is that the transistor is a bug fix. Regenerating the cell layout was very costly, taking many hours on an IBM mainframe computer. Presumably, someone found that they could just stick the necessary transistor into an unused spot in the routing channel, manually add the necessary wiring, and avoid the delay of regenerating all the cells.

The fake inverter

The simplest CMOS gate is the inverter, with an NMOS transistor to pull the output low and a PMOS transistor to pull the output high. The standard cell circuitry that I examined contains over a hundred inverters of various sizes. (Performance is improved by using inverters that aren't too small but also aren't larger than necessary for a particular circuit. Thus, the standard cell library includes inverters of multiple sizes.)

The image below shows a medium-sized standard-cell inverter under the microscope. For this image, I removed the two metal layers with acid to show the underlying polysilicon (bright green) and silicon (gray). The quality of this image is poor—it is difficult to remove the metal without destroying the polysilicon—but the diagram below should clarify the circuit. The inverter has two transistors: a PMOS transistor connected to +5 volts to pull the output high when the input is 0, and an NMOS transistor connected to ground to pull the output low when the input is 1. (The PMOS transistor needs to be larger because PMOS transistors don't function as well as NMOS transistors due to silicon physics.)

An inverter as seen on the die. The corresponding standard cell is shown below.

An inverter as seen on the die. The corresponding standard cell is shown below.

The polysilicon input line plays a key role: where it crosses the doped silicon, a transistor gate is formed. To make the standard cell more flexible, the input to the inverter can be connected on either the left or the right; in this case, the input is connected on the right and there is no connection on the left. The inverter's output can be taken from the polysilicon on the upper left or the right, but in this case, it is taken from the upper metal layer (not shown). The power, ground, and output lines are in the lower metal layer, which I have represented by the thin red, blue, and yellow lines. The black circles are connections between the metal layer and the underlying silicon.

This inverter appears dozens of times in the circuitry. However, I came across a few inverters that didn't make sense. The problem was that the inverter's output was connected to the output of a multiplexer. Since an inverter is either on or off, its value would clobber the output of the multiplexer. 4 This didn't make any sense. I double- and triple-checked the wiring to make sure I hadn't messed up. After more investigation, I found another problem: the input to a "bad" inverter didn't make sense either. The input consisted of two signals shorted together, which doesn't work.

Finally, I realized what was going on. A "bad inverter" has the exact silicon layout of an inverter, but it wasn't an inverter: it was independent NMOS and PMOS transistors with separate inputs. Now it all made sense. With two inputs, the input signals were independent, not shorted together. And since the transistors were controlled separately, the NMOS transistor could pull the output low in some circumstances, the PMOS transistor could pull the output high in other circumstances, or both transistors could be off, allowing the multiplexer's output to be used undisturbed. In other words, the "inverter" was just two more cases for the multiplexer.

The "bad" inverter. (Image is flipped vertically for comparison with the previous inverter.)

The "bad" inverter. (Image is flipped vertically for comparison with the previous inverter.)

If you compare the "bad inverter" cell below with the previous cell, they look almost the same, but there are subtle differences. First, the gates of the two transistors are connected in the real inverter, but disconnected by a small gap in the transistor pair. I've indicated this gap in the photo above; it is hard to tell if the gap is real or just an imaging artifact, so I didn't spot it. The second difference is that the "fake" inverter has two input connections, one to each transistor, while the inverter has a single input connection. Unfortunately, I assumed that the two connections were just a trick to route the signal across the inverter without requiring an extra wire. In total, this cell was used 32 times as a real inverter and 9 times as independent transistors.

Conclusions

Standard cell logic and automatic place and route have a long history before the 386, back to the early 1970s, so this isn't an Intel invention. 5 Nonetheless, the 386 team deserves the credit for deciding to use this technology at a time when it was a risky decision. They needed to develop custom software for their placing and routing needs, so this wasn't a trivial undertaking. This choice paid off and they completed the 386 ahead of schedule. The 386 ended up being a huge success for Intel, moving the x86 architecture to 32 bits and defining the dominant computer architecture for the rest of the 20th century.

If you're interested in standard cell logic, I also wrote about standard cell logic in an IBM chip . I plan to write more about the 386, so follow me on Mastodon , Bluesky , or RSS for updates. Thanks to Pat Gelsinger and Roxanne Koester for providing helpful papers.

For more on the 386 and other chips, follow me on Mastodon ( @ [email protected] ), Bluesky ( @righto.com ), or RSS . (I've given up on Twitter.) If you want to read more about the 386, I've written about the clock pin , prefetch queue , die versions , packaging , and I/O circuits .

Notes and references

The Talk Show: ‘Lincoln Bio Services’

Daring Fireball
daringfireball.net
2025-11-22 17:17:19
For your weekend listening enjoyment: a new episode of America’s favorite 3-star podcast, with special guest Stephen Robles. Topics include indie media and YouTube, Shortcuts and automation, and the state of podcasting. Sponsored by: Uncommon Goods: Out of the ordinary gifts, great for the holi...
Original Article

The Talk Show

‘Lincoln Bio Services’, With Stephen Robles

Special guest Stephen Robles joins the show. Topics include indie media and YouTube, Shortcuts and automation, and the state of podcasting.

Sponsored by:

  • Uncommon Goods : Out of the ordinary gifts, great for the holidays. Save 15% off your next purchase after following that link.

Transcripts: Unofficial but surprisingly good .

Links:

This episode of The Talk Show was edited by Caleb Sexton.

The privacy nightmare of browser fingerprinting

Hacker News
kevinboone.me
2025-11-22 17:08:36
Comments...
Original Article

I imagine that most people who take an interest in de-Googling are concerned about privacy. Privacy on the Internet is a somewhat nebulous concept, but one aspect of privacy is surely the prevention of your web browsing behaviour being propagated from one organization to another. I don’t want my medical insurers to know, for example, that I’ve been researching coronary artery disease. And even though my personal safety and liberty probably aren’t at stake, I don’t want to give any support to the global advertising behemoth, by allowing advertisers access to better information about me.

Unfortunately, while distancing yourself from Google and its services might be a necessary first step in protecting your privacy, it’s far from the last. There’s more to do, and it’s getting harder to do it, because of browser fingerprinting.

How we got here

Until about five years ago, our main concern surrounding browser privacy was probably the use of third-party tracking cookies. The original intent behind cookies was that they would allow a web browser and a web server to engage in a conversation over a period of time. The HTTP protocol that web servers use is stateless ; that is, each interaction between browser and server is expected to be complete in itself. Having the browser and the server exchange a cookie (which could just be a random number) in each interaction allowed the server to associate each browser with an ongoing conversation. This was, and is, a legitimate use of cookies, one that is necessary for almost all interactive web-based services. If the cookie is short-lived, and only applies to a single conversation with a single web server, it’s not a privacy concern.

Unfortunately, web browsers for a long time lacked the ability to distinguish between privacy-sparing and privacy-breaking uses of cookies. If many different websites issue pages that contain links to the same server – usually some kind of advertising service – then the browser would send cookies to that server, thinking it was being helpful. This behaviour effectively linked web-based services together, allowing them to share information about their users. The process is a bit more complicated than I’m making it out to be, but these third-party cookies were of such concern that, in Europe at least, legislation was enacted to force websites to disclose that they were using them.

Browsers eventually got better at figuring out which cookies were helpful and which harmful and, for the most part, we don’t need to be too concerned about ‘tracking cookies’ these days. Not only can browsers mitigate their risks, there’s a far more sinister one: browser fingerprinting.

Browser fingerprinting

Browser fingerprinting does not depend on cookies. It’s resistant, to some extent, to privacy measures like VPNs. Worst of all, steps that we might take to mitigate the risk of fingerprinting can actually worsen the risk. It’s a privacy nightmare, and it’s getting worse.

Fingerprinting works by having the web server extract certain discrete elements of information from the browser, and combining those elements into a numerical identifier. Some of the information supplied by the browser is fundamental and necessary and, although a browser could fake it, such a measure is likely to break the website.

For example, a fingerprinting system knows, just from information that my browser always supplies (and probably has to), that I’m using version 144 of the Firefox browser, on Linux; my preferred language is English, and my time-zone is GMT. That, by itself, isn’t enough information to identify me uniquely, but it’s a step towards doing so.

To get more information, the fingerprinter needs to use more sophisticated methods which the browser could, in theory, block. For example, if the browser supports JavaScript – and they nearly all do – then the fingerprinter can figure out what fonts I have installed, what browser extensions I use, perhaps even what my hardware is. Worst of all, perhaps, it can extract a canvas fingerprint . Canvas fingerprinting works by having the browser run code that draws text (perhaps invisibly), and then retrieving the individual pixel data that it drew. This pixel data will differ subtly from one system to another, even drawing the same text, because of subtle differences in the graphics hardware and the operating system.

It appears that only about one browser in every thousand share the same canvas fingerprint. Again, this alone isn’t enough to identify me, but it’s another significant data point.

Fingerprinting can make use of even what appears to be trivial information. If, for example, I resize my browser window, the browser will probably make the next window the same size. It will probably remember my preference from one day to the next. If the fingerprinter knows my preferred browser window size is, say, 1287x892 pixels, that probably narrows down the search for my identify by a factor of a thousand or more.

Why crude methods to defeat fingerprinting don’t work

You might think that a simple way to prevent, or at least hamper, fingerprinting would be simply to disable JavaScript support in the browser. While this does defeat measures like canvas fingerprinting, it generates a significant data point of its own: the fact that JavaScript is disabled. Since almost every web browser in the world now supports JavaScript, turning it off as a measure to protect privacy is like going to the shopping mall wearing a ski mask. Sure, it hides your identify; but nobody’s going to want to serve you in stores. And disabling JavaScript will break many websites, including some pages on this one, because I use it to render math equations.

Less dramatic approaches to fingerprinting resistance have their own problems. For example, a debate has long raged about whether a browser should actually identify itself at all. The fact that I’m running Firefox on Linux probably puts me in a small, easily identified group. Perhaps my browser should instead tell the server I’m running Chrome on Windows? That’s a much larger group, after all.

The problem is that the fingerprinters can guess the browser and platform with pretty good accuracy using other methods, whether the browser reports this information or not. If the browser says something different to what the fingerprinter infers, we’re back in ski-mask territory.

What about more subtle methods to spoof the client’s behaviour? Browsers (or plug-ins) can modify the canvas drawing procedures, for example, to spoof the results of canvas fingerprinting. Unfortunately, these methods leave traces of their own, if they aren’t applied subtly. What’s more, if they’re applied rigorously enough to be effective, they can break websites that rely on them for normal operation.

All in all, browser fingerprinting is very hard to defeat, and organizations that want to track us have gotten disturbingly good at it.

Is there any good news?

Not much, frankly.

Before sinking into despondency, it’s worth bearing in mind that websites that attempt to demonstrate the efficacy of fingerprinting, like amiunique and fingerprint.com do not reflect how fingerprinting works in the real world. They’re operating on comparatively small sets of data and, for the most part, they’re not tracking users over days. Real-world tracking is much harder than these sites make it out to be. That’s not to say it’s too hard but it is, at best, a statistical approach, rather than an exact one.

Oh, bugger. That’s something I don’t want to see from amiunique.org

In addition ‘uniqueness’, in itself, is not a strong measure of traceability. That my browser fingerprint is unique at some point in time is irrelevant if my fingerprint will be different tomorrow, whether it remains unique within the fingerprinter’s database or not.

Of course, these facts also mean that it’s difficult to assess the effectiveness of our countermeasures: our assessment can only be approximate, because we don’t actually know what real fingerprinters are doing.

Another small piece of good news is that browser developers are starting to realize how much of a hazard fingerprinting is, and to integrate more robust countermeasures. We don’t necessarily need to resort to plug-ins and extensions, which are themselves detectable and become part of the fingerprint. At present, Brave and Mullvad seems to be doing the most to resist fingerprinting, albeit in different ways. Librewolf has the same fingerprint resistance as Firefox, but it is turned on by default. Probably anti-fingerprinting methods will improve over time but, of course, the fingerprinters will get better at what they do, too.

So what can we do?

First, and most obviously, if you care about avoiding tracking, you must prevent long-lived cookies hanging around in the browser, and you must use a VPN. Ideally the VPN should rotate its endpoint regularly.

The fact that you’re using a VPN, of course, is something that the fingerprinters will know, and it is does make you stand out. Sophisticated fingerprinters won’t be defeated by a VPN alone. But if you don’t use a VPN, the trackers don’t even need to fingerprint you: your IP number, combined with a few other bits of routine information, will identify you immediately, and with near-certainty.

Many browsers can be configured to remove cookies when they seem not to be in use; Librewolf does this by default, and Firefox and Chrome do it in ‘incognito’ mode. The downside, of course, is that long-lived cookies are often used to store authentication status so, if you delete them, you’ll find yourself having to log in every time you look at a site that requires authentication. To mitigate this annoyance, browsers generally allow particular sites to be excluded from their cookie-burning policies.

Next, you need to be as unremarkable as possible. Fingerprinting is about uniqueness, so you should use the most popular browser on the most popular operating system on the kind of hardware you can buy from PC World. If you’re running the latest Chrome on the latest Windows 11 on a two-year-old, bog-standard laptop, you’re going to be one of a very large group. Of course Chrome, being a Google product, has its own privacy concerns, so you might be better off using a Chromium-based browser with reduced Google influence, like Brave.

You should endeavour to keep your computer in as near its stock configuration as possible. Don’t install anything (like fonts) that are reportable by the browser. Don’t install any extensions, and don’t change any settings. Use the same ‘light’ theme as everybody else, and use the browser with a maximized window, and always the same size. And so on.

If possible, use a browser that has built-in fingerprint resistance, like Mullvad or Librewolf (or Firefox with these features turned on).

If you take all these precautions, you can probably reduce the probability that you can be tracked by you browser fingerprint, over days or weeks, from about 99% to about 50%.

50% is still too high, of course.

The downsides of resisting fingerprinting

If you enable fingerprinting resistance in Firefox, or use Librewolf, you’ll immediately encounter oddities. Most obviously, every time you open a new browser window, it will be the same size. Resizing the window may have odd results, as the browser will try to constrain certain screen elements to common size multiples. In addition, you won’t be able to change the theme.

You’ll probably find yourself facing more ‘CAPTCHA’ and similar identity challenges, because your browser will be unknown to the server. Websites don’t do this out of spite: hacking and fraud are rife on the Internet, and the operators of web-based services are rightly paranoid about client behaviour.

You’ll likely find that some websites just don’t work properly, in many small ways: wrong colours, misplaced text, that kind of thing. I’ve found these issues to be irritations rather than show-stoppers, but you might discover otherwise.

The short answer, I think, is that nobody knows, even within a specific jurisdiction. In the UK, the Information Commissioner’s Office takes a dim view of it , and it probably violates the spirit of the GDPR, if not the letter.

The GDPR is, for the most part, technologically neutral, although it has specific provisions for cookies, which were a significant concern at the time it was drafted. So far as I know, nobody has yet challenged browser fingerprinting under the GDPR, even though it seems to violate the provisions regarding consent. Since there are legitimate reasons for fingerprinting, such as hacking detection, organizations that do it could perhaps defend against a legal challenge on the basis that fingerprinting is necessary to operate their services safely. In the end, we really need specific, new legislation to address this privacy threat.

I suspect that many people who take an interest in Internet privacy don’t appreciate how hard it is to resist browser fingerprinting. Taking steps to reduce it leads to inconvenience and, with the present state of technology, even the most intrusive approaches are only partially effective. The data collected by fingerprinting is invisible to the user, and stored somewhere beyond the user’s reach.

On the other hand, browser fingerprinting produces only statistical results, and usually can’t be used to track or identify a user with certainty. The data it collects has a relatively short lifespan – days to weeks, not months or years. While it probably can be used for sinister purposes, my main concern is that it supports the intrusive, out-of-control online advertising industry, which has made a wasteland of the Internet.

In the end, it’s probably only going to be controlled by legislation and, even when that happens, the advertisers will seek new ways to make the Internet even more of a hellscape – they always do.

The worst programming language of all time

Lobsters
www.youtube.com
2025-11-22 16:53:12
Comments...

Our babies were taken after 'biased' parenting test

Hacker News
www.bbc.co.uk
2025-11-22 16:49:15
Comments...
Original Article

Our babies were taken after 'biased' parenting test - now we're fighting to get them back

Keira, who has long, dark hair in a ponytail and is wearing a coat with a furry collar, stares into the distance. Behind her is a body of water, probably the sea.

Image caption,

Keira says she sobbed uncontrollably when her baby was taken away from her

By Sofia Bettiza BBC Global Health Reporter and Woody Morris BBC World Service Reporting from Denmark

When Keira's daughter was born last November, she was given two hours with her before the baby was taken into care.

"Right when she came out, I started counting the minutes," Keira, 39, recalls.

"I kept looking at the clock to see how long we had."

When the moment came for Zammi to be taken from her arms, Keira says she sobbed uncontrollably, whispering "sorry" to her baby.

"It felt like a part of my soul died."

Now Keira is one of many Greenlandic families living on the Danish mainland who are fighting to get their children returned to them after they were removed by social services.

In such cases, babies and children were taken away after parental competency tests - known in Denmark as FKUs - were used to help assess whether they were fit to be parents.

In May this year the Danish government banned the use of these tests on Greenlandic families after decades of criticism, although they continue to be used on other families in Denmark.

The assessments, which usually take months to complete, are used in complex welfare cases where authorities believe children are at risk of neglect or harm.

Keira is lying down, cradling her newborn baby Zammi, who's covered in a blanket, into her chest.

Image caption,

Keira says she was "counting the minutes" from the moment Zammi was born, knowing she only had two hours with her daughter

They include interviews with parents and children, a range of cognitive tasks, such as recalling a sequence of numbers backwards, general knowledge quizzes, and personality and emotional testing.

Defenders of the tests say they offer a more objective method of assessment than the potentially anecdotal and subjective evidence of social workers and other experts.

But critics say they cannot meaningfully predict whether someone will make a good parent.

Opponents have also long argued that they are designed around Danish cultural norms and point out they are administered in Danish, rather than Kalaallisut, the mother tongue of most Greenlanders.

This can lead to misunderstandings, they say.

The Battle to Get My Child Back

Greenlandic parents across Denmark fight to be reunited with their children.

Greenlanders are Danish citizens, enabling them to live and work on the mainland.

Thousands live in Denmark, drawn by its employment opportunities, education and healthcare, among other reasons.

Greenlandic parents in Denmark are 5.6 times more likely to have children taken into care than Danish parents, according to the Danish Centre for Social Research, a government-funded research institute.

In May, the government said it hoped in due course to review around 300 cases – including ones involving FKU tests – in which Greenlandic children were forcibly removed from their families.

But as of October, the BBC found that just 10 cases where parenting tests were used had been reviewed by the government - and no Greenlandic children had been returned as a result.

Keira's assessment in 2024, carried out when she was pregnant, concluded that she did not have "sufficient parental competencies to care for the newborn independently".

Keira says the questions she was asked included: "Who is Mother Teresa?" and "How long does it take for the sun's rays to reach the Earth?"

A baby cot is pictured in a room inside Keira's apartment. In the middle of the cot are baby clothes and blankets and photos of Zammi.

Image caption,

Keira still keeps a cot beside her bed and another in the living room of her apartment, along with baby clothes and nappies

Psychologists who defend the tests argue questions like these are intended to assess parents' general knowledge and their understanding of concepts they might encounter in society.

Keira adds that "they made me play with a doll and criticised me for not making enough eye contact".

She alleges that when she asked why she was being tested in this way the psychologist told her: "To see if you are civilised enough, if you can act like a human being."

The local authority in Keira's case said it could not comment on individual families, adding that decisions to place a child in care were made when there was serious concern about the "child's health, development, and well-being".

In 2014, Keira's other two children - who were then aged nine years and eight months - were placed into care after an FKU test at the time concluded her parenting skills were not developing fast enough to meet their needs.

Her eldest, Zoe, who is now 21, moved back home when she was 18 and currently lives in her own apartment and sees her mum regularly.

Keira hopes she will soon be reunited with her baby Zammi permanently.

The Danish government has said its review will look at whether mistakes were made in the administering of FKU tests on Greenlandic people.

In the meantime, Keira is allowed to see Zammi, who is in foster care, once a week for an hour.

Each time she visits, she takes flowers and sometimes Greenlandic food, such as chicken heart soup.

"Just so a little part of her culture can be with her," she says.

'I felt the most horrific heartbreak'

Ulrik and Johanne are pictured outdoors. In the background, are a couple of buildings, some greenery and a path sloping upwards. Ulrik is dressed in a green jacket with a red T-shirt underneath while Johanne is wearing an open-neck patterned dress.

Image caption,

Ulrik and Johanne hope the Danish government will reconsider reviewing cases like theirs where a child has been adopted

But not all Greenlandic parents who had children taken into care after completing FKUs will have their cases reviewed.

Johanne and Ulrik's son was adopted in 2020 and the Danish government has said it will not review cases where children have been adopted.

Johanne, 43, was tested in 2019 during pregnancy.

Like Zammi, her son was meant to have been taken away immediately after birth.

But because he was born prematurely on Boxing Day and social workers were on holiday, she and her husband Ulrik got to keep him for 17 days.

"It was the happiest time of my life as a father," says Ulrik, 57.

"Being with my son, holding him, changing his nappy, making sure that Johanne pumps her milk before going to bed in the evening."

Then one day, two social workers and two police officers arrived at Johanne and Ulrik's home to take their son away.

The couple say they pleaded with them not to take him.

Johanne asked if she could breastfeed him one last time.

"As I was dressing my son to hand him over to his foster parents who were on their way, I felt the most horrific heartbreak," Ulrik says.

Johanne had been tested after two children from another relationship, who were five and six, were taken into care after FKU testing in 2010.

Her 2019 assessment describes her as "narcissistic" and as having "mental retardation" - a categorisation based on designations developed by the WHO which were in use at the time.

She rejects both of these descriptions of her.

Protesters, dressed in thick winter coats with snow in the background, carry placards during a demonstration in Nuuk, Greenland. The protester at the front of the image has tape covering her mouth. Image source, Getty Images

Image caption,

A protester carries a placard that reads: "Our children are watching!! Prejudices are contagious," during a demonstration in Nuuk, Greenland's capital, earlier this year

In theory, there is no pass or fail mark for an FKU and they are one factor among others taken into consideration by local authorities who decide whether to place a child into care.

But psychologist Isak Nellemann, who used to administer the tests, says in practice they "are very important, about the most important thing, because when the tests are bad, in about 90% [of cases] they will lose their children".

Nelleman argues the tests lack scientific validity and were developed to study personality traits rather than predict parenting ability.

However, Turi Frederiksen, a senior psychologist whose team currently administers the tests, defends them, saying that while they are not perfect, "they are valuable, extensive psychological tools".

She also says she does not believe they are biased against Greenlanders.

When Johanne was asked in 2019 what she saw during a Rorschach test - a psychological test where people are asked what they see when looking at ink-blot images - she said she saw a woman gutting a seal, a familiar sight in Greenland's hunting culture.

Johanne alleges that on hearing this answer the psychologist called her a "barbarian".

The local council involved in the couple's 2019 assessment did not address Johanne's claim directly.

They said her assessment "indicated significant concern regarding the parents' overall parenting abilities" as well as "concerns about the parents' general lifestyle and functional level in daily life".

A woman with shoulder-length wavy blonde hair, Tordis Jacobsen, looks at the camera. She is wearing a black jumper and is sitting next to a desk with a computer in an office.

Image caption,

Social worker Tordis Jacobsen said the decision to place a child into care in Denmark was never taken lightly

'I never got to see his first steps'

After Johanne and Ulrik's son was taken into care, they were allowed to see him during brief, weekly visits until he was adopted in 2020.

They have never seen him since.

"I never got to see his first steps, his first word, his first tooth, his first school day," Johanne says.

However, a few days after his birth they christened him, creating an official record that includes their names and address.

"We needed to create a paper trail so he could find his way back to us," Johanne says.

Their lawyer Jeanette Gjørret hopes to take their case before the European Court of Human Rights.

But Denmark's social affairs minister Sophie Hæstorp Andersen tells the BBC the government will not reopen cases of adoption because each of these children is now settled with a "loving and caring family".

Asked about the progress of the review, she says "it sounds slow, but we are getting started".

She also says decisions to remove and adopt children are part of a "very thorough process where we look into the family's ability to take care of their child not only for a year or two, but for a long period of time".

That is echoed by Tordis Jacobsen, a social worker team leader in Aalborg Kommune in northern Denmark, who says removing a child in Denmark is never taken lightly.

She says safeguarding concerns are often first flagged by schools or hospitals, and points out that in cases where a child is permanently adopted the decision to approve this is made by a judge.

Pilinguaq, who is dressed in a cream jumper with dangly earings, holds her daughter, who is wearing a striped pink, purple and blue top. The daughter is hugging her mother around her shoulders, while in the background there are blurred trees and bushes in front of a house.

Image caption,

Pilinguaq's daughter, six, was returned to her several months ago, more than four years after being placed into care

Pilinguaq is a rare case of a Greenlandic mother who has been reunited with her child.

She and her daughter, who was placed into care aged one, were reunited a few months ago. Her daughter is now six.

Pilinguaq, 39, says she received the unexpected news in a phone call from social services.

"I started crying and laughing at the same time. I couldn't believe it. I kept thinking, 'Oh my God, she's coming home.'"

Pilinguaq's three children were all placed into care in 2021. The other two were aged six and nine at the time.

She says she agreed for her local authority to place her children in temporary care while she found a new home suitable for her children.

Pilinguaq says she believed her children would soon be returned to her, but instead she had to undergo a parenting assessment.

This concluded she had a pattern of entering "dysfunctional relationships" and was unfit to parent.

'They can take her in one hour'

A few months after her six-year-old daughter came home, Pilinguaq was told by her local authority that her other two older children will be returning to her in December.

The decision to return the children into Pilinguaq's care was made by the local authority rather than being recommended by the government review. The local authority declined to comment on her case.

Spending more than four years apart has made it difficult for Pilinguaq to rebuild her relationship with her daughter.

"If I go to the bathroom and close the door, she will have a panic attack and say 'Mum, I couldn't find you,'" Pilinguaq says.

She also says she is terrified of losing her daughter again.

"They can take her in one hour. They can do it again."

Keira, who is dressed in a red T-shirt and wearing protective gloves, is crouched behind the wooden sleigh she's been making. In the background are a couple of grey garden chairs and an open door leading into a home.

Image caption,

Keira has been making her daughter Zammi a wooden sleigh for her first birthday

Keira is now preparing for Zammi's first birthday in her absence.

She's building a traditional Greenlandic sleigh by hand from wood, with a polar bear drawn on the front.

Earlier this month, she was told that her daughter won't be coming home - for now at least - but she hasn't given up hope.

Keira still has a cot next to her bed and another in the living room, with framed photos of Zammi on the walls, along with baby clothes and nappies.

"I will not stop fighting for my children.

"If I don't finish this fight, it will be my children's fight in the future."

Graphic with Global Women written in white on a purple background, with blueish-purple arcs taken from concentrically arranged circles on the right.

  • This is part of the Global Women series from the BBC World Service, sharing untold and important stories from around the globe

Nydia Velázquez Hears Calls for Generational Change, Setting Up a Fight on the Left in New York

Intercept
theintercept.com
2025-11-22 16:35:49
The Democratic congresswoman was an early believer in Zohran Mamdani. His win showed her it was “the right time to pass the torch.” The post Nydia Velázquez Hears Calls for Generational Change, Setting Up a Fight on the Left in New York appeared first on The Intercept....
Original Article

Rep. Nydia Velázquez knew it was time to retire when Zohran Mamdani won the New York City mayoral race.

“What I saw during that election was that so many young people were hungry for a change and that they have a clear-eyed view of the problems we face and how to fix them,” Velázquez, D-N.Y., told The Intercept. “That helped convince me that this was the right time to pass the torch.”

Velázquez, a native of Puerto Rico who has served in Congress for more than 30 years, announced her retirement Thursday, in the early days of what is sure to be a frenzied 2026 midterm season across the country and in several solidly Democratic New York districts. She was not facing a notable primary challenger, unlike her House colleagues Hakeem Jeffries, Ritchie Torres, and Adriano Espaillat: three younger New York congressmen who are all considered firmly in line with the Democratic establishment, and all facing challenges from their left .

“She could be in that seat as long as she wants,” said Brooklyn Borough President Antonio Reynoso, a longtime ally whom Velázquez once described as one of her “ children .” “Nydia is at her peak. So that she would go out like that — it’s so Nydia.”

Velázquez is known as something of a den mother for a generation of younger progressive politicians in Brooklyn. She is overwhelmingly popular in her district but made few friends in the local establishment’s clubby machine politics. As Brooklyn’s electorate shifted left over the decades, she built up a formidable stable of protégés in key roles.

“My goal was to build a bench of strong, independent, progressive public servants who understood who they work for.”

“My goal was never to build a machine,” she said. “My goal was to build a bench of strong, independent, progressive public servants who understood who they work for.”

That will likely set up a competitive race to succeed Velázquez in her left-leaning 7th Congressional District, which includes Mamdani’s home base of Astoria, Queens, and solidly progressive Brooklyn neighborhoods like Bushwick, Williamsburg, and Clinton Hill. The district’s progressive profile means it’s poised to become a hot contest for candidates on the left — and may distract from the controversial candidacy of City Council Member Chi Ossé, who’s waging a long-shot challenge against Jeffries that has mired the city’s Democratic Socialists of America in debate .

Velázquez declined to say who, if anyone, she favored to become her replacement.

“I could leave today and know that the district will be in good hands,” she said.

Velázquez is bowing out at a moment when the “G word” — gerontocracy — can be heard frequently on cable news, and not just on the lips of younger political hopefuls frustrated by an aging party leadership . She joins fellow Democratic Rep. Jerry Nadler, who announced his decision to retire in September and who has already kicked off a wild, 10-way primary fight in his Upper West Side district.

“She wanted to send a message to Democrats across the country that it is time for the next generation.”

“She told me she wanted to send a message to Democrats across the country that it is time for the next generation,” said City Council Member Lincoln Restler, a protégé. “Still, every elected official I’ve spoken to is just sad that we’re losing this remarkable moral leader.”

Velázquez saw Mamdani’s promise so early in the mayoral race that she was predicting his win well before many of her younger acolytes did, Reynoso told The Intercept.

“Nydia was always like ‘Zohran is the one, and I think he can win,’” Reynoso said.

At Mamdani’s victory celebration on November 4, Velázquez was happy to flaunt her prediction. When one supporter joyfully asked if she could believe it, she replied : “I believed it a year ago.”

Velázquez, 72, was first elected in 1992, unseating a nine-term incumbent in the Democratic primary to become the first Puerto Rican woman to serve in Congress. At the time of her primary victory, the New York Times offered readers a guide to the phonetic pronunciation of her name.

“When Nydia Velázquez was first elected to Congress, it was her against the world,” said Restler. “She took on the chair of the Foreign Relations Committee, and the entrenched political power in Brooklyn was entirely against her.”

In 2010, Restler said, “she told me she felt genuinely lonely in Brooklyn, that she had so few allies that she could count on. Fifteen years later, essentially every single person in local and state elected office across her district is there because of her validation, her legitimization, and her support.”

In the wake of her announcement on Thursday, praise for Velázquez poured in not just from her mentors and close ideological allies, but also from establishment figures closer to the center as well. On X, New York Gov. Kathy Hochul called the outgoing congresswoman a “trailblazer” — a hint perhaps at the stable of potential left-wing contenders Velázquez has helped take the playing field over the years.

The Uncertain Origins of Aspirin

Hacker News
press.asimov.com
2025-11-22 16:23:06
Comments...
Original Article

On a cool starlit night in ancient Egypt, a wife brews willow-bark tea for her husband, hot with fever;

While Athens and Sparta wage war, Hippocrates hurries to the river to shave some more willow bark, intending to cool down another sick child brought to his door;

While George III marries Princess Charlotte, a son dashes off into the woods in search of willow bark to treat his mother’s worsening fever.

While these anecdotes are familiar from cinema and literature, the history of aspirin and the willow bark from which it is (supposedly) derived is far from clear.

For instance, it is often said that Hippocrates prescribed willow bark tea to treat inflammatory pain . In a perfect world, we would know this because a verified scroll from around 400 BC would exist, stating: “For inflammation, I recommend willow bark tea, and so does everyone I know. So sayeth Hippocrates of Kos, future father of medicine and writer of the eponymous oath.”

Of course, such primary material does not always come to light. And many who recount medical and scientific histories (or, indeed, any history at all) do so despite incomplete information. In these cases, however, it’s still important to cite what evidence there is: what, for example, has led so many people to believe Hippocrates prescribed willow bark tea for inflammatory pain? If the answer is “someone said so,” who was that someone, and why did they think it was true?

This history of one of the world’s most widely consumed classes of drugs demonstrates the difficulties that surround the fact-finding of its origins. Some of these difficulties are historic, such as bad record keeping or obscure translations, and some are of continued relevance, such as political upheaval or the prejudice that often interferes with science.

My goal is not only to tell the story of a pharmaceutical staple but to share what my decades-long work in evidence synthesis in epidemiology has taught me about researching scientific history. In doing so, I’ll differentiate claims that are known from original evidence — such as the writings of Hippocrates and Pliny the Elder or original journal articles — from those made without primary support. I have sourced as much of the latter as possible, but both conflicting and unsupported claims remain, which I’ll note.

The origins of aspirin make a good subject for exposing the challenges of scientific research for a couple of reasons. Firstly, the stories stretch back millennia, and secondly, aspirin is such a successful drug that there is ample material to work with. Aspirin and other non-steroidal anti-inflammatory drugs (NSAIDs) 1 are among the most widely prescribed drugs worldwide, 2 are safe enough that many are available without a prescription, and are some of the cheapest available drugs of any class. It’s often claimed that 30 million people take NSAIDs worldwide every day, although this assertion originally dates back to 1987 and wasn’t referenced . Nonetheless, if it were true, we would expect that number to be substantially higher now , given the increase in global population.

But before we can contend with the popularity of NSAIDs, we must first look at their origins.

{{signup}}

Ancient History: Antiquity to 1763 AD

When I was in medical school in 2007, I learned that willow bark tea has been used for thousands of years to treat pain and fever. I’ve read at least a handful of books — historical and fantasy alike — that make the same claim. The fifth Bridgerton book, To Sir Philip, With Love , includes a scene with an urgent search for willow bark to bring down a character’s fever. Diarmuid Jeffreys wrote an entire book on aspirin, The Remarkable Story of a Wonder Drug , which detailed how willow was used from Ancient Egypt, through Ancient Greece and Rome, to the Middle Ages in Europe, and beyond. This is repeated by journal articles , though their claims are rarely referenced.

Even if their sources are sketchy, these accounts seem plausible. Willow trees are relatively common, grow worldwide (though mostly in colder and wetter parts of the Northern Hemisphere), and have had various uses since antiquity. For example, a willow fishing net was found in Finland dating from 8,300 BC, and willow has also been used for wattle fences , wattle and daub houses , coracles , cricket bats , and even World War II parachute baskets .

The leaves and flowers of white willow, Salix alba , as sketched by Prof. Wilhelm Thomé in 1885.

A white willow tree grows in Poland. The village of Strożyska can be seen in the distance. Credit: Jakub Hałun .

However, the most plausible reason that willow bark could be the antecedent of aspirin is that it contains salicin (named for the genus of willow trees, Salix ). Salicin is converted into salicylic acid in the body when an enzyme bisects salicin, one half of which is then oxidized to become salicylic acid. Acetylsalicylic acid, known today as aspirin, is also converted into salicylic acid in the body: an enzyme simply cuts off the acetyl group to give salicylic acid. Salicylic acid is likely the active compound that blocks the enzyme cyclooxygenase , giving aspirin its efficacy, and was used as a treatment for fever and gout before aspirin was developed.

Remains of the Paleolithic Antrea net, which was uncovered in Finland in 1920.

It seems reasonable, therefore, to believe that willow bark tea could act like aspirin, reducing fever, inflammation, and pain. Willow grew in abundance across the world, and people have been using it for thousands of years. Additionally, myrtle and poplar trees, as well as meadowsweet , likewise contain salicin and could have been used for the same purpose.

The leaves and flowers of Meadowsweet, Spiraea alba , as sketched by Prof. Wilhelm Thomé in 1885.

However, there is a problem with the claim that willow bark and similar plants are the ancient equivalent of aspirin when brewed as tea or simply chewed. And it is this: most of the extant evidence referenced by histories of aspirin fail entirely to mention using willow bark or similar plants to reduce fever, pain, or inflammation.

When people talk about evidence of the ancients brewing or chewing willow and similar plants, they often invoke the Ebers papyrus , Hippocrates , Celsus , Pliny the Elder , and Pedanius Dioscorides . 3

The Ebers papyrus is an ancient Egyptian medical scroll from about 1550 BC, though it may have been copied from earlier texts. Translated by German Egyptologist Georg Ebers in 1875, the papyrus has 110 pages, is 20 meters long, and includes 700 remedies for various afflictions. As a piece of history, it’s fantastic. And some of the remedies are still practiced today: the treatment for Guinea-worm disease was, and still is , to wrap the emerging end of the worm around a stick and slowly pull it out. 4

A reproduction of the Ebers Papyrus. Credit: Wellcome Collection.

When I searched through a translation of the papyrus , however, I saw no evidence of willow bark used similarly to aspirin. I did find a treatment for an “ ear-that-discharges-foul-smelling-matter ” that used “ berry-of-the-willow” and a remedy to “ put the Heart into proper working order and make it take up nourishment ” that used “ Willow-tree, one-eighth part, added as a stiffening.

There was also a remedy to make the “met” supple (nerves or possibly blood vessels, Egyptologists appear uncertain ) that used “splinters-of-the-willow-tree,” although this particular remedy also required “hog’s dung” and “myrrh” among 34 other ingredients. Still, there was nothing about using willow bark specifically concerning pain, inflammation, or fever.

In his 1992 book Murder, Magic, and Medicine , author John Mann wrote that the Ebers papyrus contains the following remedy: “ When you examine a man with an irregular wound … and that wound is inflamed … [there is] a concentration of heat; the lips of that wound are reddened and that man is hot in consequence … then you must make cooling substances for him to draw the heat out … [from the] leaves of the willow ,” but I’ve been unable to find this in any translation of the papyrus, and Mann doesn’t reference the translation he claims to be using.

Uncertainty about the role of willow bark in treating inflammation remains as we move to Hippocrates, the ancient Greek physician and philosopher often regarded as the father of (clinical) medicine . 5 Pharmacology lecturer Philippa Martyr suggests the works ascribed to Hippocrates only reference willow once, specifically in the context of burning willow leaves to make smoke for fumigating the uterus to get rid of a miscarried pregnancy. Indeed, a French translation of Hippocrates’s complete works confirms this. I was, however, unable to find any reference to willow in other translations . It is also said that Hippocrates recommended chewing willow bark to relieve fever and pain, but this, too, lacked evidence.

A statue of Hippocrates in Mexico’s Centro Médico Nacional Siglo XXI. He is holding a Staff of Asclepius, the traditional symbol of medicine, in his right hand. Credit: Wikimedia .

Other ancient sources are similarly unclear. The Roman encyclopedist Celsus , who wrote De Medicina in the first century, suggested willow leaves boiled in vinegar could be used to treat ulcerations of the anus and prolapse of the anus and uterus (I’m not sure whether that would work, and it certainly isn’t a common use of aspirin today).

Pliny the Elder , a first-century Roman author and natural philosopher, expounds the uses of poplar and willow trees in his encyclopedia Natural History . For poplar trees, he suggests, “ A potion prepared from the bark is good for sciatica,” and “ The leaves, boiled in vinegar, are applied topically for gout .” And for willow: “ The leaves, too, boiled and beaten up with wax, are employed as a liniment for … gout.” This treatment is echoed by Pedanius Dioscorides , a Greek physician in the Roman army, also writing in the first century, who describes a similar treatment in his pharmacopeia, De materia medica . This is the earliest reference I have found to using salicin-containing plants as we’d potentially use aspirin today.

So ultimately, while we have a smattering of evidence that some version of a poplar bark potion may have been used for sciatica and that poplar and willow leaves may have been used for gout, we arrive at our second problem with the claim that willow bark and similar plants are the ancient equivalent of aspirin — that of effectiveness.

The amount of salicin in plants, even willow bark, is negligible compared to the aspirin dosages that we deem effective today. When researchers gave people willow bark extract corresponding to 240 mg of salicin, then looked at how much salicylic acid was present in their blood over time, it was the equivalent of taking 87 mg of aspirin ( 300 mg to 600 mg is recommended per dose, with up to 3600 mg allowed per day). Notably, 240 mg of salicin is the recommended daily dose specified by the European Scientific Cooperative on Phytotherapy .

But 240 mg of salicin today comes from concentrated and standardized willow bark extracts, not raw bark or leaves. The actual salicin concentration in willow bark is wildly variable, ranging between 0.04 percent and 12.06 percent across different species, according to one study . Salicin concentrations also vary across plants seasonally and based on a tree’s age. Only a fraction of the salicin would remain in tea or chewed bark.

If, for example, each cup of tea provided 240 mg salicin (possible with a good steeping and a high salicin content in the bark), then one would need to drink 41 cups of tea to get a full, therapeutic aspirin dose of 3600 mg. This is about 10 liters or two-and-a-half gallons.

Willow bark tea is also notoriously bitter , likely due to the high concentration of tannins, which can cause an upset stomach and nausea . Even if you could push through the bitterness, it’s unlikely you’d be able to stomach the bucketfuls of tea required to get enough salicin from willow bark (or similar plants) to ease your discomfort.

That’s not to say willow bark and similar plants couldn’t be effective in reducing pain, inflammation, and fever when steeped or chewed, however. Unlike aspirin, willow bark contains other compounds — flavonoids and polyphenols — that may contribute to alleviating such symptoms. However, few (if any) human studies have been conducted looking specifically at simply brewing or chewing these plants. While some research has been done on the efficacy of willow bark extract in the treatment of pain and inflammation, the extract is much more concentrated than in chewed bark, and results have varied.

Philippa Martyr adds an additional caveat when she notes :

If willow bark and leaves were handy and potent painkillers, we would have used them almost to extinction by now.

Overall, given patchy evidence and dubious efficacy, I am skeptical that the history of aspirin began in antiquity. The mythology of aspirin’s ancient history is, most likely, a coincidence: it’s easy to believe that because willow bark contains salicin, and it was possibly used for some of the same things as aspirin, and it was possibly effective, it must work, and therefore willow bark tea was ancient aspirin. It makes for a plausible story with historical continuity, but I believe it remains just that — a story.

For what I believe to be the most credible origins of aspirin and other NSAIDs, we must advance to 18th-century England.

Discovery of Aspirin: 1763 to 1877

The Reverend Edward Stone appears to have given the first published account of using willow bark extract to treat patients in Chipping Norton, England, in 1763. The patients were suffering from malarial fever (ague), which was still endemic to England at the time. He writes:

There is a bark of an Englifh tree, which I have found by experience to be a powerful aftringent, and very efficacious in curing aguifh [agues] and intermitting diforders.

Like many of his predecessors, Reverend Stone believed that the remedy to a malady could be found close to the cause. That is, if people were becoming sick in a certain place, there should be a plant or other remedy nearby. 6 As willow trees grew in abundance in “ moist and wet soil where agues chiefly abound, ” and because it was as bitter as “ Peruvian bark ” ( Chichona tree bark, which contains quinine, a traditional remedy for malaria), Reverend Stone thought he could use the dried, powdered bark to treat fevers. 7

A plaque in Oxfordshire commemorating Stone’s discovery. Unfortunately, we cannot find any photos of Stone online. Credit: Wikimedia .

Drying the bark increases its potency, which gets around the issues of having to drink bucketfuls of tea — similar to using willow bark extract today. For three months, Stone dried a pound of bark next to a baker’s oven before pulverizing it into a powder. He started small, giving his first patient “ twenty grains of powder ” in “ water, tea, small beer and such like, ” but quickly increased the dose after noticing no side effects. He treated fifty people over five years, all of whom he said were either cured or helped by the treatment.

Notably, Stone had investigated whether his treatment had precedent, but came up empty-handed:

My curiofity prompted me to look into the difpenfatories and books of botany, and examine what they faid concerning it; but there it exifted only by name. I could not find, that it hath, or ever had, any place in pharmacy, or any fuch qualities, as I fufpected afcribed to it by the botanifts.

Stone’s observation of the novelty of his treatment is corroborated by the medicinal herbalist Anne Stobart , who looked through 6,500 17th-century medicinal recipes for 40 plants with longstanding use in the UK, determining that willow had six or fewer mentions. 8 It seems fair to conclude that, at least in the UK in the 17 th and 18 th centuries, willow was not a well-known treatment for pain, fever, or inflammation. Stone documented his use of it, but it was by no means a widespread curative.

Around 1824, about 50 years after Stone’s investigation of willow bark powder, Italian pharmacist Bartolomeo Rigatelli extracted a “very bitter antipyretic” called “saline” from a plant native to Europe. The “saline” was subsequently renamed “salicin” by Francesco Fontana , who also extracted it from white willow and used it as a substitute for quinine sulfate. Fontana reported it was effective against fevers of all types, even “ quartan fevers ” (one of the four types of malaria). And while Fontana did not reference Rigatelli, he did reference someone named “Stone.” While there’s no way to be certain, it seems likely this was Reverend Stone.

The progression from salicin to aspirin would take over 70 years. In 1829, French chemist Henri Leroux refined the process used by Rigatelli, Fontana, and others and managed to extract pure salicin. His method was further refined in 1838 when Raffaele Pirìa , an Italian chemist, produced salicylic acid from salicin, after determining its molecular formula. By 1859, the German chemist Hermann Kolbe used the discovery of salicin’s molecular structure to synthesize salicylic acid directly. Kolbe’s process was refined by his assistant, Rudolf Wilhelm Schmitt, with the resulting process known as the Kolbe-Schmitt Reaction . 9

In the 1870s, salicylic acid took off as a treatment, and in 1874, Friedrich von Heyden , another student of Kolbe, opened a factory in Radebeul (near Dresden) to produce it, reportedly selling it ten times more cheaply than its naturally derived counterpart. This spurred research into its clinical applications. In 1876, a Scottish physician, Thomas MacLagan , and a German physician, Franz Stricker , both published studies showing that salicin and salicylic acid were effective in treating rheumatic fever, particularly in reducing fever and pain. A year later, in 1877, a French physician, Germain Sée , showed that chronic rheumatism and gout could also be treated with salicylic acid.

Salicylic acid had proven effective in reducing pain, fever, and inflammation, and it could be manufactured cheaply and in large quantities. However, it had several serious side effects , such as nausea, gastric irritation, and tinnitus. If a drug manufacturer were able to create a better alternative, patients and doctors would be grateful, and it would likely be extremely successful.

With their synthesis of aspirin, chemists at Bayer did exactly that.

Aspirin Synthesis Controversy: 1897 to 1949

While the events of the 18th and 19th centuries relate to a verifiable history of aspirin, the next period becomes muddied once again. However, here it does not relate to lost or convoluted records, but prejudice.

We know the following to be absolutely true: Felix Hoffman, a chemist working for Bayer, synthesized pure acetylsalicylic acid (later known as aspirin) from salicylic acid in 1897. His name is on the U.S. patent for aspirin, and his lab notebook details his synthesis of acetylsalicylic acid. Hoffman’s method was relatively simple, reliable, and efficient. Hoffman heated a combination of salicylic acid and acetic anhydride (an acetylating agent) for two hours, which resulted in a clear liquid. Upon cooling, this liquid yielded a mass of acetylsalicylic acid crystals, which he separated out and recrystallized, using dry chloroform to remove any remaining impurities.

The controversy, then, refers both to why Hoffman synthesized aspirin and what happened afterward.

A bottle of aspirin from 1899. Credit: Bayer AG.

On the one hand, we have the official story from Bayer , which claims that Hoffman created aspirin to help his father, who was suffering from severe rheumatism. Hoffman had treated his father with salicylic acid, but his father had severe side effects, including nausea, gastric irritation, and tinnitus. Hoffman, therefore, set out to produce a purer derivative of salicylic acid that would be as effective. This version of events was first reported in a footnote in a 1934 book on the history of chemical engineering . 10

On the other hand, we have a story from Arthur Eichengrün. Eichengrün was a Jewish chemist who also worked at Bayer. In 1949, he wrote a journal article describing why aspirin was synthesized, and what happened after. 11

Ernst Arthur Eichengrün, photographed in 1896.

Eichengrün’s laboratory at Bayer.

In this article, Eichengrün stated that he was appointed, in 1895, to establish and manage a pharmaceutical laboratory at Bayer. He also claimed that it was he who instructed Hoffman to synthesize acetylsalicylic acid in 1898 and that Hoffman had done so without knowing why. Notably, Eichengrün also claimed that several Bayer chemists made various derivatives of salicylic acid, each of which was tested further.

Drugs developed in Eichengrün’s laboratory were tested at Bayer’s pharmacological laboratory, led by Heinrich Dreser. 12 When the salicylic acid derivatives were tested, it was clear that acetylsalicylic acid was the most favorable, producing only minor deleterious effects on a frog heart. However, in a Bayer management meeting to discuss whether acetylsalicylic acid should go forward to clinical trials, Dreser asserted that it was a direct cardiac poison and opposed it progressing to trials. Dreser had the right to veto any drug going to clinical trials, so this is where the story of aspirin could have ended.

However, in this article, Eichengrün stated he couldn’t accept the decision to stop work on acetylsalicylic acid, and, against his contract, continued to conduct tests privately. He tested the drug on himself, then enlisted the help of doctors to test 100 grams of homemade acetylsalicylic acid. 13 None of the doctors reported side effects in their patients, and they asked for larger quantities for more detailed testing. From these tests, it was clear that acetylsalicylic acid retained the beneficial clinical effects of salicylic acid, including potent pain relief, but produced fewer side effects.

A report was sent to Bayer detailing the results of these tests, upon which Dreser commented: “ the product has no value. ” Nonetheless, Bayer decided to conduct more tests, the results of which confirmed that acetylsalicylic acid worked. Eichengrün stated he suggested the name “aspirin.” 14

Heinrich Dreser. (The quality of mustaches from this time period is astonishing!)

Despite his condemnation of aspirin, Dreser was commissioned to write the paper on Bayer’s synthesis and testing of aspirin , which was published in 1899. His article focused on the pharmacology and animal studies, omitted details of tests in humans, and contains no mention of either Hoffman or Eichengrün. Eichengrün acknowledged this was standard practice at the time: only the company that made the drug was reported, not the inventors. 15

While Eichengrün’s article makes it very clear that he and Hoffman should be credited with the invention of aspirin and not Dreser, he waited until 1949 to publish his account. The account with Hoffman as the sole inventor, however, was published in 1934. So why did Eichengrün wait?

In 1999, Walter Sneader published a reappraisal of the discovery of aspirin, in which he looked at Eichengrün’s 1949 article and other relevant documents. In it, Sneader argued that Eichengrün’s ethnicity forced him to remain quiet. Eichengrün was a Jew living in 1930s and 1940s Germany. The Nazis were making life successively more difficult for Jews, even for successful factory owners like Eichengrün. While Eichengrün had hired a gentile associate to avoid the loss of state contracts, his company was forcibly transferred to a non-Jew in 1938. And although 76 and married to an “aryan,” he was interned at Theresienstadt in 1944 for 14 months.

In his 1949 article, Eichengrün recalled that he saw a display for aspirin in the Hall of Honor of the chemical department of the German Museum in Munich in 1941, with the inscription: "Aspirin; Inventors Dreser and Hoffman." Also at the entrance to the museum was a sign prohibiting Non-Aryans from entering. To this, Eichengrün simply stated: “ Sapienti sat! ” (translated as: “ Enough for the wise! ”)

Given this context, it’s not difficult to imagine why Eichengrün may have been reluctant to claim credit, even if he were aware of the 1934 book. Aspirin was successful and important: if a Jew were to have claimed credit for its invention, when at every level Jews were being forced out of public life and their accomplishments erased, such a person would have undoubtedly provoked a hostile response.

The Theresienstadt archway, with the words “Arbeit macht frei,” meaning “work makes you free.” Theresienstadt was located in German-occupied Czechoslovakia and served as a waystation to extermination camps. Credit: Andrew Shiva .

Additionally, Eichengrün’s article is only the account of one man, written at the very end of his life (he died the same month his article was published), and could be biased, deceptive, or otherwise incorrect. In a press release responding to Sneader’s article, Bayer stated that Eichengrün’s claims cannot be proven. Bayer also claimed that Hoffman was Eichengrün’s equal hierarchically, so Eichengrün couldn’t have ordered Hoffman to create acetylsalicylic acid. The press release also questioned why Eichengrün didn’t object at the time of the patent being awarded solely to Hoffman, and seemed troubled that Eichengrün waited 50 years to claim his role in the development of aspirin.

Against this, we have the knowledge that Nazi censorship , anti-Jewish legislation and sentiment , and propaganda (including in textbooks ) could have removed Eichengrün from the history of aspirin. Even this year, we have examples of erasure of accomplishments due to people’s ethnicity , gender , or both .

Ultimately, this controversy remains unresolved. While it is entirely possible that evidence for Eichengrün’s contributions at Bayer were purged due to antisemitism, we cannot be certain.

NSAID Expansion: 1949 to present

While it is seemingly impossible to find figures for aspirin consumption or production over time, it is clear that it took off after Bayer began marketing the drug in 1899. Eichengrün noted that it wasn’t just that aspirin was effective with limited side effects, but also that it came packaged as tablets rather than as a powder in a paper bag. Tablets were a novelty at the end of the 19th century. Additionally, while the trade name “aspirin” was protected from imitation, the process for making it was not. This meant the availability of generic acetylsalicylic acid sold under different names 16 soon drove down prices.

Aspirin’s popularity diminished once paracetamol and ibuprofen became available in the late 1950s and early 1960s, as the latter drugs had even fewer gastrointestinal side effects. However, its use rose again after the ISIS-2 trial , published in 1988. This trial tested whether low-dose aspirin and streptokinase, separately or together, were effective at preventing mortality after suspected acute myocardial infarction (heart attack). The results were so unambiguously positive for aspirin (and streptokinase, although aspirin was far cheaper) that aspirin use sharply increased worldwide in acute coronary care. The use of aspirin amongst people in hospitals with acute myocardial infarction, for example , increased in the UK from 10 percent in 1987 to over 90 percent in 1989.

Further studies looked at whether aspirin was effective at preventing cardiovascular disease (including heart attacks and strokes), and led to recommendations promoting low-dose aspirin use. A study using 2017 interview data suggested that 23.4 percent of adults 40 years or older in the U.S. (estimated to be about 29 million people) took daily aspirin to prevent cardiovascular disease. Although there are disputes around who should take aspirin for cardiovascular disease prevention, it nonetheless remains extremely popular.

The origins of other NSAIDs are not nearly as dramatic as aspirin. Starting in the 1960s, multiple NSAIDs were developed and patented: indomethacin (1961), ibuprofen (1962), 17 mefenamic acid (1964), naproxen (1967), diclofenac (1978), celecoxib (1993), etoricoxib (1996), among others. 18

Despite the numerous brands of NSAIDs, their mechanism of action remained unknown until 1971, when John Vane discovered the mechanism of action of aspirin, and, by extension, other NSAIDs. These molecules, he found, work by blocking cyclooxygenase (COX) enzymes. COX enzymes produce compounds 19 that cause pain, inflammation, and fever: if you block them, fewer of those compounds are produced. 20 This discovery led to a rapid expansion in the number of available NSAIDs and a Nobel Prize for Vane in 1982.

While future NSAID development is likely to focus on maintaining or improving clinical effectiveness while reducing side effects, there may be novel roles for NSAIDs to fulfil, such as occurred in the 1980s when aspirin was promoted as a treatment and preventative for cardiovascular disease. NSAIDs may likewise play some as yet undetermined role in neurodegenerative diseases, diabetes, and cancer disease therapy .

A Search in the Cabinet

When setting out to write the history of NSAIDs, I thought the project would be reasonably straightforward; people used to drink willow bark tea, and someone clever turned that into aspirin. What I found instead was a labyrinth of oft-repeated myths, partial histories, and conjecture. While piecing the story together, I couldn’t help but feel that scientists could do a better job of preserving their work and methods for posterity.

At its heart, the issue is one of knowing when information is fact, conjecture, or mythology. This is as relevant today as it has ever been, for while the internet has allowed unprecedented access to information, it is as unclear as ever whether that information represents truth. With the rise of artificial intelligence, as well as malicious agents intent on creating and propagating misinformation, determining what is factual will become increasingly more difficult.

Knowing who synthesized aspirin for the first time and why may seem relatively trivial when compared with questions of more immediate relevance and consequence, such as whether facemasks help prevent the spread of COVID-19. But as someone who has attempted to answer both questions, the difficulties in establishing the facts are similar. We need to know who did what, how, and why.

Newton said that scientists stand on the shoulders of giants . However, scientists can only stand above the clouds when there is an unbroken chain of giants beneath them. As soon as one giant is missing, as soon as one link in the unbroken chain of scientific evidence breaks, the whole thing comes tumbling down.

I have one piece of advice for scientists and science communicators: your references matter. They are the evidence that we even have giants supporting us. All scientific claims should be referenced. It isn’t sufficient for you to state that Hippocrates prescribed willow bark tea to treat inflammatory pain. You need to show why you think that is true.

The risk of not providing reputable sources is not just that myths get propagated, it’s that lies get propagated. After all, it likely doesn’t really matter in 2025 whether Hippocrates made people drink 41 cups of willow bark tea a day, or if people write historical fiction where characters rush off to find willow bark to quell a fever. But it does matter whether people believe, for instance, that vaccination is the best way to prevent measles .

I also have one request for people who read scientific literature (of any description): demand better. People may claim to be standing on the shoulders of giants, but are actually shouting from two feet off the ground on a rusty, broken bicycle. Do authors present quality evidence to support their claims, or does it fall apart when you dig a little deeper?

Before researching its origins, I believed that willow bark tea was ancient aspirin. Now, I believe it’s simply willow bark tea. It may behave similarly to aspirin, but not because it is aspirin.

However, by using direct evidence to piece together the rest of the story, I’ve come to trust the following: Reverend Stone found a successful application of powdered willow bark sometime before 1763. He didn’t do this because of any historical precedent, but simply because he thought it might work. Fontana then extracted and named salicin, likely referencing Stone in his report. This led to the use and manufacture of salicylic acid, and then to Arthur Eichengrün, who likely asked Felix Hoffman to synthesize acetylsalicylic acid in 1897.

Ultimately, while I don’t think willow bark tea is ancient aspirin, I do think aspirin can still trace its origins back to the willow bark infusions that Reverend Stone used to treat the people of Chipping Norton over 250 years ago. So fancy that: aspirin was ultimately derived from willow bark after all, maybe just not as long ago as the histories would have us believe.

{{divider}}

Watch our behind-the-scenes interview with the author. Available now on YouTube .

Sean Harrison is an expert in epidemiology and evidence synthesis. He has spent the last decade answering research questions by finding, analysing, and synthesizing all available evidence to answer those questions. During the COVID-19 pandemic, he worked at the UK Health Security Agency, providing evidence reviews to inform COVID-19 policy. He is currently a research fellow at the University of Exeter and blogs at seanharrison.blog . Find him on BlueSky at @sean-h.bsky.social.

Cite: Harrison, S. “The Uncertain Origins of Aspirin.” Asimov Press (2025). https://doi.org/10.62211/58qw-41hg

Header image by Ella Watkins-Dulaney.

This article was published on 14 July 2025.

{{divider}}

Footnotes

  1. If you have a headache , a sprained ankle , a temperature , or some other minor complaint, there’s a good chance that if you reach for any drug, it’ll either be an NSAID, such as aspirin or ibuprofen, or paracetamol (a different class of drug).
  2. Accurate global statistics are difficult (possibly impossible) to find for the total number of prescribed NSAIDs, but in 2022 in the U.S., ibuprofen was the 33rd most prescribed medication (17.5 million prescriptions), followed by aspirin at 36th (17.0 million), diclofenac at 51st (12.5 million), naproxen at 89th (7.4 million), celecoxib at 93rd (7.0 million), and indomethacin at 256th (1.1 million). Mefenamic acid and indomethacin weren’t in the top 300 most prescribed medications.
  3. There are other, more infrequent references to ancient people using willow to reduce pain, inflammation, or fever found among the Chinese and the indigenous peoples of America and South Africa . Additionally, although it is claimed that an Ancient Assyrian clay tablet (3,500 to 2,000 BC) described the use of willow leaves for pain and inflammation, I haven’t found original evidence for this, nor the current location of the tablet, nor its translation.
  4. The symbol of medicine as a staff with a serpent coiled around it — the staff of Asclepius — may have been derived from this treatment for guinea worm. However, like the rest of the historical details in this article, we can’t be sure . We do know, however, that the “caduceus,” a rod with two snakes coiled around it, often with wings at the top, originally had nothing to do with healing or healthcare. Various healthcare-aligned groups, as well as the U.S. Public Health Service, have adopted it nevertheless, possibly confusing the two symbols. In fairness, the two symbols look similar, and a winged rod with two snakes does look better than a wingless rod with only a single snake.
  5. The Hippocratic oath was named for Hippocrates, although it was unlikely to have been written by Hippocrates , and doesn’t start with “ first, do no harm, ” as many (including myself, until recently) believe. Whether any of the surviving works attributed to Hippocrates were actually written by him is debatable , given they are anonymous, written at different times by different hands, and have different ideas about the body and healing. Franz Zacharias Ermerins, a Dutch physician and editor, apparently identified at least 19 different authors of works attributed to Hippocrates.
  6. This is, evidently, part of the Doctrine of Signatures . This is the belief that there must be some (divine) sign telling us which plants to use to cure which diseases. Often, it was the appearance of a plant that was the clue: eyebright flowers resemble eyes, so must be able to treat eye issues; liverwort resembles the shape and colour of the liver, so must be able to treat liver issues; lungwort resembles disease lungs, so must be able to treat lung issues. In this case, it was the location of willow trees: “ the general maxim, that many natural maladies carry their cures along with them, or that their remedies lie not far from their caufes, was fo very appofite to this particular cafe, that I could not help applying it; and that this might be the intention of Providence here, I muft own had fome little weight with me.
  7. Cinchona is almost certainly named for the Countess of Chinchón, a Spanish noblewoman married to the Viceroy of Peru, who developed fever and chills (likely malaria) in 1631. Jesuit priests made a remedy that included the bark of the Cinchona tree, which, presumably, previously had a different name. The remedy worked, and the tree was named Cinchona in her honor.
  8. Other plants with six or fewer mentions were buttercup, foxglove, lesser celandine, and pennywort. Notably, foxgloves were the initial source of digoxin , used to treat various heart conditions, including heart failure. The origins of digoxin also involves medical history mythology: William Withering was an 18th century botanist and physician who, in 1785, first wrote about using foxgloves to treat dropsy (edema caused by heart failure, among other things). However, in 1928, the pharmaceutical company manufacturing digoxin invented “ Mother Hutton ” as part of a marketing campaign. “Mother Hutton” was an old herbalist from Shropshire, who originally discovered that foxglove tea helped dropsy, and who was paid by Withering for that information. Many people now believe the myth that “Mother Hutton” first used foxglove tea for dropsy, despite her being entirely fictional.
  9. Charles Gerhardt was credited with the first synthesis of aspirin in 1852. He reportedly reacted sodium salicylate with acetyl chloride, creating acetylsalicylic acid. However, the final compound was unstable and impure, and the process to make it inconsistent and cumbersome, so Gerhardt reportedly did not investigate further. I was unable to verify this when translating the original evidence : it is both technical and in German, so I’m leaving this as “reportedly.” It’s one of the challenges of working with 19th century foreign-language technical journal articles …
  10. I’ve been unable to access a copy.
  11. I went to some length to find a copy of this article. It didn’t exist online, but I found a physical copy for sale on eBay in Poland. Unfortunately, the seller wouldn’t initially ship to me in the UK, so I used Google Translate to ask them (in German) to allow shipping to the UK, which they very kindly did. Subsequently, DHL managed to lose the parcel, which is a shame as it was one of the few extant copies of this article in existence. However, the British Library is, thankfully, magnificent: they had a copy, and someone very generously managed to dig it out of the archives and photograph it for me. I transcribed the German text, translated it with Google translate, and put both the original German transcription and the English translation on the Internet Archive for posterity. Weeks later, DHL found the article and delivered it, so I scanned the original and uploaded it to the Internet Archive .
  12. Eichengrün did not go into specifics of these tests in his article. Dreser’s introductory paper on aspirin mentions a number of different tests, but it is unclear when they were conducted, and by whom. Nonetheless, Dreser mentions tests using acids and alkalis, taking aspirin himself and then testing his urine (concluding the aspirin had turned into salicylic acid), and tests in rabbits (including body temperature), frogs (including blood vessel constriction, isolated heart function , and lethal dosage), and fish (including lethal dosage). Other tests may have been conducted that were not reported in this paper.
  13. This, quite obviously, could have gone horrendously wrong: thankfully, clinical testing of new drugs has far stronger safeguards these days.
  14. “Aspirin” is derived from “a-” for acetyl, “-spir-” from “Spirsaüre” (spiric acid, an older name for salicylic acid, derived from the Latin for meadowsweet: Spiraea ulmaria), and “-in”, a common chemical suffix.
  15. Eichengrün also stated that they couldn’t patent aspirin in Germany as only the preparation process, not the end result, could be patented. The process for making aspirin, however, also couldn’t be patented because an earlier publication described how acetyl chloride had been mixed with salicylic acid under pressure in a gun barrel. Although the end result was not described, this was evidently sufficient grounds to not award a patent for the process of making aspirin. As a result, and because Bayer would only financially compensate Eichengrün and Hoffman if a patent were granted for a new drug, neither benefited financially from creating aspirin.
  16. For example, salicyl acetate and acidum acetylo-salicylicum: the latter I imagine being a challenge for anyone to pronounce quickly and easily upon first reading it.
  17. The development of ibuprofen shares striking similarities to the development of aspirin. Ibuprofen was developed while searching for a derivative of aspirin that would be suitable for long-term use for rheumatoid arthritis with fewer side effects, much as aspirin was developed as an alternative to salicylic acid. The lead scientist, Dr Stewart Adams, was also happy to test the compounds on himself first (after toxicity tests), and was reportedly excited to be the first person to take a dose of ibuprofen ”, much as Eichengrün first tested aspirin on himself.
  18. I’d be lying if I said I knew all those drug names before writing this article.
  19. Thromboxanes, which play a role in platelet adhesion; prostaglandins, which cause vasodilation (expansion of blood vessels), increase body temperature by interacting with the hypothalamus, and reduce pain; and prostacyclins, which inhibit platelet activation and cause vasodilation.
  20. There are two cyclooxygenase (COX) enzymes, which are both affected to a greater or lesser extent by NSAIDs. The first of which (COX-1) is found all over the body (in particular the gastrointestinal lining and kidneys), and the second (COX-2) is only around during an inflammatory response. As they affect these two enzymes differently, NSAIDs have different levels of effectiveness and side effect profiles.

Technical Standards in Service to Humanity

Internet Exchange
internet.exchangepoint.tech
2025-11-20 16:22:23
Inside the making of a new UN report on technology and human rights....
Original Article
internet governance

Inside the making of a new UN report on technology and human rights.

Technical Standards in Service to Humanity
Photo by CHUTTERSNAP / Unsplash

By Mallory Knodel

The Office of the High Commissioner for Human Rights (OHCHR) took another significant step toward reshaping how the technical community can consider human rights in standards setting by releasing a new report , published this week titled, “Tech and Human Rights Study: Making technical standards work for humanity - New pathways for incorporating international human rights into standards development for digital technologies.”

Technical standards, the shared rules that make digital systems like the internet work, help shape the conditions for human rights online. After a year of consultations, this OHCHR report outlines an informed agenda for how standards development organizations can integrate human rights into both their processes and the standards of the technologies they design. It also describes the current landscape of global standards bodies, identifies the barriers that prevent meaningful human rights engagement, and highlights practices that support openness, inclusivity, transparency, and accountability.

Today’s tech raises critical questions about human rights

The office began work on the new report following its 2023 Human Rights Council resolution on the importance of integrating human rights into the work of technical standards bodies. That earlier resolution recognized that internet and digital technologies shape the most basic conditions for people’s rights. This new report focuses on a specific and long overdue question: how can standards development organizations support human rights through both their processes and the technical standards they create?

The report shows that technical standards play a critical role in shaping whether human rights are upheld or undermined depending on the choices embedded in their design. Standards that promote openness, interoperability, and secure communication help safeguard freedom of expression and access to information, while those that introduce filtering, traffic controls, or shutdown mechanisms can restrict them. The report also highlights that the architecture of standards shapes whether people can assemble and associate online in safe and reliable ways. And because standards determine how data is transmitted, stored, or exposed, they have significant implications for privacy, a right enshrined in Article 12 of the Universal Declaration of Human Rights. Standards can either protect users from surveillance or make intrusive monitoring easier. In short, the report shows that technical standards are not neutral: they encode decisions that can strengthen human rights by design or facilitate their erosion.

The work with the OHCHR throughout the year focused on supporting this effort. This included helping to design and run a consultative process with six focused conversations involving stakeholders from across standards development, human rights advocacy, internet governance, and emerging technology communities. One consultation also took place as a side meeting at the IETF. It gave participants a chance to speak directly to the relationship between human rights and technical standards in an engineering-focused environment. Each conversation brought different experiences into the room.

Bringing the technical and human rights communities together

The report builds on more than a decade of work by human rights organizations and public interest technologists who engage in standards development. Their work focuses on the design, development, and deployment of internet and digital technologies, including artificial intelligence. These communities analyze how technical choices influence surveillance, censorship, discrimination, and other rights concerns. Their long-term engagement shows why standards work needs direct human rights input.

All six consultations led into a final online meeting that brought every participant together with a goal of confirming that the draft captured what people shared throughout the process and to ensure that the material was accurate, clear, and useful. We circulated an early version of the report to all participants and invited written feedback. Their comments strengthened the final text and helped shape the recommendations.

The pathways towards human rights respecting standards

The timing of this report also matters. The Global Digital Compact, adopted at the United Nations General Assembly, directs the OHCHR to coordinate human rights considerations across global internet governance institutions. That includes multilateral bodies like the ITU and multistakeholder communities like the IETF. The compact reinforces the idea that governments, civil society, and standards bodies share responsibility for integrating human rights into technical work.

The report describes the current landscape of standards development organizations and outlines how each organization structures participation, transparency, documentation, and decision-making. It identifies clear points where human rights considerations can enter these processes. It also provides concrete recommendations for standards bodies, governments, and civil society. These recommendations address process design, risk assessment, participation support, and the need for sustainable engagement by public interest technologists.

This work continues. Next month the AI Standards Summit in Seoul will host a session on human rights in technical standards. Many participants from our consultations will attend. The ITU Telecommunication Standardization Advisory Group will meet in January to continue its own discussions about how to incorporate human rights considerations into its processes.

The recommendations give governments, standards bodies, and advocates practical steps they can take today. Broader awareness and stronger participation will help build an internet that better protects human rights for everyone.


Two weeks ago, Mallory and the IX team hosted a series of events related to human rights and the social web at MozFest 2025 in Barcelona. While there, Mallory joined the legendary Rabble, a.k.a Evan Henshaw-Plath (Twitter's first employee) to talk about who controls Web 2.0 and how the fediverse gives us a second chance; how she convinced the IETF to evaluate protocols for human rights implications; and why content moderation should be contextual, not universal. They also discuss how Edward Snowden’s revelations changed global internet standards, the 2025 funding crisis and how Ghost provides a model for sustainable open-source businesses.

Support the Internet Exchange

If you find our emails useful, consider becoming a paid subscriber! You'll get access to our members-only Signal community where we share ideas, discuss upcoming topics, and exchange links. Paid subscribers can also leave comments on posts and enjoy a warm, fuzzy feeling.

Not ready for a long-term commitment? You can always leave us a tip .

Become A Paid Subscriber


From the Group Chat 👥 💬

This week in our Signal community, we got talking about:

Cloudflare, one of a handful of companies that together provide a stack of critical internet infrastructure services, went offline on Tuesday affecting millions of companies including ChatGPT, X and, annoyingly for me, Moodle, my university’s online learning platform. In the IX group chat, we noted that an outage at a company used by 81.5% of all websites that rely on a reverse proxy is a reminder of how much of the internet is reliant on a few big companies. This one happens to also be moving into identity, payments, and standards-setting in ways that look a lot like building the power to paywall and ID-wall the web.


We’re Keeping An Eye On: Chat Control

EU governments have agreed on a draft of the Chat Control law that legally allows platforms to scan private messages on a voluntary basis while confirming there is no obligation to do so. The Commission wanted platforms to be obligated to scan all user communications for signs of crime and report suspicious content. The European Parliament called this mass surveillance and insisted that scanning should apply only to unencrypted content of specific suspects. The resulting draft is a compromise: there will be no obligation to scan, but voluntary scanning will be legally allowed.

Privacy experts warn the plan is unlawful, ineffective and easy to abuse, and say its age verification rules risk major privacy violations and the loss of online anonymity. Netzpolitik.org has published the classified negotiation protocol and the draft law: https://netzpolitik.org/2025/interne-dokumente-eu-staaten-einigen-sich-auf-freiwillige-chatkontrolle/

For regular coverage on this fast-moving legislation, this former MEP is posting regular detailed updates https://digitalcourage.social/@echo_pbreyer


  • Decentralised social networks highlight the value of a model that redistributes power to users and communities. In this recorded session from Decidim, Amandine Le Pape (Element), Robin Berjon (Free our Feeds), Andy Piper (Mastodon) and moderator Marta G.Franco (Laintersección) discuss the challenges and opportunities of building truly democratic social networks that are truly ours. https://www.youtube.com/watch?v=mWX8O2HWGMY

Internet Governance

Digital Rights

Technology for Society

Privacy and Security

Upcoming Events

  • Running a workshop, training, or meeting soon? Join The Session Design Lab to explore practical, inclusive session design, dig into adult learning frameworks, and design and peer-review your own session in a supportive, pay-what-you-can environment. It’s offered at two different times to accommodate multiple time zones, and as a past participant, I can personally vouch for its awesomeness. 10th-11th December . Online . https://www.fabriders.net/session-design-lab

Careers and Funding Opportunities

Opportunities to Get Involved

What did we miss? Please send us a reply or write to editor@exchangepoint.tech .

In a U.S. First, New Mexico Opens Doors to Free Child Care for All

Hacker News
www.wsj.com
2025-11-22 16:11:12
Comments...
Original Article

Please enable JS and disable any ad blocker

New Apple Study Shows LLMs Can Tell What You're Doing from Audio and Motion Data

Hacker News
9to5mac.com
2025-11-22 15:45:26
Comments...
Original Article
Apple Intelligence iOS 26 light purple

Apple researchers have published a study that looks into how LLMs can analyze audio and motion data to get a better overview of the user’s activities. Here are the details.

They’re good at it, but not in a creepy way

A new paper titled “ Using LLMs for Late Multimodal Sensor Fusion for Activity Recognition ” offers insight into how Apple may be considering incorporating LLM analysis alongside traditional sensor data to gain a more precise understanding of user activity.

This, they argue, has great potential to make activity analysis more precise, even in situations where there isn’t enough sensor data.

From the researchers:

“Sensor data streams provide valuable information around activities and context for downstream applications, though integrating complementary information can be challenging. We show that large language models (LLMs) can be used for late fusion for activity classification from audio and motion time series data. We curated a subset of data for diverse activity recognition across contexts (e.g., household activities, sports) from the Ego4D dataset. Evaluated LLMs achieved 12-class zero- and one-shot classification F1-scores significantly above chance, with no task-specific training. Zero-shot classification via LLM-based fusion from modality-specific models can enable multimodal temporal applications where there is limited aligned training data for learning a shared embedding space. Additionally, LLM-based fusion can enable model deploying without requiring additional memory and computation for targeted application-specific multimodal models.”

In other words, LLMs are actually pretty good at inferring what a user is doing from basic audio and motion signals, even when they’re not specifically trained for that. Moreover, when given just a single example, their accuracy improves even further.

One important distinction is that in this study, the LLM wasn’t fed the actual audio recording, but rather, short text descriptions generated by audio models and an IMU-based motion model (which tracks movement through accelerometer and gyroscope data), as shown below:

Diving a bit deeper

In the paper, the researchers explain that they used Ego4D, a massive dataset of media shot in first-person perspective. The data contains thousands of hours of real-world environments and situations, from household tasks to outdoor activities.

From the study:

“We curated a dataset of day-to-day activities from the Ego4D dataset by searching for activities of daily living within the provided narrative descriptions. The curated dataset includes 20 second samples from twelve high-level activities: vacuum cleaning, cooking, doing laundry, eating, playing basketball, playing soccer, playing with pets, reading a book, using a computer, washing dishes, watching TV, workout/weightlifting. These activities were selected to span a range of household and fitness tasks, and based on their prevalence in the larger dataset.”

The researchers ran the audio and motion data through smaller models that generated text captions and class predictions, then fed those outputs into different LLMs (Gemini-2.5-pro and Qwen-32B) to see how well they could identify the activity.

Then, Apple compared the performance of these models in two different situations: one in which they were given the list of the 12 possible activities to choose from (closed-set), and another where they weren’t given any options (open-ended).

For each test, they were given different combinations of audio captions, audio labels, IMU activity prediction data, and extra context, and this is how they did:

In the end, the researchers note that the results of this study offer interesting insights into how combining multiple models can benefit activity and health data, especially in cases where raw sensor data alone is insufficient to provide a clear picture of the user’s activity.

Perhaps more importantly, Apple published supplemental materials alongside the study, including the Ego4D segment IDs, timestamps, prompts, and one-shot examples used in the experiments, to assist researchers interested in reproducing the results.

Accessory deals on Amazon

Add 9to5Mac as a preferred source on Google Add 9to5Mac as a preferred source on Google

FTC: We use income earning auto affiliate links. More.

Dirk Eddelbuettel: RcppArmadillo 15.2.2-1 on CRAN: Upstream Update, OpenMP Updates

PlanetDebian
dirk.eddelbuettel.com
2025-11-22 15:44:00
Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research ...
Original Article

RcppArmadillo 15.2.2-1 on CRAN: Upstream Update, OpenMP Updates

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1286 other packages on CRAN , downloaded 42.6 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper ( preprint / vignette ) by Conrad and myself has been cited 659 times according to Google Scholar.

This versions updates to the 15.2.2 upstream Armadillo release made two days ago. It brings a few changes over the RcppArmadillo 15.2.0 release made only to GitHub (and described in this post ), and of course even more changes relative to the last CRAN release described in this earlier post . As described previously, and due to both the upstream transition to C++14 coupled with the CRAN move away from C++11, the package offers a transition by allowing packages to remain with the older, pre-15.0.0 ‘legacy’ Armadillo yet offering the current version as the default. During the transition we did not make any releases to CRAN allowing both the upload cadence to settle back to the desired ‘about six in six months’ that the CRAN Policy asks for, and for packages to adjust to any potential changes. Most affected packages have done so (as can be seen in the GitHub issues #489 and #491 ) which is good to see. We appreciate all the work done by the respective package maintainers. A number of packages are still under a (now formally expired) deadline at CRAN and may get removed. Our offer to help where we can still stands, so please get in touch if we can be of assistance. As a reminder, the meta-issue #475 regroups all the resources for the transition.

With respect to changes in the package, we once more overhauled the OpenMP detection and setup, following the approach take by package data.table but sticking with an autoconf -based configure . The detailed changes since the last CRAN release follow.

Changes in RcppArmadillo version 15.2.2-1 (2025-11-21)

  • Upgraded to Armadillo release 15.2.2 (Medium Roast Deluxe)

    • Improved reproducibility of random number generation when using OpenMP
  • Skip a unit test file under macOS as complex algebra seems to fail under newer macOS LAPACK setting

  • Further OpenMP detection rework for macOS (Dirk in #497 , #499 )

  • Define ARMA_CRIPPLED_LAPACK on Windows only if 'LEGACY' Armadillo selected

Changes in RcppArmadillo version 15.2.1-0 (2025-10-28) (GitHub Only)

  • Upgraded to Armadillo release 15.2.1 (Medium Roast Deluxe)

    • Faster handling of submatrices with one row
  • Improve OpenMP detection (Dirk in #495 fixing #493 )

Changes in RcppArmadillo version 15.2.0-0 (2025-10-20) (GitHub Only)

  • Upgraded to Armadillo release 15.2.0 (Medium Roast Deluxe)

    • Added rande() for generating matrices with elements from exponential distributions

    • shift() has been deprecated in favour of circshift() , for consistency with Matlab/Octave

    • Reworked detection of aliasing, leading to more efficient compiled code

  • OpenMP detection in configure has been simplified

Courtesy of my CRANberries , there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page . Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. If you like this or other open-source work I do, you can sponsor me at GitHub .

/code/rcpp | permanent link

What We Forgot About Socialism: Lessons From the Red Riviera

Portside
portside.org
2025-11-22 15:35:05
What We Forgot About Socialism: Lessons From the Red Riviera Kurt Stand Sat, 11/22/2025 - 10:35 ...
Original Article

Twenty years ago in November of 2005, Duke University Press published my first book: The Red Riviera: Gender, Tourism, and Postsocialism on the Black Sea . Produced in the wake of socialism’s global collapse and the riot of Western triumphalism that ensued, I deployed both qualitative and quantitative methods to advance a simple, but unpopular, argument: for most people in the former Soviet bloc, capitalism sucked.

By writing the “small histories” of men and women labouring in Bulgaria’s vibrant tourism industry in the decade following their country’s mad dash to embrace democracy and free markets, I explored how and why this small southeastern European country transformed from a relatively predictable, orderly, egalitarian society into a chaotic, lawless, world of astonishing inequality and injustice. I wrapped my critiques of the rampant neoliberalism of the “Wild, Wild, East” in thickly descriptive accounts of the lives of chambermaids, bartenders, tour guides, cooks, waitresses, and receptionists. I wanted to show, not tell.

Through a close examination of the shattered careers and broken families of ordinary men and women forced to live through the cataclysmic decade of the 1990s, I asked readers to empathize with the sheer scale of the upheavals of banking collapses, hyperinflation, unemployment, violence, suicide, and the mass emigration of youth. Capitalism promised prosperity and freedom, but for many it delivered little more than poverty and despair. The dislocations of the transition period, as I’ve documented in my subsequent books, still reverberate today. One can easily draw a straight line from the trauma of the 1990s to the rise of right-wing parties and authoritarian leaders in the region.

Some Positive Aspects of Actually Existing Socialism

Perhaps more controversial, especially back in 2005, was my claim that, despite some serious shortcomings, there were some positive aspects of socialism that should not be forgotten. In those heady days of Francis Fukuyama’s “ end of history ” narrative about the primacy of liberal democracy and free markets, to suggest that there was a baby in the bathwater was political heresy. In this contemporary moment when a Democratic Socialist is likely to be the next mayor of New York City, it may be hard to remember how passé socialism was in the first decade of the 21st century. Jacobin Magazine did not yet exist; Bernie Sanders had not yet run for president; Alexandria Ocasio-Cortez had not yet entered Congress. In an academic climate dominated by poststructuralist critiques of power, even mild sympathies for socialism drew fire from both the anti-communist right and the postmodern left.

As a young academic, I was perhaps too naïve to anticipate the sort of vitriolic criticism I would receive by listening carefully to my older informants, researching socialist-era legal codes, and conducting two large anonymous surveys of tourism workers. Although I dutifully corroborated my various findings, and wrote an honest description of what life in socialist Bulgaria had been like for ordinary people, some reviewers accused me of having been duped by communist disinformation. For example, one 2007 review in the journal Aspasia suggested that: “Ghodsee’s analysis is problematic because sometimes interpretations fall into the trap of sociological legends fabricated by communist propaganda.” As an apparent example of these “sociological legends”, the reviewer quotes me: “Bulgarian women once benefited from generous maternity leaves, free education, free healthcare, free or subsidized child care, communal kitchens and canteens, communal laundries, subsidized food and transport, subsidized holidays on the Black Sea, etc. (p. 165).” All of this was true, and the reviewer did not present any evidence to the contrary.

The Aspasia reviewer acknowledged that “many, especially among the less educated (near) pension-age women in Bulgaria” did believe that the coming of capitalism had deprived them of these universal basic services, but she maintained that this was only because they had been brainwashed by the socialist system. My Bulgarian informants in the late 1990s were apparently incapable of understanding that capitalism would bring higher salaries with which one could purchase supposedly better-quality housing, education, healthcare, and childcare, and that this would be far preferable to having lower wages but receiving these things for free. The reviewer then asked: “The question is, why would a researcher ‘from outside’ buy into this propaganda in a similar way?”

Capitalism and Poverty

Part of the “propaganda” that I apparently bought into was that the radical dismantling of social safety nets following the introduction of free market economies would push millions of Bulgarians into poverty, and that the process would be distinctly gendered to most women’s disadvantage. This turned out simply to be true, as I and others have documented (see Milanovic 2014 , Ghodsee 2018 , and Ghodsee and Orenstein 2021 , for example). You didn’t need to be a Marxist to understand the black humor behind common jokes being told in the late 1990s:

Q: What did Bulgarians light their homes with before they used candles?
A: Electricity.
Q: What was the worst thing about communism?
A: The thing that came after it.

This is not to deny that there were some appalling things about the communist regimes, including its lack of genuinely representative government, its attacks on political speech the government didn’t like, and its use of repressive and secretive police outside the rule of law. One should condemn such infringements of basic human rights, both as they occurred under communism and as they are happening now in the United States.

However, constantly preaching about the negative aspects of 20th century state socialism can make it harder for us to see the things that socialism got right. It may even be a deliberate strategy. Those with the most to gain from capitalism want us to forget the good things that happened under socialism, lest we try to do anything to change a system in which wealth flows up into the hands of the rich and powerful.

Doing the research necessary to write The Red Riviera convinced me that there are indeed many things we can learn from the experiences of those who lived through a real and relatively long-lasting alternative to capitalism. The experiences of socialist countries in Eastern Europe remind us that societies can achieve a great deal when they treat people’s basic needs as a shared responsibility. Education, healthcare, childcare, housing, and a reasonable minimal standard of living were seen not as privileges but as something we should collectively guarantee for all.

My subjects did complain about having to wake up early for neighborhood work on a “Lenin Saturday” but also noted that socialism promoted a belief in the power of community and the dignity of every person’s contribution. Women entered schools and workplaces in greater numbers, finding new confidence and independence. Cultural life – music, theater, literature – was made accessible to everyone, helping people feel connected to something larger than themselves. Planned microdistricts (an early version of what are now called “15-minute cities”), and socialist workplaces often became centers of shared activity and mutual support.

Even though these societies faced serious political and economic challenges, their social ideals of equality, solidarity, and collective care remain relevant to us in 2025. They remind us that success isn’t only about material wealth or technology but about how we choose to care for one another. When an economy is guided by social purpose instead of profit, it can serve the common good and lay a foundation for long-term progress, a lesson that we should all remember as we face the existential threat of the climate crisis.

I’m not as naïve as I was in 2005. These days, I expect that my critics will see me as the hapless victim of red “propaganda” and will accuse me of underplaying the repression that occurred in the Soviet bloc countries.

But I’ve also come to conclude that there is a place for naïveté. Naïvely listening to how ordinary people remember their lives (even the “less educated (near) pension-age women”!) can be far better than going into a project with preconceived ideas about how your subjects have been brainwashed by propaganda from an evil system and without fear about how you will be criticized for taking those subjects seriously.

I’ve learned that good scholarship, like good politics, depends on empathy as much as on evidence. Listening carefully to how ordinary people remember their lives under socialism isn’t an endorsement; it’s an effort to understand what they valued and why. Those memories, often complex and sometimes contradictory, reveal the texture of daily life that grand theories tend to miss. They remind us that the past is never as simple as our ideologies make it out to be. If we can take those lessons seriously, if we can listen with curiosity rather than judgment, we might find inspiration for new forms of solidarity and care in the uncertain world we inhabit today. •

This article first published on the Duke University Press website.

Kristen R. Ghodsee is the award-winning author of twelve books and Professor of Russian and East European Studies at the University of Pennsylvania. She blogs at .

The Bullet is an online publishing venue for the socialist Left in Canada and around the world. It solicits, publishes and republishes articles that advance socialist ideas and analysis of important events, topics and issues. It publishes articles by socialist thinkers and activists, movement builders and organizers, workers and trade unionists, and all those seeking to go beyond capitalism.

LLM APIs are a Synchronization Problem

Lobsters
lucumr.pocoo.org
2025-11-22 15:27:29
Comments...
Original Article

written on November 22, 2025

The more I work with large language models through provider-exposed APIs, the more I feel like we have built ourselves into quite an unfortunate API surface area. It might not actually be the right abstraction for what’s happening under the hood. The way I like to think about this problem now is that it’s actually a distributed state synchronization problem.

At its core, a large language model takes text, tokenizes it into numbers, and feeds those tokens through a stack of matrix multiplications and attention layers on the GPU. Using a large set of fixed weights, it produces activations and predicts the next token. If it weren’t for temperature (randomization), you could think of it having the potential of being a much more deterministic system, at least in principle.

As far as the core model is concerned, there’s no magical distinction between “user text” and “assistant text”—everything is just tokens. The only difference comes from special tokens and formatting that encode roles (system, user, assistant, tool), injected into the stream via the prompt template. You can look at the system prompt templates on Ollama for the different models to get an idea.

The Basic Agent State

Let’s ignore for a second which APIs already exist and just think about what usually happens in an agentic system. If I were to have my LLM run locally on the same machine, there is still state to be maintained, but that state is very local to me. You’d maintain the conversation history as tokens in RAM, and the model would keep a derived “working state” on the GPU—mainly the attention key/value cache built from those tokens. The weights themselves stay fixed; what changes per step are the activations and the KV cache.

From a mental-model perspective, caching means “remember the computation you already did for a given prefix so you don’t have to redo it.” Internally, that usually means storing the attention KV cache for those prefix tokens on the server and letting you reuse it, not literally handing you raw GPU state.

There are probably some subtleties to this that I’m missing, but I think this is a pretty good model to think about it.

The Completion API

The moment you’re working with completion-style APIs such as OpenAI’s or Anthropic’s, abstractions are put in place that make things a little different from this very simple system. The first difference is that you’re not actually sending raw tokens around. The way the GPU looks at the conversation history and the way you look at it are on fundamentally different levels of abstraction. While you could count and manipulate tokens on one side of the equation, extra tokens are being injected into the stream that you can’t see. Some of those tokens come from converting the JSON message representation into the underlying input tokens fed into the machine. But you also have things like tool definitions, which are injected into the conversation in proprietary ways. Then there’s out-of-band information such as cache points.

And beyond that, there are tokens you will never see. For instance, with reasoning models you often don’t see any real reasoning tokens, because some LLM providers try to hide as much as possible so that you can’t retrain your own models with their reasoning state. On the other hand, they might give you some other informational text so that you have something to show to the user. Model providers also love to hide search results and how those results were injected into the token stream. Instead, you only get an encrypted blob back that you need to send back to continue the conversation. All of a sudden, you need to take some information on your side and funnel it back to the server so that state can be reconciled on either end.

In completion-style APIs, each new turn requires resending the entire prompt history. The size of each individual request grows linearly with the number of turns, but the cumulative amount of data sent over a long conversation grows quadratically because each linear-sized history is retransmitted at every step. This is one of the reasons long chat sessions feel increasingly expensive. On the server, the model’s attention cost over that sequence also grows quadratically in sequence length, which is why caching starts to matter.

The Responses API

One of the ways OpenAI tried to address this problem was to introduce the Responses API, which maintains the conversational history on the server (at least in the version with the saved state flag). But now you’re in a bizarre situation where you’re fully dealing with state synchronization: there’s hidden state on the server and state on your side, but the API gives you very limited synchronization capabilities. To this point, it remains unclear to me how long you can actually continue that conversation. It’s also unclear what happens if there is state divergence or corruption. I’ve seen the Responses API get stuck in ways where I couldn’t recover it. It’s also unclear what happens if there’s a network partition, or if one side got the state update but the other didn’t. The Responses API with saved state is quite a bit harder to use, at least as it’s currently exposed.

Obviously, for OpenAI it’s great because it allows them to hide more behind-the-scenes state that would otherwise have to be funneled through with every conversation message.

State Sync API

Regardless of whether you’re using a completion-style API or the Responses API, the provider always has to inject additional context behind the scenes—prompt templates, role markers, system/tool definitions, sometimes even provider-side tool outputs—that never appears in your visible message list. Different providers handle this hidden context in different ways, and there’s no common standard for how it’s represented or synchronized. The underlying reality is much simpler than the message-based abstractions make it look: if you run an open-weights model yourself, you can drive it directly with token sequences and design APIs that are far cleaner than the JSON-message interfaces we’ve standardized around. The complexity gets even worse when you go through intermediaries like OpenRouter or SDKs like the Vercel AI SDK, which try to mask provider-specific differences but can’t fully unify the hidden state each provider maintains. In practice, the hardest part of unifying LLM APIs isn’t the user-visible messages—it’s that each provider manages its own partially hidden state in incompatible ways.

It really comes down to how you pass this hidden state around in one form or another. I understand that from a model provider’s perspective, it’s nice to be able to hide things from the user. But synchronizing hidden state is tricky, and none of these APIs have been built with that mindset, as far as I can tell. Maybe it’s time to start thinking about what a state synchronization API would look like, rather than a message-based API.

The more I work with these agents, the more I feel like I don’t actually need a unified message API. The core idea of it being message-based in its current form is itself an abstraction that might not survive the passage of time.

Learn From Local First?

There’s a whole ecosystem that has dealt with this kind of mess before: the local-first movement. Those folks spent a decade figuring out how to synchronize distributed state across clients and servers that don’t trust each other, drop offline, fork, merge, and heal. Peer-to-peer sync, and conflict-free replicated storage engines all exist because “shared state but with gaps and divergence” is a hard problem that nobody could solve with naive message passing. Their architectures explicitly separate canonical state, derived state, and transport mechanics — exactly the kind of separation missing from most LLM APIs today.

Some of those ideas map surprisingly well to models: KV caches resemble derived state that could be checkpointed and resumed; prompt history is effectively an append-only log that could be synced incrementally instead of resent wholesale; provider-side invisible context behaves like a replicated document with hidden fields.

At the same time though, if the remote state gets wiped because the remote site doesn’t want to hold it for that long, we would want to be in a situation where we can replay it entirely from scratch—which for instance the Responses API today does not allow.

Future Unified APIs

There’s been plenty of talk about unifying message-based APIs, especially in the wake of MCP (Model Context Protocol). But if we ever standardize anything, it should start from how these models actually behave, not from the surface conventions we’ve inherited. A good standard would acknowledge hidden state, synchronization boundaries, replay semantics, and failure modes — because those are real issues. There is always the risk that we rush to formalize the current abstractions and lock in their weaknesses and faults. I don’t know what the right abstraction looks like, but I’m increasingly doubtful that the status-quo solutions are the right fit.

This entry was tagged ai

copy as / view markdown

Migrating to Bazel symbolic macros

Lobsters
www.tweag.io
2025-11-22 15:25:19
Comments...
Original Article

In Bazel, there are two types of macros: legacy macros and symbolic macros, that were introduced in Bazel 8. Symbolic macros are recommended for code clarity, where possible. They include enhancements like typed arguments and the ability to define and limit the visibility of the targets they create.

This post is intended for experienced Bazel engineers or those tasked with modernizing the build metadata of their codebases. The following discussion assumes a solid working knowledge of Bazel’s macro system and build file conventions. If you are looking to migrate legacy macros or deepen your understanding of symbolic macros, you’ll find practical guidance and nuanced pitfalls addressed here.


What are symbolic macros?

Macros instantiate rules by acting as templates that generate targets. As such, they are expanded in the loading phase , when Bazel definitions and BUILD files are loaded and evaluated. This is in contrast with build rules that are run later in the analysis phase. In older Bazel versions, macros were defined exclusively as Starlark functions (the form that is now called “legacy macros”). Symbolic macros are an improvement on that idea; they allow defining a set of attributes similar to those of build rules.

In a BUILD file, you invoke a symbolic macro by supplying attribute values as arguments. Because Bazel is explicitly aware of symbolic macros and their function in the build process, they can be considered “first-class macros”. See the Symbolic macros design document to learn more about the rationale. Symbolic macros also intend to support lazy evaluation , a feature that is currently being considered for a future Bazel release. When that functionality is implemented, Bazel would defer evaluating a macro until the targets defined by that macro are actually requested.

Conventions and restrictions

There is already good documentation that explains how to write symbolic macros. In this section, we are going to take a look at some practical examples of the restrictions that apply to their implementation, which you can learn more about in the Restrictions docs page.

Naming

Any targets created by a symbolic macro must either match the macro’s name parameter exactly or begin with that name followed by a _ (preferred), . , or - . This is different from legacy macros which don’t have naming constraints.

This symbolic macro

# defs.bzl
def _simple_macro_impl(name):
    native.genrule(
        name = "genrule" + name,
        outs = [name + "_out.data"],
        srcs = ["//:file.json"],
    )

# BUILD.bazel
simple_macro(name = "tool")

would fail when evaluated:

$ bazel cquery //...
ERROR: in genrule rule //src:genruletool: Target //src:genruletool declared in symbolic macro 'tool'
violates macro naming rules and cannot be built.

This means simple_macro(name = "tool") may only produce files or targets named tool or starting with tool_ , tool. , or tool- . In this particular macro, tool_genrule would work.

Access to undeclared resources

Symbolic macros must follow Bazel’s standard visibility rules: they cannot directly access source files unless those files are passed in as arguments or are made public by their parent package. This is different from legacy macros, whose implementations were effectively inlined into the BUILD file where they were called.

Attributes

Positional arguments

In legacy macro invocations, you could have passed the attribute values as positional arguments. For instance, these are perfectly valid legacy macro calls:

# defs.bzl
def special_test_legacy(name, tag = "", **kwargs):
    kwargs["name"] = name
    kwargs["tags"] = [tag] if tag else []
    cc_test(**kwargs)


# BUILD.bazel
special_test_legacy("no-tag")
special_test_legacy("with-tag", "manual")

With the macro’s name and tags collected as expected:

$ bazel cquery //test/package:no-tag --output=build
cc_test(
  name = "no-tag",
  tags = [],
  ...
)

$ bazel cquery //test/package:with-tag --output=build
cc_test(
  name = "with-tag",
  tags = ["manual"],
  ...
)

You can control how arguments are passed to functions by using an asterisk ( * ) in the parameter list of a legacy macro, as per the Starlark language specs . If you are a seasoned Python developer (Starlark’s syntax is heavily inspired by Python), you might have already guessed that this asterisk separates positional arguments from keyword-only arguments:

# defs.bzl
def special_test_legacy(name, *, tag = "", **kwargs):
    kwargs["name"] = name
    kwargs["tags"] = [tag] if tag else []
    cc_test(**kwargs)

# BUILD.bazel
special_test_legacy("no-tag")  # okay
special_test_legacy("with-tag", tag="manual") # okay
# Error: special_test_legacy() accepts no more than 1 positional argument but got 2
special_test_legacy("with-tag", "manual")

Positional arguments are not supported in symbolic macros as attributes must either be declared in the attrs dictionary (which would make it automatically a keyword argument) or be inherited in which case it should also be provided by name.

Arguably, avoiding positional arguments in macros altogether is helpful because it eliminates subtle bugs caused by incorrect order of parameters passed and makes them easier to read and easier to process by tooling such as buildozer .

Default values

Legacy macros accepted default values for their parameters which made it possible to skip passing certain arguments:

# defs.bzl
def special_test_legacy(name, *, purpose = "dev", **kwargs):
    kwargs["name"] = name
    kwargs["tags"] = [purpose]
    cc_test(**kwargs)

# BUILD.bazel
special_test_legacy("dev-test")
special_test_legacy("prod-test", purpose="prod")

With symbolic macros, the default values are declared in the attrs dictionary instead:

# defs.bzl
def _special_test_impl(name, purpose = "dev", **kwargs):
    kwargs["tags"] = [purpose]
    cc_test(
        name = name,
        **kwargs
    )

special_test = macro(
    inherit_attrs = native.cc_test,
    attrs = {
        "purpose": attr.string(configurable = False, default = "staging"),
        "copts": None,
    },
    implementation = _special_test_impl,
)

# BUILD.bazel
special_test(
    name = "my-special-test-prod",
    srcs = ["test.cc"],
    purpose = "prod",
)

special_test(
    name = "my-special-test-dev",
    srcs = ["test.cc"],
)

Let’s see what kind of tags are going to be set for these cc_test targets:

$ bazel cquery //test/package:my-special-test-prod --output=build
cc_test(
  name = "my-special-test-prod",
  tags = ["prod"],
  ...
)

$ bazel cquery //test/package:my-special-test-dev --output=build
cc_test(
  name = "my-special-test-dev",
  tags = ["staging"],
  ...
)

Notice how the default dev value declared in the macro implementation was never used. This is because the default values defined for parameters in the macro’s function are going to be ignored, so it’s best to remove them to avoid any confusion.

Also, all the inherited attributes have a default value of None , so make sure to refactor your macro logic accordingly. Be careful when processing the keyword arguments to avoid subtle bugs such as checking whether a user has passed [] in a keyword argument merely by doing if not kwargs["attr-name"] as None would also be evaluated to False in this context.

This might be potentially confusing as the default value for many common attributes is not None . Take a look at the target_compatible_with attribute which normally has the default value [] when used in a rule, but when used in a macro, would still by default be set to None . Using bazel cquery //:target --output=build with some print calls in your .bzl files can help when refactoring.

Inheritance

Macros are frequently designed to wrap a rule (or another macro), and the macro’s author typically aims to pass most of the wrapped symbol’s attributes using **kwargs directly to the macro’s primary target or the main inner macro without modification.

To enable this behavior, a macro can inherit attributes from a rule or another macro by providing the rule or macro symbol to the inherit_attrs parameter of macro() . Note that when inherit_attrs is set, the implementation function must have a **kwargs parameter. This makes it possible to avoid listing every attribute that the macro may accept, and it is also possible to disable certain attributes that you don’t want macro callers to provide. For instance, let’s say you don’t want copts to be defined in macros that wrap cc_test because you want to manage them internally within the macro body instead:

# BUILD.bazel
special_test(
    name = "my-special-test",
    srcs = ["test.cc"],
    copts = ["-std=c++22"],
)

This can be done by setting the attributes you don’t want to inherit to None .

# defs.bzl
special_test = macro(
    inherit_attrs = native.cc_test,
    attrs = { "copts": None },
    implementation = _special_test_impl,
)

Now the macro caller will see that copts is not possible to declare when calling the macro:

$ bazel query //test/package:my-special-test
        File "defs.bzl", line 19, column 1, in special_test
                special_test = macro(
Error: no such attribute 'copts' in 'special_test' macro

Keep in mind that all inherited attributes are going to be included in the kwargs parameter with the default value of None unless specified otherwise. This means you have to be extra careful in the macro implementation function if you refactor a legacy macro: you can no longer merely check for the presence of a key in the kwargs dictionary.

Mutation

In symbolic macros, you will not be able to mutate the arguments passed to the macro implementation function.

# defs.bzl
def _simple_macro_impl(name, visibility, env):
    print(type(env), env)
    env["some"] = "more"

simple_macro = macro(
    attrs = {
        "env": attr.string_dict(configurable = False)
    },
    implementation = _simple_macro_impl
)

# BUILD.bazel
simple_macro(name = "tool", env = {"state": "active"})

Let’s check how this would get evaluated:

$ bazel cquery //...
DEBUG: defs.bzl:36:10: dict {"state": "active"}
        File "defs.bzl", line 37, column 17, in _simple_macro_impl
                env["some"] = "more"
Error: trying to mutate a frozen dict value

This, however, is no different to legacy macros where you could not modify mutable objects in place either. In situations like this, creating a new dict with env = dict(env) would be of help.

In legacy macros you can still modify objects in place when they are inside the kwargs , but this arguably leads to code that is harder to reason about and invites subtle bugs that are a nightmare to troubleshoot in a large codebase. See the Mutability in Starlark section to learn more.

This is still possible in legacy macros:

# defs.bzl
def special_test_legacy(name, **kwargs):
    kwargs["name"] = name
    kwargs["env"]["some"] = "more"
    cc_test(**kwargs)

# BUILD.bazel
special_test_legacy("small-test", env = {"state": "active"})

Let’s see how the updated environment variables were set for the cc_test target created in the legacy macro:

$ bazel cquery //test/package:small-test --output=build
...
cc_test(
  name = "small-test",
  ...
  env = {"state": "active", "some": "more"},
)

This is no longer allowed in symbolic macros:

# defs.bzl
def _simple_macro_impl(name, visibility, **kwargs):
    print(type(kwargs["env"]), kwargs["env"])
    kwargs["env"]["some"] = "more"

It would fail to evaluate:

$ bazel cquery //...

DEBUG: defs.bzl:35:10: dict {"state": "active"}
        File "defs.bzl", line 36, column 27, in _simple_macro_impl
                kwargs["env"]["some"] = "more"
Error: trying to mutate a frozen dict value

Configuration

Symbolic macros, just like legacy macros, support configurable attributes , commonly known as select() , a Bazel feature that lets users determine the values of build rule (or macro) attributes at the command line.

Here’s an example symbolic macro with the select toggle:

# defs.bzl
def _special_test_impl(name, **kwargs):
    cc_test(
        name = name,
        **kwargs
    )
special_test = macro(
    inherit_attrs = native.cc_test,
    attrs = {},
    implementation = _special_test_impl,
)

# BUILD.bazel
config_setting(
    name = "linking-static",
    define_values = {"static-testing": "true"},
)

config_setting(
    name = "linking-dynamic",
    define_values = {"static-testing": "false"},
)

special_test(
    name = "my-special-test",
    srcs = ["test.cc"],
    linkstatic = select({
        ":linking-static": True,
        ":linking-dynamic": False,
        "//conditions:default": False,
    }),
)

Let’s see how this expands in the BUILD file:

$ bazel query //test/package:my-special-test --output=build
cc_test(
  name = "my-special-test",
  ...(omitted for brevity)...
  linkstatic = select({
    "//test/package:linking-static": True,
    "//test/package:linking-dynamic": False,
    "//conditions:default": False
  }),
)

The query command does show that the macro was expanded into a cc_test target, but it does not show what the select() is resolved to. For this, we would need to use the cquery (configurable query) which is a variant of query that runs after select() s have been evaluated.

$ bazel cquery //test/package:my-special-test --output=build
cc_test(
  name = "my-special-test",
  ...(omitted for brevity)...
  linkstatic = False,
)

Let’s configure the test to be statically linked:

$ bazel cquery //test/package:my-special-test --output=build --define="static-testing=true"
cc_test(
  name = "my-special-test",
  ...(omitted for brevity)...
  linkstatic = True,
)

Each attribute in the macro function explicitly declares whether it tolerates select() values, in other words, whether it is configurable . For common attributes, consult the Typical attributes defined by most build rules to see which attributes can be configured. Most attributes are configurable, meaning that their values may change when the target is built in different ways; however, there are a handful which are not. For example, you cannot assign a *_test target to be flaky using a select() (e.g., to mark a test as flaky only on aarch64 devices).

Unless specifically declared, all attributes in symbolic macros are configurable (if they support this) which means they will be wrapped in a select() (that simply maps //conditions:default to the single value), and you might need to adjust the code of the legacy macro you migrate. For instance, this legacy code used to append some dependencies with the .append() list method, but this might break:

# defs.bzl
def _simple_macro_impl(name, visibility, **kwargs):
    print(kwargs["deps"])
    kwargs["deps"].append("//:commons")
    cc_test(**kwargs)

simple_macro = macro(
    attrs = {
        "deps": attr.label_list(),
    },
    implementation = _simple_macro_impl,
)

# BUILD.bazel
simple_macro(name = "simple-test", deps = ["//:helpers"])

Let’s evaluate the macro:

$ bazel cquery //...
DEBUG: defs.bzl:35:10: select({"//conditions:default": [Label("//:helpers")]})
        File "defs.bzl", line 36, column 19, in _simple_macro_impl
                kwargs["deps"].append("//:commons")
Error: 'select' value has no field or method 'append'

Keep in mind that select is an opaque object with limited interactivity. It does, however, support modification in place, so that you can extend it, e.g., with kwargs["deps"] += ["//:commons"] :

$ bazel cquery //test/package:simple-test --output=build
...
cc_test(
  name = "simple-test",
  generator_name = "simple-test",
  ...
  deps = ["//:commons", "//:helpers", "@rules_cc//:link_extra_lib"],
)

Be extra vigilant when dealing with attributes of bool type that are configurable because the return type of select converts silently in truthy contexts to True . This can lead to some code being legitimate, but not doing what you intended. See Why does select() always return true? to learn more.

When refactoring, you might need to make an attribute configurable, however, it may stop working using the existing macro implementation. For example, imagine you need to pass different files as input to your macro depending on the configuration specified at runtime:

# defs.bzl
def _deployment_impl(name, visibility, filepath):
    print(filepath)
    # implementation

simple_macro = macro(
    attrs = {
        "filepath": attr.string(),
    },
    implementation = _deployment_impl,
)

# BUILD.bazel
deployment(
    name = "deploy",
    filepath = select({
        "//conditions:default": "deploy/config/dev.ini",
        "//:production": "deploy/config/production.ini",
    }),
)

In rules, select() objects are resolved to their actual values, but in macros, select() creates a special object of type select that isn’t evaluated until the analysis phase, which is why you won’t be able to get actual values out of it.

$ bazel cquery //:deploy
...
select({
    Label("//conditions:default"): "deploy/config/dev.ini",
    Label("//:production"): "deploy/config/production.ini"
    })
...

In some cases, such as when you need to have the select ed value available in the macro function, you can have the select object resolved before it’s passed to the macro. This can be done with the help of an alias target, and the label of a target can be turned into a filepath using the special location variable :

# defs.bzl
def _deployment_impl(name, visibility, filepath):
    print(type(filepath), filepath)
    native.genrule(
        name = name + "_gen",
        srcs = [filepath],
        outs = ["config.out"],
        cmd = "echo '$(location {})' > $@".format(filepath)
    )

deployment = macro(
    attrs = {
        "filepath": attr.label(configurable = False),
    },
    implementation = _deployment_impl,
)

# BUILD.bazel
alias(
    name = "configpath",
    actual = select({
        "//conditions:default": "deploy/config/dev.ini",
        "//:production": "deploy/config/production.ini",
    }),
    visibility = ["//visibility:public"],
)

deployment(
    name = "deploy",
    filepath = ":configpath",
)

You can confirm the right file is chosen when passing different configuration flags before building the target:

$ bazel cquery //tests:configpath --output=build
INFO: Analyzed target //tests:configpath (0 packages loaded, 1 target configured).
...
alias(
  name = "configpath",
  visibility = ["//visibility:public"],
  actual = "//tests:deploy/config/dev.ini",
)
...

$ bazel build //tests:deploy_gen && cat bazel-bin/tests/config.out
...
DEBUG: defs.bzl:29:10: Label //tests:configpath
...
tests/deploy/config/dev.ini

Querying macros

Since macros are evaluated when BUILD files are queried, you cannot use Bazel itself to query “raw” BUILD files. Identifying definitions of legacy macros is quite difficult, as they resemble Starlark functions, but instantiate targets. Using bazel cquery with the --output=starlark might help printing the properties of targets to see if they have been instantiated from macros.

When using --output=build , you can also inspect some of the properties:

  • generator_name (the name attribute of the macro)
  • generator_function (which function generated the rules)
  • generator_location (where the macro was invoked)

This information with some heuristics might help you to identify the macros. Once you have identified the macro name, you can run bazel query --output=build 'attr(generator_function, simple_macro, //...)' to find all targets that are generated by a particular macro. Finding symbolic macros, in contrast, is trivial as you would simply need to grep for macro() function calls in .bzl files.

To query unprocessed BUILD files, you might want to use buildozer which is a tool that lets you query the contents of BUILD files using a static parser. The tool will come in handy for various use cases when refactoring, such as migrating the macros. Because both legacy and symbolic macros follow the same BUILD file syntax, buildozer can be used to query build metadata for either type.

Let’s write some queries for these macro invocations:

# BUILD.bazel
perftest(
  name = "apis",
  srcs = ["//:srcA", "//:srcB"],
  env = {"type": "performance"},
)

perftest(
  name = "backend",
  srcs = ["//:srcC", "//:srcD"],
  env = {"type": "performance"},
)

Print all macro invocations (raw) across the whole workspace:

$ buildozer 'print rule' "//...:%perftest"

perftest(
    name = "apis",
    srcs = [
        "//:srcA",
        "//:srcB",
    ],
    env = {"type": "performance"},
)

perftest(
    name = "backend",
    srcs = [
        "//:srcC",
        "//:srcD",
    ],
    env = {"type": "performance"},
)

Print attribute’s values for all macro invocations:

$ buildozer 'print label srcs' "//...:%perftest"
//test/package:apis [//:srcA //:srcB]
//test/package:backend [//:srcC //:srcD]

Print path to files where macros are invoked:

$ buildozer 'print path' "//...:%perftest" | xargs realpath --relative-to "$PWD" | sort | uniq
test/package/BUILD.bazel

The path can be combined with an attribute, e.g., print path and the srcs to make reviewing easier:

$ buildozer 'print path srcs' "//...:%perftest"
/home/user/code/project/test/package/BUILD.bazel [//:srcA //:srcB]
/home/user/code/project/test/package/BUILD.bazel [//:srcC //:srcD]

Remove an attribute from a macro invocation (e.g., env will be set up in the macro implementation function):

$ buildozer 'remove env' "//...:%perftest"
fixed /home/user/code/project/test/package/BUILD.bazel

You might also want to check that no macro invocation passes an attribute that is not supposed to be passed. In the command output, the missing means the attribute doesn’t exist; these lines can of course be ignored with grep -v missing :

$ buildozer -quiet 'print path env' "//...:%perftest" 2>/dev/null
/home/user/code/project/test/package/BUILD.bazel {"type": "performance"}
/home/user/code/project/test/package/BUILD.bazel (missing)

We hope that these practical suggestions and examples will assist you in your efforts to modernize the use of macros throughout your codebase. Remember that you can compose legacy and symbolic macros, which may be useful during the transition. Also, legacy macros can still be used and are to remain supported in Bazel for the foreseeable future. Some organizations may even choose not to migrate at all, particularly if they rely on the current behavior of the legacy macros heavily.

CiviConf & CiviSprint Paris 5-9 october 2026

CiviCRM
civicrm.org
2025-11-20 15:20:57
Save the Date: CiviConf & CiviSprint Paris – October 5–9, 2026 We're thrilled to announce that the global CiviCRM community is gathering in Paris for CiviConf & CiviSprint Paris 2026! Join us for an inspiring week of collaboration, connection, and learning, set at the ...
Original Article

We're thrilled to announce that the global CiviCRM community is gathering in Paris for CiviConf & CiviSprint Paris 2026 ! Join us for an inspiring week of collaboration, connection, and learning, set at the HI Paris Yves Robert Hostel—just a short walk from Gare du Nord and minutes away from the legendary Montmartre neighbourhood

Dates

Monday, October 5 to Friday, October 9, 2026

Mark your calendar and get ready to be part of the most international CiviCRM event of the year!

Program Highlights

  • Monday, 9:30 AM – 6:00 PM:
    Conference day! Meet partners, discover community innovations, hear real-world CiviCRM stories. The day features open forums, technical showcases, client success sessions, and networking breaks.
  • Tuesday to Friday, 9 AM – 11:00 PM :
    Training and Sprint sessions—choose your track:
    • Advanced User Training (English & French): Boost your skills, learn best practices, and connect with power users and CiviCRM experts.
    • Developer Training (English): Dive into CiviCRM’s technical ecosystem, contribute to the open source codebase, and get hands-on with the latest features.
    • Daily Sprint: Collaborate with global contributors on documentation, core improvements, and translation projects. All skill levels and backgrounds are welcome!
  • Social & Community Experience:
    Experience Paris beyond the conference! Join us for informal outings to nearby Montmartre—only 10 minutes on foot from Gare du Nord—and enjoy the local culture, food, and an energizing Parisian vibe.

Who Should Attend?

  • Non-profit, association and foundation staff
  • CiviCRM administrators and advanced users
  • Developers (PHP, Drupal, WordPress, Joomla, more)
  • Partners, consultants, and tech agencies
  • Community members, old and new

Venue

HI Paris Yves Robert Hostel
20, Esplanade Nathalie Sarraute, 75018 Paris

  • 15 mins walk from Gare du Nord (Eurostar, Airport direct access)
  • 20 mins walk from Gare de l’Est
  • 24 mins by metro from Gare de Lyon
  • Easy access to CDG / Orly airports

Registration and More Info

Registration will open in early 2026—stay tuned for detailed program updates, speaker announcements, and travel tips.

If you’re interested in presenting, sponsoring, or supporting the event, contact us contact@all-in-appli.com

Book your calendars and prepare to meet the global community in Paris!

#CiviConfParis2026 #CiviCRM #OpenSource #Community

Save the date: Paris 2026 CiviConf & CiviSprint (5-9 october 2026)

CiviCRM
civicrm.org
2025-11-20 15:20:57
Save the Date: CiviConf & CiviSprint Paris – October 5–9, 2026 We're thrilled to announce that the global CiviCRM community is gathering in Paris for CiviConf & CiviSprint Paris 2026! Join us for an inspiring week of collaboration, connection, and learning, set at the ...
Original Article

We're thrilled to announce that the global CiviCRM community is gathering in Paris for CiviConf & CiviSprint Paris 2026 ! Join us for an inspiring week of collaboration, connection, and learning, set at the HI Paris Yves Robert Hostel—just a short walk from Gare du Nord and minutes away from the legendary Montmartre neighbourhood

Dates

Monday, October 5 to Friday, October 9, 2026

Mark your calendar and get ready to be part of the most international CiviCRM event of the year!

Program Highlights

  • Monday, 9:30 AM – 6:00 PM:
    Conference day! Meet partners, discover community innovations, hear real-world CiviCRM stories. The day features open forums, technical showcases, client success sessions, and networking breaks.
  • Tuesday to Friday, 9 AM – 11:00 PM :
    Training and Sprint sessions—choose your track:
    • Advanced User Training (English & French): Boost your skills, learn best practices, and connect with power users and CiviCRM experts.
    • Developer Training (English): Dive into CiviCRM’s technical ecosystem, contribute to the open source codebase, and get hands-on with the latest features.
    • Daily Sprint: Collaborate with global contributors on documentation, core improvements, and translation projects. All skill levels and backgrounds are welcome!
  • Social & Community Experience:
    Experience Paris beyond the conference! Join us for informal outings to nearby Montmartre—only 10 minutes on foot from Gare du Nord—and enjoy the local culture, food, and an energizing Parisian vibe.

Who Should Attend?

  • Non-profit, association and foundation staff
  • CiviCRM administrators and advanced users
  • Developers (PHP, Drupal, WordPress, Joomla, more)
  • Partners, consultants, and tech agencies
  • Community members, old and new

Venue

HI Paris Yves Robert Hostel
20, Esplanade Nathalie Sarraute, 75018 Paris

  • 15 mins walk from Gare du Nord (Eurostar, Airport direct access)
  • 20 mins walk from Gare de l’Est
  • 24 mins by metro from Gare de Lyon
  • Easy access to CDG / Orly airports

Registration and More Info

Registration will open in early 2026—stay tuned for detailed program updates, speaker announcements, and travel tips.

If you’re interested in presenting, sponsoring, or supporting the event, contact us contact@all-in-appli.com

Book your calendars and prepare to meet the global community in Paris!

#CiviConfParis2026 #CiviCRM #OpenSource #Community

The Pentagon Can't Trust GPS Anymore

Hacker News
www.wsj.com
2025-11-22 15:19:32
Comments...
Original Article

Please enable JS and disable any ad blocker

Cox Enterprises discloses Oracle E-Business Suite data breach

Bleeping Computer
www.bleepingcomputer.com
2025-11-22 15:16:23
Cox Enterprises is notifying impacted individuals of a data breach that exposed their personal data to hackers who breached the company network after exploiting a zero-day flaw in Oracle E-Business Suite. [...]...
Original Article

Cox

Cox Enterprises is notifying impacted individuals of a data breach that exposed their personal data to hackers who breached the company network after exploiting a zero-day flaw in Oracle E-Business Suite.

The compromise occurred in August, but the company didn’t detect the intrusion until late September, when it launched its internal investigation.

“On September 29, 2025, we became aware of suspicious activity involving Oracle’s E-Business Suite, which is a platform we use for some of our back-office business operations,” reads the notice.

Wiz

“We learned the suspicious activity was the result of cybercriminals taking advantage of a previously unknown security flaw (called a “zero-day” vulnerability) in Oracle’s E-Business Suite between Aug. 9-14, 2025.”

Cox Enterprises is a major American conglomerate engaged in media (Cox Media Group), telecommunications (Cox Communications), and automotive services (Cox Automotive).

The company has 55,000 employees and an annual revenue of $23 billion, with its businesses having an international reach.

The company has not named the attackers, but the Cl0p ransomware has taken credit for exploiting CVE-2025-61882 as a zero-day vulnerability, long before Oracle released a patch on October 5.

Cl0p hackers are known for leveraging zero-days in popular software products used by a large number of organizations.

Incidents where Cl0p exploited unknown vulnerabilities include the Cleo file transfer in 2024, the MOVEit Transfer and GoAnywhere MFT in 2023, the SolarWinds Serv-U FTP in 2021, and the Accellion FTA in 2020.

Breaches related to Oracle E-Business Suite have been confirmed by multiple companies, among them Logitech , Washington Post , GlobalLogic , Envoy Air , and Harvard University .

The threat actor added Cox Enterprises to their data leak website on the dark web on October 27 and published the stolen information.

Clop
Entry on Cl0p extortion portal
Source: BleepingComputer.com

Earlier today, Cl0p listed 29 new companies as their victims. The batch includes major organizations in the automotive, software, and technology sectors.

In the notification to 9,479 impacted individuals, Cox provides instructions on how to enroll in identity theft protection and credit monitoring services through IDX at no cost for 12 months.

The company did not specify what types of data were exposed in the notification sample shared with authorities .

Cox Communications suffered a separate breach in June 2024 , in which attackers exploited an exposed backend API to reset millions of customer modems and steal their personal data.

Earlier, in October 2021, Cox Media Group was hit by ransomware that impacted live TV and radio broadcast streams.

Wiz

7 Security Best Practices for MCP

As MCP (Model Context Protocol) becomes the standard for connecting LLMs to tools and data, security teams are moving fast to keep these new services safe.

This free cheat sheet outlines 7 best practices you can start using today.

'The French people want to save us': help pours in for glassmaker Duralex

Hacker News
www.theguardian.com
2025-11-22 15:14:45
Comments...
Original Article

D rop a Duralex glass and it will most likely bounce, not break. The French company itself has tumbled several times in the past two decades and always bounced back, but never quite as spectacularly as when, earlier this month, it asked the public for money.

An appeal for €5m (£4.4m) of emergency funding to secure the immediate future of the glassworks took just five hours and 40 minutes to reach its target. Within 48 hours, the total amount pledged had topped €19m.

François Marciano, 59, the director general of Duralex, said the response had astonished everyone at the company. “We thought it would take five or six weeks to raise the €5m. When it reached nearly €20m we had to say stop. Enough,” he said.

Man holds orange glass tumbler.
François Marciano, chief executive of Duralex, holds up a Picardie glass. Photograph: Magali Delporte/The Guardian

As a staff cooperative, €5m is the maximum Duralex can accept in public investment under financial rules.

Beloved French brand

Mention Duralex to any French person and they will be transported back to childhood and a school canteen. The brand evokes a mix of nostalgia and pride and is a symbol of French patriotism and industrial savoir faire.

“We’re like Proust’s madeleines ,” Marciano said. “The French people want to save us. They are fed up with factories closing and the country’s industries declining.”

At the Duralex factory on an industrial estate in La Chapelle-Saint-Mesmin on the banks of the Loire just outside Orléans, Marciano says he and his colleagues are “floating on a cloud” after the appeal.

Eighteen months ago, Marciano oversaw a staff buyout of the company, which had been placed in receivership for the fourth time in 20 years. Today, 180 of the 243 employees are “associates” in the company.

Suliman El Moussaoui stands next to machinery.
Suliman El Moussaoui, the leader of the CFDT union at Duralex. Photograph: Magali Delporte/The Guardian

Suliman El Moussaoui, 44, a union representative at the factory where he has worked for 18 years, said the appeal had prompted “a tsunami of orders, so many that we’re struggling to keep up. Every time the company is mentioned on the television or radio we have more orders. It’s been amazing.”

Inside the factory, a simple but magical alchemy takes place. A mix of sand, soda ash and limestone, the exact proportions of which are a closely guarded secret, is heated in a vast overhead oven to 1,400C. Glowing globs of molten glass drop into iron casts that are blasted with a flame of gas. The red-hot glass is instantly pounded into shape, sprung from the mould, snatched by metal pincers and placed on a conveyor belt.

Glass tumblers are carried and dropped by factory machine.
The iconic Picardie glasses coming out of the 1,440-degree oven Photograph: Magali Delporte/The Guardian
Duralex production video

The process has changed little since Duralex – which is said to take its name from the Latin expression Dura lex, sed lex , meaning “the law is harsh, but it is the law” – opened in 1945. When the Guardian visited, the production line was turning out small clear glasses in the Provence range.

A worker operates machinery.
Workers in the factory starting a new production of glasses. Photograph: Magali Delporte/The Guardian
A worker holds a glass up to the light.
Each glass is carefully inspected. Photograph: Magali Delporte/The Guardian

A worker brandishing tongs lifted a glass to the light to inspect it for faults. During a production run, more than a dozen samples of whatever is being made – glasses, plates, bowls – will be randomly removed and subjected to stress tests. In the quality control room, they will be heated to 150C then plunged into cold water to see if they resist a thermic shock, and dropped from the height of a kitchen counter on to a metal sheet to see if they shatter. They will be tested for stackability and then weighed and the glass thickness measured. If they pass, they are thrown in a bin and the production line is given a thumbs up. If they fail, everything stops and the machines are recalibrated.

‘The ultimate drinking vessel’

It is not known who invented the company’s trademark Picardie glass, the tumbler used in school canteens with a thick curved rim and semi-fluted shape that first appeared in 1954. The British design guru Patrick Taylor has ranked the Picardie alongside Levi’s jeans and the Swiss Army knife as an icon of modern design. Taylor describes it as: “An object whose form gives the impression it was discovered rather than designed. It is the ultimate drinking vessel created by man, and of its type cannot be improved.”

Production of the iconic Picardie glasses
The instantly recognisable Picardie glasses. Photograph: Magali Delporte/The Guardian

Duralex says its glass is microwave, freezer and dishwasher-safe and will not turn cloudy or lose its colour, which is in the glass rather than on it. When they do break, Duralex glasses shatter into small pieces rather than shards, reducing the injury risk.

Joël Cardon, 59, who has worked at the factory for 35 years, said the soaring cost of gas and electricity were the firm’s largest and most worrying expense.

On his screen, the oven containing the liquid glass showed a temperature of 1,440C. It can never be allowed to cool or the glass will solidify. Another screen showed the factory was using 360 cubic metres of gas an hour. According to the regulator Ofgem, the average UK house uses 97.3 cubic metres of gas a year.

Glasses in front of window into factory.
Duralex’s oven, in the background, is held at a temperature of 1,440C. Photograph: Magali Delporte/The Guardian

Last weekend, potential investors were asked to come good on their promises on a first come, first served basis. They will be issued with securities that pay 8% interest over seven years but give no company voting rights. The maximum investment was set at €1,000.

“We want to involve as many people as possible but with almost €20m in pledges obviously some people will be disappointed,” Marciano said.

Since the company became a staff cooperative, turnover has increased by 22% and Marciano said he hoped Duralex would be breaking even by 2027.

The €5m raised will be used to modernise the factory and develop new products. These include a partnership with the Élysée presidential palace shop to sell a set of three of its Gigogne glasses in red, white and blue, marked RF for République Française.

Three glasses with red, white or blue bands.
Set of 3 Gigogne glasses in Tricolor, advertised online for €24.90 Photograph: boutique.elysee.fr

Duralex plans to commission moulds to make “pint” glasses with a measure line for British pubs and bars and the US, both regions identified by the company as untapped markets.

“Selling abroad is more difficult because there isn’t the same nostalgia for Duralex as there is in France ,” said Vincent Vallin, the head of strategy and development. “Interest in the company is high and this is positive, but now we have to focus on increasing sales.”

A looming 'insect apocalypse' could endanger global food supplies

Hacker News
www.livescience.com
2025-11-22 14:56:39
Comments...
Original Article
An illustration of a person pulling back a curtain full of colorful insects to reveal a deserted landscape
Insect populations are plummeting almost everywhere they've been studied. That portends a bleak future for the world's food supplies. But there are ways to reverse the decline. (Image credit: Myriam Wares)

Imagine driving down a highway in the summer. The windows are down, the music is loud, and the wind is whipping through your hair. Now picture your car's windshield. You might expect to see a handful of splats from unfortunate bugs. But 30 years ago, there would have been significantly more buggy skid marks plastered on the front of your vehicle.

"When I was a kid, you could go out driving in the summer, and you would come home and your car windshield was covered in bugs," said Cheryl Schultz , an ecologist at Washington State University. "Now, you can go across many areas at the same time of year and your windshield is clean."

This phenomenon, called the "windshield test," is indicative of a larger, very worrying trend: Insects, particularly the flying ones that pollinate many crops, are in steep decline. This nosedive is disrupting ecosystems around the world, and could jeopardize the global food supply. But tracking the decrease of insect populations over the past three decades has proved tricky — and stopping the decline may be even harder.

However, researchers are working quickly to find ways to stem the tide and even reverse the trend. Key to that is a collaborative approach that includes local and federal conservation efforts, new pollinator habitats, and a reduction in pesticide use.

The age of the "insect apocalypse"

Both the total number of insects and the number of insect species have been declining for decades in pretty much every place scientists have looked — prompting researchers to dub it "the insect apocalypse." Global bee biodiversity is down 25% compared with pre-1995 numbers, according to research published in 2021. A sweeping 2025 study showed that butterfly abundance across the U.S. fell by 22% over the past two decades. And a study in Germany found a whopping 76% loss of flying insects in some of the country's forested areas over 27 years.

"It's a worrisome thing," Scott Black , executive director of the nonprofit Xerces Society for Invertebrate Conservation, told Live Science.

By and large, experts know why insects are becoming scarcer. The first factor is climate change . As the planet warms, key host plants for insects start to bloom earlier each year. This can cause a mismatch in life cycles for certain species, putting many newly hatched or metamorphosed bugs out of sync with their food sources. And extreme heat, reduced snowpack, severe storms and megadroughts can chip away at previously robust insect numbers. Many populations simply can't keep up. Meanwhile, milder winters can benefit a few adaptable pest species , which may outcompete sensitive insects and wreak ecological and agricultural havoc in some regions.

Get the world’s most fascinating discoveries delivered straight to your inbox.

a close-up of a windshield splattered with bugs

A rough way to gauge insect abundance is called the "windshield" or "splat test." Windshields now have far fewer buggy skid marks than they did 30 years ago, a sign of significant insect population declines. (Image credit: Dina Ivanova via Getty Images)

The second driver is habitat loss — the inexorable creep of urbanization, deforestation and sterile suburban lawns, which host fewer and less-diverse ranges of insects. As humans encroach on insect habitats, insects like ground-dwelling bees are left without space to build nests, rear young and overwinter, leading to population declines.

an image that says &quot;Science Spotlight&quot; with a blue and yellow gradient background

Science Spotlight takes a deeper look at emerging science and gives you, our readers, the perspective you need on these advances. Our stories highlight trends in different fields, how new research is changing old ideas, and how the picture of the world we live in is being transformed thanks to science.

Finally, there are pesticides. For instance, neonicotinoids (often labeled as the active ingredients acetamiprid, clothianidin, dinotefuran, imidacloprid and thiamethoxam), have been identified as a major threat to wild bees, and they're still used in the U.S. and some other industrialized countries, including parts of Canada and Australia. Other pesticides, like the common weed killer glyphosate, have been shown to weaken bees' ability to regulate hive temperature , leaving them vulnerable to plunging winter temperatures.

"It's really extremely rapid environmental changes that we're seeing," Roel van Klink , a researcher at the German Center for Integrative Biodiversity Research, told Live Science. "Those species that were adapted to the conditions that we had maybe 50 or 100 years ago are not adapted to the conditions now anymore. And so they go down."

Collecting data on the scale and scope of these declines has been challenging, however. For one thing, some insects are easier to find than others. Flying insects like beetles and dragonflies are much more mobile, and therefore easier to spot, than earthbound bugs like earwigs and ants. Likewise, charismatic insects like bees and butterflies tend to have more historical records of their numbers and are usually easier to identify.

But there's another reason these insects' declines have gotten more scientific attention: They are extremely important for global food security.

The importance of diverse pollinators

Disappearing insects are bad news for the global food system. As the world's population continues to grow, the stress that insect declines — and dropping pollinator numbers, in particular — put on the food system could lead to an agricultural economic collapse, as well as increased food scarcity.

"Preventing further declines is no longer enough," Francesca Mancini , an ecological modeler at the UK Centre for Ecology & Hydrology, told Live Science. "We need to restore insect biodiversity to past levels."

In the U.K. alone, insect pollinators provide an estimated $1 billion in economic value each year, Mancini said. For the U.S., it's in the ballpark of $34 billion .

a close-up of a cacao flower

Cacao flowers are completely reliant on a species of fly for pollination. (Image credit: Helder Faria via Getty Images)

Worldwide, three-quarters of the crops we eat — and just over one-third of total crop yields — depend on pollination by insects.The degree to which these crops rely on pollinators falls along a spectrum. Some, like soybeans, would be much less productive without insect pollination. Others would cease to exist. "Coffee and chocolate are actually 100% dependent on pollination by insects," van Klink said.

A lot of that pollination work is done by managed European honeybees ( Apis mellifera ), which beekeepers around the world diligently maintain, transport and unleash upon fields across the globe each year. But to flourish, many crops need more than just honeybees.

For example, fruits native to North America, like blueberries and tomatoes ( which is technically a fruit ), are more effectively pollinated by native bumblebees, such as Bombus fraternus . That's because bumblebees can perform what's known as " buzz pollination ," where they land on a flower and vibrate rapidly to release even the most deeply held pollen grains. Cacao trees ( Theobroma cacao ) — the source of the cocoa beans used to make chocolate — are entirely pollinated by chocolate midges. And cotton yields would plummet by up to 50% without butterfly pollinators .

Some staple crops, like soybeans, can make it without insects. However, research has shown that soybean fields visited by pollinators have significantly higher yields .

a close-up of an alfalfa leaf-cutting bee on a purple flower

Alfalfa fields must be pollinated, yet honeybees aren't the best insects to do the job. Crop yields rise significantly when the alfalfa leaf-cutting bee ( Megachile rotundata ) is involved in the pollination. (Image credit: Tanja Nik via Getty Images)

Then, there are crops like alfalfa ( Medicago sativa ). This legume isn't widely consumed by humans, but it is a staple for livestock — particularly dairy and beef cattle. Like blueberries and tomatoes, alfalfa depends on insect pollinators to thrive. However, honeybees will only pollinate it reluctantly; given the choice, they'd rather buzz around plants with flowers that are easier for them to access. But wild bees, particularly the alfalfa leaf-cutting bee ( Megachile rotundata ), are extremely effective alfalfa pollinators.

A recent study found that alfalfa fields visited by a mix of honeybees, wild bees and other pollinators, like wasps and butterflies, produced significantly more and larger seeds than fields visited by honeybees alone. This higher yield translates to more food for cattle — and thus more milk, cheeseburgers and steaks for us.

Glimmers of hope

Of course, restoring insect abundance and biodiversity is no easy task, especially in the face of an all-encompassing threat like global climate change. Experts told Live Science that coordinated federal regulations aimed at slowing climate change, reducing industrial pesticide use, and preventing the destruction of wild spaces are essential for protecting insects. But there are also actions people can take at the local and personal level that can have a positive impact.

Although the current U.S. administration's cuts to federal science programs and green energy have dealt a harsh blow to progress on these fronts, many experts still see reasons for optimism.

"As much as the overall picture is overwhelming, there's lots of places for hope," Schultz told Live Science. In a detailed report about the state of U.S. butterflies written this year in collaboration with the Xerces Society, Schultz highlighted a number of "success stories" — species that bucked the trend and increased in abundance thanks to years of focused work at both the federal and local levels.

Chief among them is the Fender's blue ( Icaricia icarioides fenderi ), a tiny azure butterfly native to Oregon. In 2000, the U.S. Fish and Wildlife Service listed it as endangered. In 2023, it became the second-ever insect to be downlisted to "threatened."

And the benefits of conservation efforts for one species had knock-on effects: Of the 342 butterfly species and subspecies analyzed in the report, 65 others had increased in number, and most were not on the endangered species list. This suggests that protections to conserve one insect could benefit others as well.

a close-up of a Fender&#039;s blue butterfly

The Fender's blue butterfly ( Icaricia icarioides fenderi ) , native to Oregon, was listed as endangered in 2000. But thanks to concerted conservation measures, the population has recovered somewhat. A new report found that those conservation efforts also improved the population numbers of dozens of other insect species. (Image credit: U.S. Army Corps of Engineers)

Increasing healthy habitat

One of the best ways to help butterflies and other pollinators is to create more habitat for them. Unlike grizzly bears or elk, these insects don't need large stretches of unbroken wilderness . Even something as small as a backyard butterfly garden or a flower-filled window box can go a long way, Wendy Leuenberger , an ecologist at Michigan State University, told Live Science.

One study in the Pacific Northwest found that converting a 5,400-square-foot (500 square meter) plot of land — roughly half the size of the average American lawn — into an insect-friendly habitat full of native or wild plants can increase pollinator species' richness and abundance by about 90%. However, that effect was fairly localized, and it dissipated when these patches were placed in plots of more than 150,000 square meters (37 acres) — about the size of seven or eight blocks in Chicago.

Some pollinators, like hoverflies ( Syrphidae spp. ) and certain types of bees, can cover miles in search of flowering plants. But others, including many butterflies, tend to stay closer to home — within a 650-foot 200 meter radius for more delicate species. This suggests that plots of native or wild flora are most effective at bolstering our food supply when interspersed within larger agricultural fields.

a close-up of a hoverfly on a flower

Hoverflies are incidental pollinators that help boost production of apples and strawberries. (Image credit: Victoria Caruso via Getty Images)

"I would say it's the closer, the better for your crops," Andy Grinstead , a conservation manager at Pollinator Partnership, told Live Science.

In agricultural communities, experts like Grinstead recommend planting "buffer strips" of native vegetation near (or, if possible, in between) crops. He also suggests planting hedgerows of woody, flowering plants around fields to act as both pollinator habitat and wind protection. But you don't have to be a farmer to support pollinators. Folks living within a few miles of farms can plant "bee lawns," which are filled with low-growing flowering plants like clover, instead of pure turfgrass.

And for those without yards, growing micro-plots of native wildflowers — even just a pot on a rooftop or balcony or hanging from a window — can create green "stepping stones" for bees, hoverflies, migratory butterflies and beetles passing through urban areas.

"Pollinator-friendly practices are valuable across all landscapes," Grinstead said. "It takes very little space to actually make an impact."

Reducing pesticide use on an industrial scale can also benefit pollinators, Black said.

One way to do this is to adopt an integrated pest management framework . This can mean rotating crops to keep soil healthy; accurately identifying pests before applying pesticides; and carefully spraying in targeted areas (away from blooms) when the wind is low to prevent the pesticides from drifting into the surrounding environment.

But even home gardeners can help reduce pesticides by replacing lawns or ornamental plants with hardier native species, hand-weeding rather than blanket-spraying small plots, and using screens or draining standing water instead of spraying for pests like mosquitoes, Black said. Taken together, these actions can help create havens where pollinators can thrive.

Taking action

Crucially, scientists are still researching the full scope of global insect declines, especially for species that have been historically understudied. This means we need field research to estimate insect numbers, Black said.

Community pollinator counts, whether as part of a formal program or through apps like iNaturalist, are also essential, Leuenberger told Live Science. These data help experts pinpoint which species are most vulnerable and which conservation efforts are most effective.

But with the future of the global food system hanging in the balance, it's important to try to restore these numbers now — not wait till researchers have published comprehensive data on how and where insect numbers are plummeting, Black said. "We don't want to wait until we have everything tucked into a perfect paper before we take action," he said. "We know how to take action."

Joanna Thompson is a science journalist and runner based in New York. She holds a B.S. in Zoology and a B.A. in Creative Writing from North Carolina State University, as well as a Master's in Science Journalism from NYU's Science, Health and Environmental Reporting Program. Find more of her work in Scientific American, The Daily Beast, Atlas Obscura or Audubon Magazine.

The Resistible Rise of Donald Trump

Portside
portside.org
2025-11-22 14:37:13
The Resistible Rise of Donald Trump Kurt Stand Sat, 11/22/2025 - 09:37 ...
Original Article

Official 2025 inaugural portrait of Donald Trump - Credit Wikimedia CC \ Daniel Torok

Nearly a full year since Trump’s re-election, his drive to consolidate absolute executive power continues.  The leaders of the armed forces have been given notice that they will be deployed in American cities.  Roving paramilitary gangs of armed and masked federal ICE and border patrol agents are sowing fear and chaos in immigrant communities.  Trump and his minions are issuing ukases against organisations of “domestic terrorists” (namely, anyone who resists the imposition of authoritarian tyranny).

Trump and his enablers now proudly acknowledge the ambitious Project 2025 blueprints for seizing power that prior to his election, Trump claimed not to have even read.   The checks and balances intended to limit usurpation of power in constitutional governments appear ineffective against the daily onslaught of executive orders and presidential social media posts.

Rulings by lower court judges have temporarily blocked the implementation of some of the most outrageous miscarriages of justice.  However, the Supreme Court has a majority that often overrules any limitations on presidential powers.  The Republican majorities in the House and Senate generally go along with whatever Trump asks them to do.  The leaders of the Democratic minority in Congress meekly protest but seem content to wait for future elections to return them to majority status.

Many major media outlets, law firms and universities have bowed their heads and bent their knees to the new order, allowing Trump to extort major financial and policy concessions by withholding federal funds or threats of prosecution.

A bleak picture indeed, but not one without hope. While elite institutions capitulate, resistance to Trump’s neo-fascist agenda is building among large segments of the population.

On “No Kings Day”, October 18, close to seven million people rallied in more than a thousand towns and cities. It was promoted by a host of progressive organizations, some organized into local hubs capable of following up.  For example, Indivisible has conducted mass conference calls with thousands of participants to teach methods of non-compliance.

While people participated for many reasons, my own personal experience suggests that a major organizing theme was resistance to the assaults on our immigrant neighbors.  Throughout the USA, in large cities and small towns, tens of thousands have mobilized against ICE raids into immigrant support networks.

The ongoing successes of the Fighting Oligarchy tours, led by Bernie Sanders and various younger Left members of Congress, in mobilizing large rallies in contested purple and Republican red states indicate the widespread potential of anti-authoritarian politics.

Under the pretext of “fighting crime”, Trump is using National Guard units and threatens to use even the professional military itself in suppressing popular resistance in cities with Democratic mayors.   Though unlikely to result in armed confrontations, his dangerous strategy risks further escalation into warring camps while trying to provoke sufficient civil disorders to justify further militarized interventions.  Organizational resistance is emerging mostly from big cities in blue states as Democratic mayors and governors denounce the unpopular shows of force by federal stormtroopers in Los Angeles, Chicago, and Portland.  While unable to prevent such encroachments, they can help legitimate community resistance to them. To date, civil opponents have maintained enough self-discipline to thwart Trump’s reckless schemes.

Zohran Mamdani is likely to be elected as mayor of New York City on 4th November.   Campaigning for rent control, availability and affordability of food and housing for working-class New Yorkers, Mamdani’s candidacy has crystallized the hopes of many and aroused fears in others.   The billionaire donor class and the pro-Netanyahu diehards of AIPAC are spending millions of dollars to derail Mamdani’s candidacy.  Trump is blowing a lot of smoke about how he will not permit a “Communist Hamas supporter” from taking office.   Trump is tacitly in an unholy alliance with Democratic Congressional leaders Senator Chuck Schumer and Representative Hakeem Jeffries to block Mamdani, who decisively defeated both corrupt former New York governor Andrew Cuomo and discredited current NYC mayor Eric Adams in the Democratic primary.

By rejecting the choice of the New York City Democratic electorate, Schumer and Jeffries are widening the rift within the Democratic Party between those dependent on corporate donors who fear the possible negative blowback elsewhere in the country from a victorious democratic socialist in a prominent elected office and those who advocate a more populist, pro-working-class politics for the Democratic Party. Some DSA members outside New York have expressed serious angst over Mamdani’s campaign.   Unwilling to attack Mamdani directly, ultra-left caucuses are trotting out stale platitudes about the inevitable evils of Mamdani running in a Democratic primary, and the undeniable fact that no election will bring about a “Revolution”.  At the DSA Convention, they focused their sectarian fire on an increasingly pragmatic Alexandria Ocasio-Cortez as a safer target.

The prospect of a skilled and charismatic politician with impeccable DSA credentials becoming mayor of New York City poses challenges across the American political spectrum.   By the time you read this report, you may know better what comes next.

Chartist is the bi-monthly political magazine of the democratic left. In honouring the Chartists of the 19th century, our idea of democratic socialism is as much about the political movement and means of mobilisation used to advance political ideas as it is about the ideas themselves. Chartist seeks to provide a space for those who subscribe to this broad ideal.

Meet the AI workers who tell their friends and family to stay away from AI

Guardian
www.theguardian.com
2025-11-22 14:00:40
When the people making AI seem trustworthy are the ones who trust it the least, it shows that incentives for speed are overtaking safety, experts say Krista Pawloski remembers the single defining moment that shaped her opinion on the ethics of artificial intelligence. As an AI worker on Amazon Mecha...
Original Article

K rista Pawloski remembers the single defining moment that shaped her opinion on the ethics of artificial intelligence . As an AI worker on Amazon Mechanical Turk – a marketplace that allows companies to hire workers to perform tasks like entering data or matching an AI prompt with its output – Pawloski spends her time moderating and assessing the quality of AI-generated text, images and videos, as well as some factchecking.

Roughly two years ago, while working from home at her dining room table, she took up a job designating tweets as racist or not. When she was presented with a tweet that read “Listen to that mooncricket sing”, she almost clicked on the “no” button before deciding to check the meaning of the word “mooncricket”, which, to her surprise, was a racial slur against Black Americans.

“I sat there considering how many times I may have made the same mistake and not caught myself,” said Pawloski.

The potential scale of her own errors and those of thousands of other workers like her made Pawloski spiral. How many others had unknowingly let offensive material slip by? Or worse, chosen to allow it?

After years of witnessing the inner workings of AI models, Pawloski decided to no longer use generative AI products personally and tells her family to steer clear of them.

“It’s an absolute no in my house,” said Pawloski, referring to how she doesn’t let her teenage daughter use tools like ChatGPT . And with the people she meets socially, she encourages them to ask AI about something they are very knowledgable in so they can spot its errors and understand for themselves how fallible the tech is. Pawloski said that every time she sees a menu of new tasks to choose from on the Mechanical Turk site, she asks herself if there is any way what she’s doing could be used to hurt people – many times, she says, the answer is yes.

A statement from Amazon said that workers can choose which tasks to complete at their discretion and review a task’s details before accepting it. Requesters set the specifics of any given task, such as allotted time, pay and instruction levels, according to Amazon.

“Amazon Mechanical Turk is a marketplace that connects businesses and researchers, called requesters, with workers to complete online tasks, such as labeling images, answering surveys, transcribing text or reviewing AI outputs,” said Montana MacLachlan, an Amazon spokesperson.

Pawloski isn’t alone. A dozen AI raters , workers who check an AI’s responses for accuracy and groundedness, told the Guardian that, after becoming aware of the way chatbots and image generators function and just how wrong their output can be, they have begun urging their friends and family not to use generative AI at all – or at least trying to educate their loved ones on using it cautiously. These trainers work on a range of AI models – Google’s Gemini, Elon Musk’s Grok, other popular models, and several smaller or lesser-known bots.

One worker, an AI rater with Google who evaluates the responses generated by Google Search’s AI Overviews, said that she tries to use AI as sparingly as possible, if at all. The company’s approach to AI-generated responses to questions of health, in particular, gave her pause, she said, requesting anonymity for fear of professional reprisal. She said she observed her colleagues evaluating AI-generated responses to medical matters uncritically and was tasked with evaluating such questions herself, despite a lack of medical training.

At home, she has forbidden her 10-year-old daughter from using chatbots. “She has to learn critical thinking skills first or she won’t be able to tell if the output is any good,” the rater said.

“Ratings are just one of many aggregated data points that help us measure how well our systems are working, but do not directly impact our algorithms or models,” a statement from Google reads. “We also have a range of strong protections in place to surface high quality information across our products.”

Bot watchers sound the alarm

These people are part of a global workforce of tens of thousands who help chatbots sound more human. When checking AI responses, they also try their best to ensure that a chatbot doesn’t spout inaccurate or harmful information.

When the people who make AI seem trustworthy are those who trust it the least, however, experts believe it signals a much larger issue.

“It shows there are probably incentives to ship and scale over slow, careful validation, and that the feedback raters give is getting ignored,” said Alex Mahadevan, director of MediaWise at Poynter, a media literacy program. “So this means when we see the final [version of the] chatbot, we can expect the same type of errors they’re experiencing. It does not bode well for a public that is increasingly going to LLMs for news and information.”

AI workers said they distrust the models they work on because of a consistent emphasis on rapid turnaround time at the expense of quality. Brook Hansen, an AI worker on Amazon Mechanical Turk, explained that while she doesn’t mistrust generative AI as a concept, she also doesn’t trust the companies that develop and deploy these tools. For her, the biggest turning point was realizing how little support the people training these systems receive.

“We’re expected to help make the model better, yet we’re often given vague or incomplete instructions, minimal training and unrealistic time limits to complete tasks,” said Hansen, who has been doing data work since 2010 and has had a part in training some of Silicon Valley’s most popular AI models. “If workers aren’t equipped with the information, resources and time we need, how can the outcomes possibly be safe, accurate or ethical? For me, that gap between what’s expected of us and what we’re actually given to do the job is a clear sign that companies are prioritizing speed and profit over responsibility and quality.”

Dispensing false information in a confident tone, rather than offering no answer when none is readily available, is a major flaw of generative AI, experts say. An audit of the top 10 generative AI models including ChatGPT, Gemini and Meta’s AI by the media literacy non-profit NewsGuard revealed that the non-response rates of chatbots went down from 31% in August 2024 to 0% in August 2025. At the same time, the chatbots’ likelihood of repeating false information almost doubled from 18% to 35% , NewsGuard found. None of the companies responded to NewsGuard’s request for a comment at the time.

“I wouldn’t trust any facts [the bot] offers up without checking them myself – it’s just not reliable,” said another Google AI rater, requesting anonymity due to a nondisclosure agreement she has signed with the contracting company. She warns people about using it and echoed another rater’s point about people with only cursory knowledge being tasked with medical questions and sensitive ethical ones, too. “This is not an ethical robot. It’s just a robot.”

skip past newsletter promotion

“We joke that [chatbots] would be great if we could get them to stop lying,” said one AI tutor who has worked with Gemini, ChatGPT and Grok, requesting anonymity, having signed nondisclosure agreements.

‘Garbage in, garbage out’

Another AI rater who started his journey rating responses for Google’s products in early 2024 began to feel he couldn’t trust AI around six months into the job. He was tasked with stumping the model – meaning he had to ask Google’s AI various questions that would expose its limitations or weaknesses. Having a degree in history, this worker asked the model historical questions for the task.

“I asked it about the history of the Palestinian people, and it wouldn’t give me an answer no matter how I rephrased the question,” recalled this worker, requesting anonymity, having signed a nondisclosure agreement. “When I asked it about the history of Israel, it had no problems giving me a very extensive rundown. We reported it, but nobody seemed to care at Google.” When asked specifically about the situation the rater described, Google did not issue a statement.

For this Google worker, the biggest concern with AI training is the feedback given to AI models by raters like him. “After having seen how bad the data is that goes into supposedly training the model, I knew there was absolutely no way it could ever be trained correctly like that,” he said. He used the term “garbage in, garbage out”, a principle in computer programming which explains that if you feed bad or incomplete data into a technical system, then the output would also have the same flaws.

The rater avoids using generative AI and has also “advised every family member and friend of mine to not buy newer phones that have AI integrated in them, to resist automatic updates if possible that add AI integration, and to not tell AI anything personal”, he said.

Fragile, not futuristic

Whenever the topic of AI comes up in a social conversation, Hansen reminds people that AI is not magic – explaining the army of invisible workers behind it, the unreliability of the information and how environmentally damaging it is .

“Once you’ve seen how these systems are cobbled together – the biases, the rushed timelines, the constant compromises – you stop seeing AI as futuristic and start seeing it as fragile,” said Adio Dinika, who studies the labor behind AI at the Distributed AI Research Institute, about people who work behind the scenes. “In my experience it’s always people who don’t understand AI who are enchanted by it.”

The AI workers who spoke to the Guardian said they are taking it upon themselves to make better choices and create awareness around them, particularly emphasizing the idea that AI, in Hansen’s words, “is only as good as what’s put into it, and what’s put into it is not always the best information”. She and Pawloski gave a presentation in May at the Michigan Association of School Boards spring conference. In a room full of school board members and administrators from across the state, they spoke about the ethical and environmental impacts of artificial intelligence, hoping to spark a conversation.

“Many attendees were shocked by what they learned, since most had never heard about the human labor or environmental footprint behind AI,” said Hansen. “Some were grateful for the insight, while others were defensive or frustrated, accusing us of being ‘doom and gloom’ about technology they saw as exciting and full of potential.”

Pawloski compares AI ethics to that of the textile industry: when people didn’t know how cheap clothes were made, they were happy to find the best deal and save a few bucks. But as the stories of sweatshops started coming out, consumers had a choice and knew they should be asking questions. She believes it’s the same for AI.

“Where does your data come from? Is this model built on copyright infringement? Were workers fairly compensated for their work?” she said. “We are just starting to ask those questions, so in most cases the general public does not have access to the truth, but just like the textile industry, if we keep asking and pushing, change is possible.”

A Lost Planet Created the Moon. Now, We Know Where It Came From.

403 Media
www.404media.co
2025-11-22 14:00:14
The remains of Theia are scattered deep inside the Earth and its satellite. By analyzing these remnants, scientists have proposed an origin....
Original Article

Welcome back to the Abstract! Here are the studies this week that overthrew the regime, survived outer space, smashed planets, and crafted an ancient mystery from clay.

First, a queen gets sprayed with acid—and that’s not even the most horrifying part of the story. Then: a moss garden that is out of this world, the big boom that made the Moon, and a breakthrough in the history of goose-human relations.

As always, for more of my work, check out my book First Contact: The Story of Our Obsession with Aliens , or subscribe to my personal newsletter the BeX Files .

What is this, a regime change for ants?

Shimada, Taku et al. “Socially parasitic ant queens chemically induce queen-matricide in host workers.” Current Biology.

Every so often, a study opens with such a forceful hook that it is simply best for me to stand aside and allow it to speak for itself. Thus:

“Matricide—the killing of a mother by her own genetic offspring—is rarely observed in nature, but not unheard-of. Among animal species in which offspring remain with their mothers, the benefits gained from maternal care are so substantial that eliminating the mother almost never pays, making matricide vastly rarer than infanticide.”

“Here, we report matricidal behavior in two ant species, Lasius flavus and Lasius japonicus , where workers kill resident queens (their mothers) after the latter have been sprayed with abdominal fluid by parasitic ant queens of the ants Lasius orientalis and Lasius umbratus .”

Mad props to this team for condensing an entire etymological epic into three sentences. Such murderous acts of dynastic usurpation were first observed by Taku Shimada, an ant enthusiast who runs a blog called Ant Room . Though matricide is sometimes part of a life cycle—like mommy spiders sacrificing their bodies for consumption by their offspring—there is no clear precedent for the newly-reported form of matricide, in which neither the young nor mother benefits from an evolutionary point of view.

In what reads like an unfolding horror, the invading parasitic queens “covertly approach the resident queen and spray multiple jets of abdominal fluid at her”—formic acid, as it turns out—that then “elicits abrupt attacks by host workers, which ultimately kill their own mother,” report Shimada and his colleagues.

“The parasitic queens are then accepted, receive care from the orphaned host workers and produce their own brood to found a new colony,” the team said. “Our findings are the first to document a novel host manipulation that prompts offspring to kill an otherwise indispensable mother.”

My blood is curdling and yet I cannot look away! Though this strategy is uniquely nightmarish, it is not uncommon for invading parasitic ants to execute queens in any number of creative ways. The parasites are just usually a bit more hands-on (or rather, tarsus-on) about the process.

“Queen-killing” has “evolved independently on multiple occasions across [ant species], indicating repeated evolutionary gains,” Shimada’s team said. “Until now, the only mechanistically documented solution was direct assault: the parasite throttles or beheads the host queen, a tactic that has arisen convergently in several lineages.”

When will we get an ant Shakespeare?! Someone needs to step up and claim that title, because these queens blow Lady MacBeth out of the water.

In other news…

That’s one small stem for a plant, one giant leaf for plant-kind

Maeng, Chang-hyun et al. “Extreme environmental tolerance and space survivability of the moss, Physcomitrium patens.” iScience,

Scientists simply love to expose extremophile life to the vacuum of space to, you know, see how well they do out there. In a new addition to this tradition, a study reports that spores from the moss Physcomitrium patens survived a full 283 days chilling on the outside of the International Space Station, which is generally not the side of an orbital habitat you want to be stuck on.

A reddish-brown spore similar to those used in the space exposure experiment. Image: Tomomichi Fujita

Even wilder, most of the spacefaring spores were reproductively successful upon their return to Earth. “Remarkably, even after 9 months of exposure to space conditions, over 80% of the encased spores germinated upon return to Earth,” said researchers led by Chang-hyun Maeng of Hokkaido University. “To the best of our knowledge, this is the first report demonstrating the survival of bryophytes”—the family to which mosses belong—”following exposure to space and subsequent return to the ground.”

Congratulations to these mosses for boldly growing where no moss has grown before.

Hints of a real-life ghost world

Hopp, Timo et al. “The Moon-forming impactor Theia originated from the inner Solar System.” Science.

Earth had barely been born before a Mars-sized planet, known as Theia, smashed into it some 4.5 billion years ago. The debris from the collision coalesced into what is now our Moon, which has played a key role in Earth’s habitability, so we owe our lives in part to this primordial punch-up.

undefined
KABLOWIE! Image: NASA/JPL-Caltech

Scientists have now revealed new details about Theia by measuring the chemical makeup of “lunar samples, terrestrial rocks, and meteorites…from which Theia and proto-­Earth might have formed,” according to a new study. They conclude that Theia likely originated in the inner solar system based on the chemical signatures that this shattered world left behind on the Moon and Earth.

“We found that all of Theia and most of Earth’s other constituent materials originated from the inner Solar System,” said researchers led by Timo Hopp of The University of Chicago and the Max Planck Institute for Solar System Research. “Our calculations suggest that Theia might have formed closer to the Sun than Earth did.”

Wherever its actual birthplace, what remains of Theia is buried on the Moon and as giant undigested slabs inside Earth’s mantle. Rest in pieces, sister.

Goosebumps of yore

Davin, Laurent et al. “A 12,000-year-old clay figurine of a woman and a goose marks symbolic innovations in Southwest Asia. Proceedings of the National Academy of Sciences.

You’ve heard of the albatross around your neck, but what about the goose on your back? A new study reports the discovery of a 12,000-year-old artifact in Israel that is the “earliest known figurine to depict a human–animal interaction” with its vision of a goose mysteriously draped over a woman’s spine and shoulders.

The tiny, inch-high figurine was recovered from a settlement built by the prehistoric Natufian culture and it may represent some kind of sex thing.

An image of the artifact, and an artistic reconstruction. Image: Davin, Laurent et al.

“We…suggest that by modeling a goose in this specific posture, the Natufian manufacturer intended to portray the trademark pattern of the gander’s mating behavior,” said researchers led by Laurent Davin of the Hebrew University of Jerusalem. “This kind of imagined mating between humans and animal spirits is typical of an animistic perspective, documented in cross-cultural archaeological and ethnographic records in specific situations” such as an “erotic dream” or “shamanistic vision.”

First, the bizarre Greek myth of Leda and the Swan, and now this? What is it about ancient cultures and weird waterfowl fantasies? In any case, my own interpretation is that the goose was just tired and needed a piggyback (or gaggle-back).

Thanks for reading! See you next week.

The Dangers of Doing Away With Monsters-Reflections of a Soldier Fighting Hitler

Portside
portside.org
2025-11-22 13:59:19
The Dangers of Doing Away With Monsters-Reflections of a Soldier Fighting Hitler Kurt Stand Sat, 11/22/2025 - 08:59 ...
Original Article
The Dangers of Doing Away With Monsters-Reflections of a Soldier Fighting Hitler Published

Gene Bruskin’s father before entering military service in early 1940s

My father, John Bruskin, served in WWII, and received two Purple Hearts when he was blown out of the tanks he was driving. While in the military he used a service they provided to record a series of “45s”, 7-inch vinal records, and send them to my mom, Gertrude. This “poem” is a compilation of his messages to my mom from France and other military locations. His hopes and anguish are palpable.

He had been a young Jewish activist in the Communist Party in Philadelphia in the late 30’s but left the Party, disillusioned by its top-down structure.  Working in a radio repair shop when WWII broke out, he entered the service to fight fascism and ironically endured antisemitism while being trained in Georgia.

Although the notion of PTSD didn’t exist then, my father was clearly impacted for the rest of his life. He died young at 61 having been a TV repairman his entire life. The war never left his consciousness. I remember while growing up our summer “vacations” were yearly trips to gatherings of The Fourth Armored Division, held in a variety of cities, giving my father a couple days with the guys to reminisce and leaving me and my sisters bored and playing on the elevators. He read every book he could find on the war and watched every news reel documentary shown on TV.

Like many from the “greatest generation” the defeat of Nazism and the ascendancy of the US as the world’s superpower didn’t do him much good. He essentially worked himself to death, died without health insurance or even a bank account. The hypocritical notion of “Thank you for your service” continues today and vets struggle to earn a living and recover from service in endless US initiated wars. The best we could do as “patriots” is to end US militarism and thank the teachers and nurses and many others for their service in making our country function.


(RECORDED WORDS OF JOHN BRUSKIN DURING WW II)

Gert,

Monsters are roaming the earth
Dinosaurs, leaving anguish, pain, death and sorrow in their wake
That’s why I’m somewhere far away
Fighting to get rid of these creatures
So our children will never see such things
It has to be done now, right?

It’s not an easy life
We all try to keep our chin up
Millions of us
No complaining
No getting sad, you know

Don’t worry
I’m going to survive
Get that one-way ticket home
Some day
Pretty sure

We can start over where we left off
Like that wonderful day when we were so excited
When we permanently joined as man and wife
The nicest day of my life
And yours too, huh?

Sure, we’re poor
But we’ll manage
A little place of our own, our dream spot
Our beautiful little baby gal Francie
Maybe another one later

And I have you.
That’s what makes life worth living.

So that’s it
We don’t want much
The simple things in life
We don’t ask for more

But it’s out of the question today
And that makes me very sad
Especially after seeing the baby for the first time
She’s so cute
She’ll be a fine girl
Like her mother is
Like her mother always was

So be a good soldier Gert
Keep your chin up
You are strong
You got common sense
You understand what’s taking place

I feel from the bottom of my soul
This thing will be over
We will be together and never be separated.
Perhaps it’s inevitable for many of us to go.
We all hope and pray it don’t happen.
But if it happens
Well, you understand.
Don’t you Gert?

Gene Bruskin is a retired 45 year veteran of labor and international solidarity work. He has served as a USWA Local president, spent two years as Jesse Jackson’s labor director in the 1990s and 10 years as Secretary Treasurer of the Food and Allied Service Trades of the AFL-CIO. He was the UFCW Justice@Smithfield Campaign Director organizing 5,000 meatpacking workers in North Carolina in 2008. During the Iraq war he was a founder of US Labor Against the War and has done extensive international labor solidarity work for struggles in Central America, South Africa, Cuba , the Philippines, Iraq and Israel/Palestine . He recently helped establish the National Labor Network for Ceasefire in Gaza. He is currently mentoring Amazon workers in various locations Since retirement from AFT in 2012 he has written and produced 3 musicals, and his current production is called The Return of John Brown. ( www.thereturnofjohnbrown.com )

Piecing Together the Puzzle: A Qilin Ransomware Investigation

Bleeping Computer
www.bleepingcomputer.com
2025-11-22 13:45:53
Huntress analysts reconstructed a Qilin ransomware attack from a single endpoint, using limited logs to reveal rogue ScreenConnect access, failed infostealer attempts, and the ransomware execution path. The investigation shows how validating multiple data sources can uncover activity even when visib...
Original Article

Huntress header

Written by Lindsey O’Donnell-Welch, Ben Folland, Harlan Carvey of Huntress Labs.

A big part of a security analyst’s everyday role is figuring out what actually happened during an incident. We can do that by piecing together breadcrumbs–whether that’s through logs, antivirus detections, and other clues–that help us understand how the attacker achieved initial access and what they did after.

However, it’s not always cut and dry: sometimes there are external factors that limit our visibility. The Huntress agent might not be deployed across all endpoints, for example, or the targeted organization might install the Huntress agent after a compromise has already occurred.

In these cases, we need to get creative and look at multiple data sources in order to determine what actually happened.

Recently, we analyzed an incident where both of the above factors were true: on October 11, an organization installed the Huntress agent post-incident, and initially on one endpoint.

When it comes to visibility, this incident was less about looking through a keyhole, and more akin to looking through a pinhole. Even so, Huntress analysts were able to derive a great deal of information regarding the incident.

The Qilin incident: What we started with

The Huntress agent was installed on a single endpoint following a Qilin ransomware infection. What does that mean from the perspective of an analyst trying to figure out what happened?

We had limited clues to start: there was no endpoint detection and response (EDR) or SIEM telemetry available, and Huntress-specific ransomware canaries weren’t tripped. Because we were also on one endpoint, our visibility was limited to the activity that had occurred on that specific endpoint within the broader environment’s infrastructure.

As a result, all Huntress analysts had to start with to unravel this incident was the managed antivirus (MAV) alerts. Once the Huntress agent was added to the endpoint, the SOC was alerted to existing MAV detections, some of which are illustrated in Figure 1.

Figure 1: MAV alerts that tripped after the ransom note was dropped
Figure 1: MAV alerts that tripped after the ransom note was dropped

Simplify Your Path to CMMC Compliance

Preparing for CMMC Level 2 certification doesn’t have to be overwhelming.

Huntress provides the tools, documentation, and expert guidance you need to streamline your audit process and protect your contracts. Let us help you achieve compliance faster and more affordably.

Learn More

Analysts began tasking files from the endpoint, starting with a specific subset of the Windows Event Logs (WEL).

From those logs, analysts could see that on 8 Oct 2025, the threat actor accessed the endpoint and installed the Total Software Deployment Service , as well as a rogue instance of the ScreenConnect RMM , one that pointed to IP address 94.156.232[.]40 .

Searching VirusTotal for the IP address provided the insight illustrated in Figure 2.

Figure 2: VirusTotal response for the IP address 94.156.232[.]40
Figure 2: VirusTotal response for the IP address 94.156.232[.]40

An interesting aspect of the installation was that LogMeIn was apparently legitimately installed on the endpoint on 20 Aug 2025 from the file %user%\Downloads\LogMeIn.msi .

Then, on 8 Oct, the rogue ScreenConnect instance was installed from the file C:\Users\administrator\AppData\Roaming\Installer\LogmeinClient.msi .

Further, the timeline indicates that on 2 Oct, the file %user%\Downloads\LogMeIn Client.exe was submitted by Windows Defender for review, and no other action was taken after that event.

Pivoting from the ScreenConnect installation to ScreenConnect activity events within the timeline of activity, analysts saw that on 11 Oct, three files were transferred to the endpoint via the ScreenConnect instance; r.ps1 , s.exe , and ss.exe .

Digging in a bit deeper, only r.ps1 was still found on the endpoint (shown below).

$RDPAuths = Get-WinEvent -LogName
'Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational'
-FilterXPath @'
<QueryList><Query Id="0"><Select>
  *[System[EventID=1149]]
</Select></Query></QueryList>
'@
# Get specific properties from the event XML
[xml[]]$xml=$RDPAuths|Foreach{$_.ToXml()}
$EventData = Foreach ($event in $xml.Event) {
  # Create custom object for event data
  New-Object PSObject -Property @{
   TimeCreated = (Get-Date ($event.System.TimeCreated.SystemTime) -Format 'yyyy-MM-dd hh:mm:ss K')
   User = $event.UserData.EventXML.Param1
   Domain = $event.UserData.EventXML.Param2
   Client = $event.UserData.EventXML.Param3
  }
}
$EventData | FT

Based on the contents of the script, it would appear that the threat actor was interested in determining IP addresses, domains, and usernames associated with RDP accesses to the endpoint.

However, the Windows Event Log contained a Microsoft-Windows-PowerShell/4100 message stating:

Error Message = File C:\WINDOWS\systemtemp\ScreenConnect\22.10.10924.8404\Files\r.ps1 cannot be loaded because running scripts is disabled on this system.

This message was logged within 20 seconds of the script being transferred to the endpoint, and the threat actor attempting to run it.

Parsing through PCA logs

The other two files, s.exe and ss.exe , took a bit more work to unravel, because they were no longer found on the endpoint.

However, Huntress analysts were able to take advantage of data sources on the Windows 11 endpoint, specifically the AmCache.hve file and the Program Compatibility Assistant (PCA) log files to obtain hashes for the files, and to see that while the threat actor had attempted to execute the files, both apparently failed.

The threat actor disabled Windows Defender, which were seen in Windows Defender event records , starting with event ID 5001, indicating that the Real-Time Protection feature was disabled. This was followed by several event ID 5007 records, indicating that features such as SpyNetReporting and SubmitSamplesConsent had been modified (in this case, disabled), as well as SecurityCenter messages indicating that Windows Defender had entered a SECURITY_PRODUCT_STATE_SNOOZED state.

The threat actor then attempted to launch s.exe , which was almost immediately followed by the message “Installer failed” in the PCA logs. Based on the identified VirusTotal detections shown in Figure 3, and the behaviors identified by VirusTotal , this file appears to be an infostealer .

Figure 3: VirusTotal response for s.exe file
Figure 3: VirusTotal response for s.exe file

The messages in the PCA logs provide indications that the file, identified as an installer, failed to execute.

Seven seconds later, the threat actor attempted to run ss.exe , which was immediately followed by the legitimate Windows application, c:\windows\syswow64\werfault.exe , being launched. The PCA logs then contained three consecutive messages stating, “PCA resolve is called, resolver name: CrashOnLaunch, result: 0” with respect to ss.exe , all indicating that the application did not run.

Again, prior to attempting to run the above two files, the threat actor disabled Windows Defender at 2025-10-11 01:34:21 UTC , resulting in the Windows Defender status being reported as SECURITY_PRODUCT_STATE_SNOOZED . At 2025-10-11 03:34:56 UTC , the threat actor accessed the endpoint remotely, and then at 2025-10-11 03:35:13 UTC , there were multiple Windows Defender detections for attempts to create ransom notes (i.e., Behavior:Win32/GenRansomNote ), as well as Windows Defender messages indicating that remediation attempts failed.

At this point, the Windows Defender status was reported as SECURITY_PRODUCT_STATE_ON . The Windows Defender detection, coupled with the preceding remote login, appears to indicate that the ransomware executable was launched from another endpoint, against network shares .

Figure 4 illustrates an excerpt of a Qilin ransom note found on the endpoint.

Figure 4: Qilin ransom note excerpt
Figure 4: Qilin ransom note excerpt

Qilin ransomware is a “ransomware-as-a-service” (RaaS) variant, meaning that while the ransomware logistics is managed from a central location, each affiliate likely follows a different attack pattern, leaving behind different traces and artifacts.

For example, a number of Qilin incidents observed by Huntress analysts have started with the threat actor logging in via the Remote Desktop Protocol (RDP), and all included similar ransom notes and encrypted file extensions.

However, in only one incident did analysts observe the use of s5cmd for data exfiltration .

The value of multiple data sources in an investigation

Throughout this investigation, Huntress analysts weren’t looking through a keyhole. Remember, the Huntress agent was installed post-incident, so there was no EDR telemetry, no SIEM data, and no ransomware canaries on which to build an understanding of the incident progression.

In addition, at the time the MAV alerts were received in the Huntress portal, this was the only endpoint within the infrastructure with a Huntress agent installed.

Rather than looking through a keyhole, analysts were looking through a pinhole. Yet, relying on multiple data sources led not only to a deeper understanding of the threat actor’s attempted activities on the endpoint, but also served to validate findings and provide a clearer picture of what actually occurred.

For example, understanding that the threat actor used a rogue ScreenConnect instance to attempt to deploy several malicious files–including one that appeared to be an infostealer–can help inform the victim company when they are trying to determine the scope of the incident and how to respond.

During an investigation, particularly one that is time-sensitive or even just assumed to be, it’s easy to fall prey to finding an artifact and building a story around it without first verifying or validating it. It can be easy to think, “...this is anomalous to me…”, without really considering if it’s anomalous within the infrastructure itself, particularly if the investigation is being performed by looking through a pinhole.

Validating activity across multiple data sources, and not jumping to the first indicator as the basis for malicious activity, provides a much more accurate picture of the threat actor’s activities, and provides a foundation for more accurate decisions and remediations.

Meet Huntress: Demo & AMA

Cyber threats don’t take breaks, and neither do we. At Huntress, we’re always innovating because the job never stops when it comes to leveling up security and protecting businesses like yours.

Bring your toughest questions, real-world scenarios, and security headaches. Let's tackle them together.

Reserve for the webinar!

IOCs

Indicator

Description

63bbb3bfea4e2eea

Rogue ScreenConnect instance ID

af9925161d84ef49e8fbbb08c3d276b49d391fd997d272fe1bf81f8c0b200ba1

s.exe  hash

ba79cdbcbd832a0b1c16928c9e8211781bf536cc

ss.exe hash

README-RECOVER-<extension>.txt

Ransom note

Sponsored and written by Huntress Labs .

Move Expressions

Lobsters
smallcultfollowing.com
2025-11-22 11:41:31
Comments...
Original Article

This post explores another proposal in the space of ergonomic ref-counting that I am calling move expressions . To my mind, these are an alternative to explicit capture clauses , one that addresses many (but not all ) of the goals from that design with improved ergonomics and readability.

TL;DR

The idea itself is simple, within a closure (or future), we add the option to write move($expr) . This is a value expression (“rvalue”) that desugars into a temporary value that is moved into the closure. So

|| something(&move($expr))

is roughly equivalent to something like:

{ 
    let tmp = $expr;
    || something(&{tmp})
}

How it would look in practice

Let’s go back to one of our running examples, the “Cloudflare example”, which originated in this excellent blog post by the Dioxus folks . As a reminder, this is how the code looks today – note the let _some_value = ... lines for dealing with captures:

// task:  listen for dns connections
let _some_a = self.some_a.clone();
let _some_b = self.some_b.clone();
let _some_c = self.some_c.clone();
tokio::task::spawn(async move {
  	do_something_else_with(_some_a, _some_b, _some_c)
});

Under this proposal it would look something like this:

tokio::task::spawn(async {
    do_something_else_with(
        move(self.some_a.clone()),
        move(self.some_b.clone()),
        move(self.some_c.clone()),
    )
});

There are times when you would want multiple clones. For example, if you want to move something into a FnMut closure that will then give away a copy on each call, it might look like

data_source_iter
    .inspect(|item| {
        inspect_item(item, move(tx.clone()).clone())
        //                      ----------  -------
        //                           |         |
        //                   move a clone      |
        //                   into the closure  |
        //                                     |
        //                             clone the clone
        //                             on each iteration
    })
    .collect();

// some code that uses `tx` later...

Credit for this idea

This idea is not mine. It’s been floated a number of times. The first time I remember hearing it was at the RustConf Unconf, but I feel like it’s come up before that. Most recently it was proposed by Zachary Harrold on Zulip , who has also created a prototype called soupa . Zachary’s proposal, like earlier proposals I’ve heard, used the super keyword. Later on @simulacrum proposed using move , which to me is a major improvement, and that’s the version I ran with here.

This proposal makes closures more “continuous”

The reason that I love the move variant of this proposal is that it makes closures more “continuous” and exposes their underlying model a bit more clearly. With this design, I would start by explaining closures with move expressions and just teach move closures at the end, as a convenient default:

A Rust closure captures the places you use in the “minimal way that it can” – so || vec.len() will capture a shared reference to the vec , || vec.push(22) will capture a mutable reference, and || drop(vec) will take ownership of the vector.

You can use move expressions to control exactly what is captured: so || move(vec).push(22) will move the vector into the closure. A common pattern when you want to be fully explicit is to list all captures at the top of the closure, like so:

|| {
    let vec = move(input.vec); // take full ownership of vec
    let data = move(&cx.data); // take a reference to data
    let output_tx = move(output_tx); // take ownership of the output channel

    process(&vec, &mut output_tx, data)
}

As a shorthand, you can write move || at the top of the closure, which will change the default so that closures > take ownership of every captured variable. You can still mix-and-match with move expressions to get more control. > So the previous closure might be written more concisely like so:

move || {
    process(&input.vec, &mut output_tx, move(&cx.data))
    //       ---------       ---------       --------      
    //           |               |               |         
    //           |               |       closure still  
    //           |               |       captures a ref
    //           |               |       `&cx.data`        
    //           |               |                         
    //       because of the `move` keyword on the clsoure,
    //       these two are captured "by move"
    //       
}

This proposal makes move “fit in” for me

It’s a bit ironic that I like this, because it’s doubling down on part of Rust’s design that I was recently complaining about. In my earlier post on Explicit Capture Clauses I wrote that:

To be honest, I don’t like the choice of move because it’s so operational . I think if I could go back, I would try to refashion our closures around two concepts

  • Attached closures (what we now call || ) would always be tied to the enclosing stack frame. They’d always have a lifetime even if they don’t capture anything.
  • Detached closures (what we now call move || ) would capture by-value, like move today.

I think this would help to build up the intuition of “use detach || if you are going to return the closure from the current stack frame and use || otherwise”.

move expressions are, I think, moving in the opposite direction. Rather than talking about attached and detached, they bring us to a more unified notion of closures, one where you don’t have “ref closures” and “move closures” – you just have closures that sometimes capture moves, and a “move” closure is just a shorthand for using move expressions everywhere. This is in fact how closures work in the compiler under the hood, and I think it’s quite elegant.

Why not suffix?

One question is whether a move expression should be a prefix or a postfix operator. So e.g.

|| something(&$expr.move)

instead of &move($expr) .

My feeling is that it’s not a good fit for a postfix operator because it doesn’t just take the final value of the expression and so something with it, it actually impacts when the entire expression is evaluated. Consider this example:

|| process(foo(bar()).move)

When does bar() get called? If you think about it, it has to be closure creation time, but it’s not very “obvious”.

We reached a similar conclusion when we were considering .unsafe operators. I think there is a rule of thumb that things which delineate a “scope” of code ought to be prefix – though I suspect unsafe(expr) might actually be nice, and not just unsafe { expr } .

Edit: I added this section after-the-fact in response to questions.

Conclusion

I’m going to wrap up this post here. To be honest, what this design really has going for it, above anything else, is its simplicity and the way it generalizes Rust’s existing design . I love that. To me, it joins the set of “yep, we should clearly do that” pieces in this puzzle:

  • Add a Share trait (I’ve gone back to preferring the name share 😁)
  • Add move expressions

These both seem like solid steps forward. I am not yet persuaded that they get us all the way to the goal that I articulated in an earlier post :

“low-level enough for a Kernel, usable enough for a GUI”

but they are moving in the right direction.

ADHD and Monotropism (2023)

Hacker News
monotropism.org
2025-11-22 11:31:30
Comments...
Original Article

Fergus Murray with Sonny Hallett (2023)

Monotropism was formulated as a theory of autism. It seeks to explain the experiences and traits of autistic people in terms of a tendency for resources like attention to be concentrated on a small number of things at a time, with little left over for everything else. Through this lens we can make sense of autistic social, sensory and executive functioning differences, as laid out in Monotropism – Explanations .

As time has gone on, it has become clear that many diagnosed with Attention Deficit Hyperactivity Disorder (ADHD) also identify strongly with many aspects of monotropism. I want to explore this by looking at the diverse ways that autism and ADHD present; where the traits associated with ADHD fit in with monotropism in an obvious way, and where they might seem to be in tension; and what this might mean for how we think about diagnoses and neurodiversity. Much of what I have to say here is necessarily speculative, all of it calls for further research, and parts of it may be in tension with some of the ways that many people are used to talking about neurodivergence.

The way that ADHD and autism are characterised in diagnostic manuals is completely different. ADHD is treated as primarily an attentional difference; autism as chiefly social in nature. Where descriptions do overlap, they can seem contradictory: autism is apparently characterised by rigid, restricted interests, while ADHD is said to cause impulsive behaviour and an inability to concentrate.

So the facts that anywhere from 30% to 80% of autistic people seemingly fit the diagnostic criteria for ADHD, and the two clearly run in the same families, might initially seem surprising. It cries out for an explanation. One possibility is that autism and ADHD – or a Kinetic Cognitive Style (KCS), as I prefer to call it – share an underlying cause. Monotropism has been put forward as one candidate for this, for example in Patrick Dwyer’s Revisiting Monotropism .

It is well established that autism can manifest very differently in different people, in ways that can seem contradictory. We know that autism can come with hyperlexia, or serious language difficulties. We know that it’s associated with sensory seeking and sensory avoidance. We understand that it might come with with crystal-clear memories, or forgetfulness. All of these things can coexist in one person, or just a selection.

With this in mind, it is perhaps not such a stretch to suggest that impulsivity, inattention and hyperactivity might share cognitive or neurological roots with their apparent opposites, like inflexibility, hyperfocus and inertia . When and how such traits manifest might depend on a person’s interests and experiences, or it might have to do with innate neurocognitive differences. Understanding this kind of variation fully would take far more research on the life experiences and psychological development of people with a variety of cognitive styles, without assuming that current diagnostic categories reflect objectively real categories of human being .

Impulsivity could come from the monotropic tendency to lose awareness of things as soon as our attention shifts away from them. Inattention is a very familiar thing among autistic people – not an attention deficit , which was never the right term, but profound difficulty steering attention in directions which don’t align with our current interests. Hyperfocusing is common with KCS, as it is with autism.

Hyperactivity can refer to a need to keep moving, which bears a striking resemblance to the autistic need to stim. It can also refer to a cognitive tendency which is a little harder to reconcile with how monotropism has been characterised: a habit of hopping mentally from one thing to another. In contrast, difficulty shifting from one attention tunnel to another has been a central feature of the ways monotropism has been described. This tension is worth digging into.

It might be that a Kinetic Cognitive Style arises out of a combination of a relatively monotropic processing style combined with other factors – difficulty accessing flow states , for example, as suggested by some recent research ( Grotewiel et al 2022 ). There are all kinds of reasons why people might not be able to enter ‘ flowy focus tunnels ‘, as Jamie Knight calls them. They might have too many distractions, or too much nervous energy; they might not feel safe enough to lose themselves in the flow; they might have had bad experiences being told off for doing so, or been wrenched out of them too many times. They might just be too depleted to be able to connect deeply with their passions, something which also occurs during autistic burnout .

We know that novelty-seeking is a trait that varies greatly between people. It’s also possible that some people just have naturally very mobile attention, which might compensate for the monotropic tendency for attention to get sucked into one thing at a time. And maybe some of that apparent attention-hopping happens within an attention tunnel anyway, and other people just aren’t seeing the connections! KCS might look like polytropism sometimes, but I think that can be misleading. I delayed getting my own autism assessment for years because I mistook my serial monotropism for polytropism: I told myself I was multi-tasking, when it would probably be more accurate to say I repeatedly forgot what I was supposed to be doing.

Meanwhile, it is likely that monotropism doesn’t necessarily give rise to autism in the sense required by diagnostic manuals – but that above a certain level of intensity, or in combination with other factors, it causes the familiar social differences, fixity and so on. An early intense interest in other people, and how they behave, might equip someone with tools that will allow them to avoid being seen as too socially weird. The ability to present a ‘normal-looking’ face to the world is likely a major factor in the under-identification of autistic girls, who face far more social pressure to blend in than boys do. None of this changes a person’s cognitive style; but then, autism, like ADHD, has always been assessed based on outward presentation. One hope for Monotropism as a theory is that it helps us to make sense of these things from an internal perspective, rather than looking only at the surface level.

It is, I think, too early to say with any confidence that autism and ADHD (or KCS) share a common root in monotropism, but the overlapping traits of the people receiving each label clearly demand some kind of explanation, and preliminary results do suggest that each is strongly correlated with monotropism – especially in combination. With any luck, we will see a good deal more research on this in coming years.

A possibly-illustrative photo: Two cats have been fighting. Fur lies scattered way across the path. The ginger cat is lying down, looking at the black cat, who looks the other way, still bristling.

Agent Design Is Still Hard

Hacker News
lucumr.pocoo.org
2025-11-22 11:27:24
Comments...
Original Article

written on November 21, 2025

I felt like it might be a good time to write about some new things I’ve learned. Most of this is going to be about building agents, with a little bit about using agentic coding tools.

TL;DR: Building agents is still messy. SDK abstractions break once you hit real tool use. Caching works better when you manage it yourself, but differs between models. Reinforcement ends up doing more heavy lifting than expected, and failures need strict isolation to avoid derailing the loop. Shared state via a file-system-like layer is an important building block. Output tooling is surprisingly tricky, and model choice still depends on the task.

Which Agent SDK To Target?

When you build your own agent, you have the choice of targeting an underlying SDK like the OpenAI SDK or the Anthropic SDK, or you can go with a higher level abstraction such as the Vercel AI SDK or Pydantic. The choice we made a while back was to adopt the Vercel AI SDK but only the provider abstractions, and to basically drive the agent loop ourselves . At this point we would not make that choice again. There is absolutely nothing wrong with the Vercel AI SDK, but when you are trying to build an agent, two things happen that we originally didn’t anticipate:

The first is that the differences between models are significant enough that you will need to build your own agent abstraction. We have not found any of the solutions from these SDKs that build the right abstraction for an agent. I think this is partly because, despite the basic agent design being just a loop, there are subtle differences based on the tools you provide. These differences affect how easy or hard it is to find the right abstraction (cache control, different requirements for reinforcement, tool prompts, provider-side tools, etc.). Because the right abstraction is not yet clear, using the original SDKs from the dedicated platforms keeps you fully in control. With some of these higher-level SDKs you have to build on top of their existing abstractions, which might not be the ones you actually want in the end.

We also found it incredibly challenging to work with the Vercel SDK when it comes to dealing with provider-side tools. The attempted unification of messaging formats doesn’t quite work. For instance, the web search tool from Anthropic routinely destroys the message history with the Vercel SDK, and we haven’t yet fully figured out the cause. Also, in Anthropic’s case, cache management is much easier when targeting their SDK directly instead of the Vercel one. The error messages when you get things wrong are much clearer.

This might change, but right now we would probably not use an abstraction when building an agent, at least until things have settled down a bit. The benefits do not yet outweigh the costs for us.

Someone else might have figured it out. If you’re reading this and think I’m wrong, please drop me a mail. I want to learn.

Caching Lessons

The different platforms have very different approaches to caching. A lot has been said about this already, but Anthropic makes you pay for caching. It makes you manage cache points explicitly, and this really changes the way you interact with it from an agent engineering level. I initially found the manual management pretty dumb. Why doesn’t the platform do this for me? But I’ve fully come around and now vastly prefer explicit cache management. It makes costs and cache utilization much more predictable.

Explicit caching allows you to do certain things that are much harder otherwise. For instance, you can split off a conversation and have it run in two different directions simultaneously. You also have the opportunity to do context editing. The optimal strategy here is unclear, but you clearly have a lot more control, and I really like having that control. It also makes it much easier to understand the cost of the underlying agent. You can assume much more about how well your cache will be utilized, whereas with other platforms we found it to be hit and miss.

The way we do caching in the agent with Anthropic is pretty straightforward. One cache point is after the system prompt. Two cache points are placed at the beginning of the conversation, where the last one moves up with the tail of the conversation. And then there is some optimization along the way that you can do.

Because the system prompt and the tool selection now have to be mostly static, we feed a dynamic message later to provide information such as the current time. Otherwise, this would trash the cache. We also leverage reinforcement during the loop much more.

Reinforcement In The Agent Loop

Every time the agent runs a tool you have the opportunity to not just return data that the tool produces, but also to feed more information back into the loop. For instance, you can remind the agent about the overall objective and the status of individual tasks. You can also provide hints about how the tool call might succeed when a tool fails. Another use of reinforcement is to inform the system about state changes that happened in the background. If you have an agent that uses parallel processing, you can inject information after every tool call when that state changed and when it is relevant for completing the task.

Sometimes it’s enough for the agent to self-reinforce. In Claude Code, for instance, the todo write tool is a self-reinforcement tool. All it does is take from the agent a list of tasks that it thinks it should do and echo out what came in. It’s basically just an echo tool; it really doesn’t do anything else. But that is enough to drive the agent forward better than if the only task and subtask were given at the beginning of the context and too much has happened in the meantime.

We also use reinforcements to inform the system if the environment changed during execution in a way that’s problematic for the agent. For instance, if our agent fails and retries from a certain step forward but the recovery operates off broken data, we inject a message informing it that it might want to back off a couple of steps and redo an earlier step.

Isolate Failures

If you expect a lot of failures during code execution, there is an opportunity to hide those failures from the context. This can happen in two ways. One is to run tasks that might require iteration individually. You would run them in a subagent until they succeed and only report back the success, plus maybe a brief summary of approaches that did not work. It is helpful for an agent to learn about what did not work in a subtask because it can then feed that information into the next task to hopefully steer away from those failures.

The second option doesn’t exist in all agents or foundation models, but with Anthropic you can do context editing. So far we haven’t had a lot of success with context editing, but we believe it’s an interesting thing we would love to explore more. We would also love to learn if people have success with it. What is interesting about context editing is that you should be able to preserve tokens for further down the iteration loop. You can take out of the context certain failures that didn’t drive towards successful completion of the loop, but only negatively affected certain attempts during execution. But as with the point I made earlier: it is also useful for the agent to understand what didn’t work, but maybe it doesn’t require the full state and full output of all the failures.

Unfortunately, context editing will automatically invalidate caches. There is really no way around it. So it can be unclear when the trade-off of doing that compensates for the extra cost of trashing the cache.

Sub Agents / Sub Inference

As I mentioned a couple of times on this blog already, most of our agents are based on code execution and code generation. That really requires a common place for the agent to store data. Our choice is a file system—in our case a virtual file system—but that requires different tools to access it. This is particularly important if you have something like a subagent or subinference.

You should try to build an agent that doesn’t have dead ends. A dead end is where a task can only continue executing within the sub-tool that you built. For instance, you might build a tool that generates an image, but is only able to feed that image back into one more tool. That’s a problem because you might then want to put those images into a zip archive using the code execution tool. So there needs to be a system that allows the image generation tool to write the image to the same place where the code execution tool can read it. In essence, that’s a file system.

Obviously it has to go the other way around too. You might want to use the code execution tool to unpack a zip archive and then go back to inference to describe all the images so that the next step can go back to code execution and so forth. The file system is the mechanism that we use for that. But it does require tools to be built in a way that they can take file paths to the virtual file system to work with.

So basically an ExecuteCode tool would have access to the same file system as the RunInference tool which could take a path to a file on that same virtual file system.

The Use Of An Output Tool

One interesting thing about how we structured our agent is that it does not represent a chat session. It will eventually communicate something to the user or the outside world, but all the messages that it sends in between are usually not revealed. The question is: how does it create that message? We have one tool which is the output tool. The agent uses it explicitly to communicate to the human. We then use a prompt to instruct it when to use that tool. In our case the output tool sends an email.

But that turns out to pose a few other challenges. One is that it’s surprisingly hard to steer the wording and tone of that output tool compared to just using the main agent loop’s text output as the mechanism to talk to the user. I cannot say why this is, but I think it’s probably related to how these models are trained.

One attempt that didn’t work well was to have the output tool run another quick LLM like Gemini 2.5 Flash to adjust the tone to our preference. But this increases latency and actually reduces the quality of the output. In part, I think the model just doesn’t word things correctly and the subtool doesn’t have sufficient context. Providing more slices of the main agentic context into the subtool makes it expensive and also didn’t fully solve the problem. It also sometimes reveals information in the final output that we didn’t want to be there, like the steps that led to the end result.

Another problem with an output tool is that sometimes it just doesn’t call the tool. One of the ways in which we’re forcing this is we remember if the output tool was called. If the loop ends without the output tool, we inject a reinforcement message to encourage it to use the output tool.

Model Choice

Overall our choices for models haven’t dramatically changed so far. I think Haiku and Sonnet are still the best tool callers available, so they make for excellent choices in the agent loop. They are also somewhat transparent with regards to what the RL looks like. The other obvious choices are the Gemini models. We so far haven’t found a ton of success with the GPT family of models for the main loop.

For the individual sub-tools, which in part might also require inference, our current choice is Gemini 2.5 if you need to summarize large documents or work with PDFs and things like that. That is also a pretty good model for extracting information from images, in particular because the Sonnet family of models likes to run into a safety filter which can be annoying.

There’s also probably the very obvious realization that token cost alone doesn’t really define how expensive an agent. A better tool caller will do the job in fewer tokens. There are some cheaper models available than sonnet today, but they are not necessarily cheaper in a loop.

But all things considered, not that much has changed in the last couple of weeks.

Testing and Evals

We find testing and evals to be the hardest problem here. This is not entirely surprising, but the agentic nature makes it even harder. Unlike prompts, you cannot just do the evals in some external system because there’s too much you need to feed into it. This means you want to do evals based on observability data or instrumenting your actual test runs. So far none of the solutions we have tried have convinced us that they found the right approach here. Unfortunately, I have to report that at the moment we haven’t found something that really makes us happy. I hope we’re going to find a solution for this because it is becoming an increasingly frustrating aspect of building an agent.

Coding Agent Updates

As for my experience with coding agents, not really all that much has changed. The main new development is that I’m trialing Amp more. In case you’re curious why: it’s not that it’s objectively a better agent than what I’m using, but I really quite like the way they’re thinking about agents from what they’re posting. The interactions of the different sub agents like the Oracle with the main loop is beautifully done, and not many other harnesses do this today. It’s also a good way for me to validate how different agent designs work. Amp, similar to Claude Code, really feels like a product built by people who also use their own tool. I do not feel every other agent in the industry does this.

Quick Stuff I Read And Found

That’s just a random assortment of things that I feel might also be worth sharing:

This entry was tagged ai

copy as / view markdown

Why DETRs are replacing YOLOs for real-time object detection

Hacker News
blog.datameister.ai
2025-11-22 11:21:27
Comments...
Original Article

​Real-time object detection lies at the heart of any system that must interpret visual data efficiently, from video analytics pipelines to autonomous robotics. Detector architectures for such tasks need to deliver both high throughput and accuracy in order to excel.

In our own pipelines, we phased out older CNN-based detectors in favor of D-Fine , a more recent model that is part of the DEtection Transformer (DETR) family. Transformer-based detectors have matured quickly, and D-Fine in particular provides stronger accuracy while maintaining competitive inference speed.

Our office dog Nala sitting on a chair, as detected our own D-Fine model in the DM vision library.

YOLO has long been the leading standard for real-time detection, but the latest DETR variants are now consistently proving to be the better alternative. Beyond the accuracy gains, an equally important advantage is the far more permissive license that comes with it.

YOLO’s licensing issue

The YOLO series is developed and maintained by Ultralytics . All YOLO code and weights are released under the AGPL-3.0 license . Long story short, this license only allows commercial usage under the strict condition that any code modifications or weights should be made publicly available. On the contrary, all DETR models to date were released under the Apache 2.0 License, allowing for free use and modifications for commercial and proprietary use.

Next to licensing, there are others reasons why we like working with DETRs:

  • DETRs treat object detection as a direct set-prediction problem. This eliminates hand-crafted components such as non-maximum suppression that introduce additional hyperparameters and slow down the detection pipeline.
  • Modern GPU architectures are heavily optimized for efficient attention operations such as flash attention , making transformers increasingly more suitable for real-time applications.
  • Transfer learning from vision foundation models such as the recent DINOv3 fundamentally augments the capabilities of DETRs.

We have had nothing but great experiences with DETRs so far. They adapt remarkably well to new datasets, even when trained from scratch. For the right use cases, pre-training the models on datasets such as COCO and Objects365 further boosts performance. About time for a post on this exciting topic!

​A short overview of what you can expect of the remainder of this blogpost. We will:

  • dive in detail into the original DETR paper to understand its core concepts ;
  • discuss the most important advancements leading to the real-time adoption of DETRs;
  • compare two leading DETR models to the latest YOLO 11 model to draw some important conclusions.

Let’s go!

DETR: transformer for NMS-free object detection

All Detection Transformer architectures have the same underlying structure. A (CNN-) backbone is used to extract image features. These features are fed to a transformer encoder-decoder structure that is able to predict accurate bounding boxes for object in the image. The resulting N decoder output embeddings are independently projected to bounding box coordinates and class labels.

High-level overview of the DETR architecture, adapted from the original paper (2020)

Why transformers?

Intuitively, the encoder in DETR transforms the dense backbone features into a semantically structured representation of the image that captures relationships between regions through global self-attention.

​The transformer decoder takes a fixed set of N learned object queries, each representing a potential object slot. It then iteratively refines these to produce final bounding boxes and class predictions . It does this through two attention operations:

  • Self-attention among the queries, enabling them to model interactions and avoid duplicate detections (e.g., two queries focusing on the same object).
  • Cross-attention between the queries and the encoder’s output features, allowing each query to attend to the most relevant parts of the image and extract the corresponding visual evidence.
Attention layer in the DETR decoder. The output embedding of the cross-attention module serves as the content query for the next layer. The output features of the encoder are the key and value for cross-attention. The positional query is learnable and shared over self-attention and cross-attention in all layers

Through the clever use of attention in the decoder, DETR replaces traditional components like anchor boxes and non-maximum suppression with a fully end-to-end transformer-based detection process.

Direct set prediction

DETR reframes object detection as a direct set-prediction problem. Given an image, it predicts a fixed set of N bounding boxes corresponding to the object queries. Because N typically exceeds the number of actual objects, many predictions correspond to a special “no-object” class and are discarded at inference. During training, the Hungarian algorithm performs bipartite matching between predicted and ground-truth boxes, ensuring each ground-truth box is paired with exactly one prediction in a permutation-invariant way. The loss is then computed on these matched pairs.

Overcoming DETRs shortcomings

Despite its elegance and powerful prediction paradigm, slow training converge and low performance on small objects limited adoption in practical systems early on. Over the years, several enhancements drastically improved the performance of Detection Transformers:

  • Deformable DETR introduced deformable attention, an efficient multi-scale attention mechanism tailored to the task of object detection.
  • The authors of Efficient DETR were the first to use top-k query selection for better initialization of the object queries for the decoder.
  • DN-DETR drastically improved training convergence using an auxiliary denoising task of training bounding boxes.
DETR evolution throughout time. Real-time variants arose from 2024 in two families: the RT-DETR family indicated in blue, and the LW-DETR indicated in purple.

Real-time transformer object detection

From 2024 onwards DETRs really started to challenge YOLO in real-time detection, eventually surpassing them in accuracy while remaining competitive in speed and efficiency. There are two schools of thought that compete for the state-of-the art nowadays:

  • RT-DETR ( real-time DETR ) sticks to the original DETR architecture and focuses on optimizing the encoder and the initialization of the object queries. D-Fine currently leads this family with a heavily optimized training strategy centered on the decoder. Very recently, DEIMv2 extends it further by integrating DINOv3 features in its backbone.
  • LW-DETR ( light-weight DETR ) adopts a simpler idea: replace the traditional CNN backbone and encoder with a pure Vision Transformer (ViT). RF-DETR ( Roboflow DETR) leverages this especially well by starting from a pretrained DINOv2 encoder.

Work on Detection Transformers is very much alive: DEIMv2 was released less than two months ago, while Roboflow put their paper on RF-DETR on Arxiv just last week!

Object detection performance

How do these advancements reflect on performance benchmarks? The figure here underneath summarizes the performance of YOLO11, D-Fine, and RF-DETR for relevant model sizes on the well-known COCO dataset .

Performance comparison between leading model architectures for their corresponding nano (N), small (S), medium (M), and large (L) variants. Indicative latency measures for each model size indicated between brackets.
*Not pretrained on Objects365 dataset **RF-DETR L is not released yet

Some important take-aways from these numbers:

  1. Both D-Fine and RF-DETR clearly outperform YOLO 11 for all sizes.
  2. RF-DETR’s smaller models stand out, with the nano variant outperforming the others by a wide margin. This is likely because RF-DETR-N already benefits from a strong DINOv2 backbone.
  3. D-Fine’s performance scales the best with model size, with the large variant scoring a whopping 57.4 mAP.

Parameter count

So, RF-DETR for small, very fast models and D-Fine when things get more complex? There is another side to the story. To finish of this post, I’d like to highlight an important difference between D-Fine and RF-DETR. For that, let’s take a look at the following figure:

Model sizes in million parameters for YOLO11, D-Fine and RF-DETR for their corresponding nano (N), small (S), medium (M) and large (L) variants. YOLO11 shows the best downward trend for larger model sizes with D-Fine close.

One of the first things to stand out is that D-Fine and YOLO11 become significantly lighter as their model sizes shrink, while RF-DETR’s parameter count declines by only around 5 million. This somewhat surprising observation results from the fact that RF-DETR was trained with a technique called Neural Architecture Search (NAS). NAS automatically finds network architectures that are Pareto optimal for the accuracy-latency trade-off.

Interestingly, the “small” RF-DETR architectures found by NAS end up only slightly lighter than the “large” variants. RF-DETR model sizes thus reflect speed rather than parameter count. D-Fine‘s model sizes on the contrary are on par with YOLO 11, making them the more versatile DETR architecture that can be adapted in a wide range of scenarios, including resource-constrained edge environments.

Conclusion

Real-time Detection Transformers represent one of the most significant recent shifts in computer vision. Their rapid evolution shows how transformers have become not only viable but actually preferred in scenarios that demand both high speed and high accuracy , even in resource-constrained scenarios. Just as important, their Apache 2.0 License makes them easy to use, enabling practical adoption beyond academic benchmarks.

D-Fine and RF-DETR have set the new standard for real-time object detection moving forward. D-Fine shows the best scaling in both speed, accuracy, and model size. The small RF-DETR variants are remarkably accurate and fast for their size, but the bigger models fall short of D-Fine when evaluated on the well-known COCO dataset. However, the field keeps on changing rapidly, so we’ll keep on tracking progress on both to make the best possible choices for every problem.

If you’re working on demanding detection problems where accuracy, robustness, and efficiency matter, we can help. We tailor DETR-based models to your specific application, integrate them in video processing pipelines, and set up continuous improvement loops to ensure performance keeps rising as new data comes in. Reach out ; we’d be excited to turn cutting-edge Detection Transformer research into real, production-grade impact for your system.

My private information is worth $30

Hacker News
blog.melashri.net
2025-11-22 11:04:15
Comments...
Original Article

A couple of weeks ago, I was notified that I can be part of class action settlement against University of Minnesota for a data breach that exposed my personal information. According to the details, In 2021, the University of Minnesota experienced a data breach that exposed personal information of "individuals who submitted information to the university as a prospective student, attended the university as a student, worked at the university as an employee or participated in university programs between 1989 and Aug. 10, 2021." source . I'm an alumnus of this university, so my information was part of that breach.

The university of course as a classical cooperative entity took the easy route that the legal system provides. They refuse to admit any wrongdoing, but they agreed to pay $5 million to settle the class action lawsuit. The settlement is open to anyone who had their personal information exposed in the breach, which includes names, addresses, dates of birth, Social Security numbers, and other sensitive data.

What is more insulting than that the university did not issue a formal apology to the affected individuals, is that they are offering a mere $30 per person as compensation for the breach. Yes to be honest they include this standard 24 months of dark web monitoring and identity theft protection services, but the value of my personal information is set to $30. Which even would be less if the number of people submitting exceeds the funding available for the settlement.

So according the university that sends me two or three emails per week asking me to donate to them, my personal information is worth $30. I understand that my Social Security number and other personal information got exposed in other breaches (Thanks to T-mobile and others). But the current status quo is that it does not matter whether it is a commercial entity or a public one, they will act in the same way. They will not take responsibility for their actions, and they will not compensate you for the damage they caused. They will just offer you a small amount of money and hope that you will forget about it.

The University of Minnesota is not the only one doing this. Many other institutions and companies have been caught in data breaches and have offered similar settlements. But it is still disappointing to see that they are not taking the issue seriously. This same university which promised a life access to email address which they did not honor, is now offering me $30 for my personal information. It is a slap in the face to all of us who have been affected by this breach. So I will not be submitting a claim for the settlement. I will not be accepting their offer of $30. I would have much preferred if they had taken responsibility for their actions and issued a healthy apology. But they did not. This would have been a good start. But they did not. And they will not.

The basic problem is that they do not care about us. They care about their reputation and their bottom line. They do not care about the damage they caused to our personal information. They do not care about the trust they have broken. They just want to move on and forget about it. When this happens from a corporation or a company, I can understand it. But when it happens from a public institution that is supposed to serve the public interest, it is unacceptable. How would I trust anything coming from them in the future? They have shown that they don't care about their alumni or their students.

The regulation is very weak, and the courts/laws are not doing enough to hold these institutions accountable. The fines are too low, and the settlements are too small. The only way to change this is to demand better regulations and stronger penalties for data breaches. We need to hold these institutions accountable for their actions and make them pay for the damage they cause. If the fines and compensation were higher, then the incentives would be aligned, and they would take data security more seriously. And would invest more in protecting our personal information instead of the ever-increasing administrative costs and salaries of the top executives.

US Universities are not only charging high tuition fees for education, but they are charging even researchers with external grants to use their facilities. If you get NSF or NIH grant, you have to pay the university a percentage of the grant as an indirect cost. The percentage varies from one university to another, but it is usually around 50%. This means that if you get a 100,000 USD grant, the university will take out 50,000 USD as indirect costs (NSF or NIH will end up paying 150,000 USD). This is a huge amount of money that could be used for research, but it is going to the university's administrative costs and salaries of the ever-increasing number of administrators.

For what it is worth that the universities is currently under fire for a variety of reasons, mostly politically motivated, but there are many valid reasons to be critical of the way they are run. The way they handle data breaches is just one of them. The amount of disrespect they show to their alumni and students is another. The way they prioritize administrative costs over education and research is yet another. It is time for us to demand better from our universities and hold them accountable for their actions.

After writing this post and trying to proofread it, I realized that I repeated "My personal information is worth $30" multiple times. I guess it is a sign that I am still angry about it. But also realized that if I had written this in Arabic it would have been much more concise. The poetic nature of writing in grievance in Arabic is much more effective than in English. But I will leave that for another time.

AIPAC Donors Back Real Estate Tycoon Who Opposed Gaza Ceasefire For Deep Blue Chicago Seat

Intercept
theintercept.com
2025-11-22 11:00:00
Progressive Rep. Danny Davis rejected AIPAC cash at the end of his career. Now the Israel lobby is coming for his seat. The post AIPAC Donors Back Real Estate Tycoon Who Opposed Gaza Ceasefire For Deep Blue Chicago Seat appeared first on The Intercept....
Original Article

Pro-Israel donors have picked a candidate to replace Rep. Danny Davis in Chicago.

Jason Friedman, one of 18 candidates vying to replace Davis in the March Democratic primary next year, has pulled ahead of the pack in fundraising. His campaign reported donations totaling over $1.5 million in its October filing with the Federal Election Commission.

About $140,000 of that money comes from major funders of pro-Israel groups, including the American Israel Public Affairs Committee PAC and its super PAC, United Democracy Project. The two groups spent more than $100 million on elections last year and ousted two leading critics of Israel from Congress . The pro-Israel donors’ support this year is an early sign that Friedman’s race is on AIPAC’s radar.

A former Chicago real estate mogul , Friedman launched his campaign in April, before Davis announced his retirement. From 2019 to 2024, he was chair of government affairs for the Jewish United Fund, a charitable organization that promotes pro-Israel narratives, noting on its website that “Israel does not intentionally target civilians,” “Israel does not occupy Gaza,” and “There is no Israeli ‘apartheid.’” Friedman has not made Israel a part of his campaign platform, but last month, the Joint Action Committee for Political Affairs, a pro-Israel PAC, held an event for its members to meet him.

AIPAC has not said publicly whether it’s backing a candidate in the race, but more than 35 of its donors have given money to Friedman’s campaign. Among them, 17 have donated to the United Democracy Project, and eight have donated to both. Together, the Friedman donors have contributed just under $2 million to AIPAC and UDP since 2021.

That includes more than $1.6 million to UDP and more than $327,000 to AIPAC, with several donors giving six or five-figure contributions to the PACs. Friedman’s donors have also given $85,500 to DMFI PAC, the political action committee for the AIPAC offshoot Democratic Majority for Israel , and another $115,000 to the pro-Israel group To Protect Our Heritage PAC, which endorsed another candidate in the race, Chicago City Treasurer Melissa Conyears-Ervin. The Conyears-Ervin campaign and To Protect Our Heritage PAC did not respond to a request for comment.

Friedman is running largely on taking on President Donald Trump on issues from health care to education and the economy. His campaign website says he supports strong unions, access to education, reducing gun violence, and job training and support. Prior to his tenure leading his family real estate empire, Friedman worked in politics under former President Bill Clinton and for Sen. Dick Durbin on the Senate Judiciary Committee.

Reached by phone, the pro-Israel donor Larry Hochberg told The Intercept that he was supporting Friedman because he thought he’d be a good candidate. “I’ll leave it at that,” Hochberg said.

A former AIPAC national director, Hochberg sits on the board of Friends of the Israel Defense Forces and co-founded the pro-Israel advocacy group ELNET , which has described itself as the AIPAC of Europe. Hochberg has given $10,000 to AIPAC, $5,000 to DMFI PAC, and just under $30,000 to To Protect Our Heritage PAC. In September, he gave $1,000 to Friedman’s campaign. Asked about his support for AIPAC and DMFI, he told The Intercept: “I don’t think I want to say any more than that.”

Former Rep. Marie Newman, a former target of pro-Israel donors who represented Illinois’s nearby 3rd District and was ousted from Congress in 2022, criticized Friedman for the influx in cash.

“If you receive money from AIPAC donors who believe in genocide and are funding genocide, then in fact, you believe in genocide,” Newman told The Intercept. She’s backing another candidate in the race, gun violence activist Kina Collins, who ran against Davis three times and came within 7 percentage points of unseating him in 2022.

Friedman is running against 17 other Democratic candidates, including Collins and Conyears-Ervin. During Collins’s third run against Davis last year, United Democracy Project spent just under half a million dollars against her. Davis, who received support from a dark-money group aligned with Democratic leaders in his 2022 race, has endorsed state Rep. La Shawn Ford to replace him. Other candidates include former Cook County Commissioner Richard Boykin, former Forest Park Mayor Rory Hoskins, immigrant advocate Anabel Mendoza, organizer Anthony Driver Jr., emergency room doctor Thomas Fisher, and former antitrust attorney Reed Showalter, who has pledged not to accept money from AIPAC.

Friedman’s campaign did not respond to a request for comment.

The genocide in Gaza has aggravated fault lines among Democrats in Chicago. Last year, the Chicago City Council narrowly passed a resolution calling for a ceasefire in Gaza, with Mayor Brandon Johnson casting the tie-breaking vote. As chair of government affairs for the Jewish United Fund, Friedman signed a letter to Johnson last year from the group and leaders of Chicago’s Jewish community, saying they were “appalled” at the result. Friedman’s campaign did not respond to questions about his position on U.S. military funding for Israel or the war on Gaza.

At least 17 Friedman donors have given to the United Democracy Project, with contributions totaling over $1.6 million. That includes nine people who gave six-figure contributions to UDP and seven who gave five-figures. Twenty-nine Friedman donors have given to AIPAC PAC, including eight of the same UDP donors.

Among those supporters are gaming executive Greg Carlin, who has given $255,000 to UDP and gave $3,500 to Friedman’s campaign in April; investor Tony Davis, who has given $250,000 to UDP and also gave $3,500 to Friedman’s campaign in April; and attorney Steven Lavin, who has given $125,000 to UDP and gave $7,000 to Friedman’s campaign in June. Carlin, Davis, and Lavin did not respond to a request for comment.

Attorneys Douglas Gessner and Sanford Perl, who work at Friedman’s previous law firm, Kirkland & Ellis, have given $105,000 and $100,000 to UDP. Both have also given to AIPAC PAC: Gessner over $50,000 and Perl over $44,000. Gessner gave $3,000 to Friedman’s campaign in September, and Perl gave $3,400 in April. Gessner and Perl did not respond to requests for comment.

“If you’re taking money from people who are supporting a far right-wing government that is executing a genocide, what does that say about you?”

Three other donors who have each given $1 million to UDP have given to Friedman’s campaign: Miami Beach biotech executive Jeff Aronin, Chicago marketing founder Ilan Shalit, and Jerry Bednyak, a co-founder of Vivid Seats who runs a private equity company focused on e-commerce.

“You could be the nicest person in the world,” said Newman, the former Illinois congresswoman. “But if you’re taking money from people who are supporting a far right-wing government that believes in genocide and is executing a genocide, what does that say about you?”

Friedman’s campaign coffers saw six-figure boosts on three days in June and September — vast outliers compared to most days in his first quarter. Those kinds of fundraising boosts are often associated with a blast email from a supportive political group to its network of donors, according to a Democratic strategist with knowledge of the race. AIPAC did not respond to a request for comment about whether the group had sent such an email encouraging supporters to contribute to Friedman’s campaign.

Friedman’s fundraising boost has also come largely from the finance and real estate industries, where just under a quarter of his donors work. He has also given $36,750 of his own money to his campaign.

Hindsight – Type-safe and evolvable event sourcing for Haskell

Lobsters
hindsight.events
2025-11-22 10:30:49
Comments...
Original Article

Type-safe and evolvable event sourcing for Haskell

Hindsight is a type-safe event sourcing system that provides strong compile-time guarantees for event handling, versioning, and consistency with multiple storage backends.

Hindsight in Action

Type-Safe Event Definition

Define events with compile-time versioning guarantees. No runtime surprises.

-- Event definition
instance Event "user_registered"

-- Event payload
data UserInfo = UserInfo
  { userId :: Text
  , email :: Text
  } deriving (Generic, FromJSON, ToJSON)

-- Version declaration
type instance MaxVersion UserRegistered = 0
type instance Versions UserRegistered =
  '[UserInfo]

-- Migration (automatic for single version)
instance MigrateVersion 0 UserRegistered

Backend-Agnostic Subscriptions

Subscribe to events with handlers that work across all backends.

{-# LANGUAGE RequiredTypeArguments #-}

-- Subscribe to events (works with any backend)
subscribeToUsers :: BackendHandle backend -> IO (SubscriptionHandle backend)
subscribeToUsers store =
  subscribe store
    ( match "user_registered" handleUser :?
      MatchEnd )
    (EventSelector AllStreams FromBeginning)
  where
    -- Handler runs for each event
    handleUser envelope = do
      let user = envelope.payload
      putStrLn $ "New user: " <> user.email
      return Continue

SQL Projection Handlers

Transform events into queryable read models with ACID guarantees.

{-# LANGUAGE RequiredTypeArguments #-}

-- Projection handler (PostgreSQL transactions)
userProjection :: ProjectionHandler "user_registered" backend
userProjection envelope = do
  let user = envelope.payload

  -- Execute SQL in transaction
  statement () $ Statement sql encoder decoder True

  where
    sql = "INSERT INTO users (id, email) VALUES ($1, $2)"
    encoder = contrazip2
      (param (nonNullable text))
      (param (nonNullable text))

Flexible Backend Choice

Start simple, scale when ready. Same API, different backends.

-- File system store
fsStore :: IO (BackendHandle FilesystemStore)
sqlStore =
  newFilesystemStore "./events"

-- PostgreSQL store
sqlStore :: IO (BackendHandle PostgreSQLStore)
sqlStore = do
  pool <- createPool postgresSettings
  newPostgreSQLStore pool

-- Same operations, different backends
insertEvents devStore   Nothing batch
insertEvents sqlStore  Nothing batch

Key Features

  • Type-safe events with automatic upcasts and golden test generation
  • Real-time subscriptions for event stream processing
  • Multiple backends : Memory, Filesystem, and PostgreSQL
  • Projection system for building read models
  • Strong consistency and ordering guarantees

Get Started

Check out our documentation

Jack Ma's family shifted wealth to UK after years-long 'disappearance'

Hacker News
www.source-material.org
2025-11-22 10:19:27
Comments...
Original Article

Billionaire’s wife bought London mansion as he reconciled with Chinese authorities

The family of Chinese billionaire Jack Ma bought a £19.5 million London mansion amid a rapprochement with Chinese authorities after years of scrutiny and political exile.

Ma’s wife, Cathy Ying Zhang, acquired the six-bedroom Edwardian house, formerly the Italian embassy, in London’s elite Belgravia district in October 2024, property records show.

The purchase came after Ma’s return to public life after disappearing from view in the aftermath of a speech criticising China’s financial system. It could be seen as a “precautionary diversification” in case Ma again provokes Beijing’s ire, said Sari Arho Havrén, a China specialist at the Royal United Services Institute.

“Wealthy families are hedging against regime risk—one never knows when policies may turn hostile again,” she said. “Affluent families are diversifying quietly. Rule of law societies still hold considerable appeal.”

Ma, 61, is the founder of Alibaba Group, whose online commerce platforms have earned him a fortune of around $30 billion. The Belgravia house, the Ma family’s first known property holding in the UK, may have been funded by the sale of Alibaba shares in 2023, Havrén said.

Ma’s wife Zhang, who has taken Singaporean citizenship, is reportedly the sole director of an offshore company that Ma used to buy a château and vineyards in France.

Last year it was reported that Zhang spent up to $38 million on three commercial properties in Singapore. The buying spree is part of a trend that has seen prominent Chinese businesspeople move money abroad for fear of asset freezes or capital controls.

Many have left China altogether. As many as 13,800 “high-net-worth individuals” emigrated in 2024—a 28 percent rise from 2022, according to investment migration consultants Henley & Partners.

The sale of the Belgravia mansion, managed by Knight Frank and Beauchamp Estates and handled by law firm Withers LLP, was rushed through ahead of a rise in the UK’s stamp duty surcharge for overseas buyers, according to a November 2024 report that did not name the buyer.

Beauchamp and Knight Frank declined to comment. Zhang and Ma Withers did not respond to questions put to them via Withers.

In 2015, it was reported that Ma family purchased ‘Asia’s most expensive home’ in Hong Kong’s Victoria Peak which was formerly owned by the Belgian government. In the same year, it was reported that Ma had bought a 28,000 acre property in upstate New York for $23 million.

Ma vanished from public view in late 2020 after he criticised China’s financial regulators. Beijing reportedly punished him with a fine of nearly $3 billion and halted a stock market listing by Ant Group, an offshoot of Alibaba.

He resurfaced in China in 2023 after an apparent reconciliation with the administration of President Xi Jinping, occasionally attending public events. In February 2025, he was seen shaking Xi’s hand at event with Chinese industry leaders. However, Ma’s public remarks went unreported by official state media, prompting analysts to suggest that he had not been “completely rehabilitated”.

In April, The Guardian reported that Chinese authorities enlisted Ma as part of a campaign to pressure a dissident businessman to return to China from France to help prosecute an official who had angered the regime.

“They said I’m the only one who can persuade you to return,” Ma reportedly told the unnamed businessman in a telephone call. The Chinese government called the allegations “pure fabrication”.

Headline picture: Beauchamp Estates

Libpng 1.6.51: Four buffer overflow vulnerabilities fixed

Hacker News
www.openwall.com
2025-11-22 10:08:31
Comments...
Original Article
Openwall
[<prev] [day] [month] [year] [list]
Message-ID: <CAAoVtZw-pkvsSTaXAHjDdUC3NRDwvwVNT8D4BpO5z3d79W-FVg@mail.gmail.com>
Date: Sat, 22 Nov 2025 03:27:35 +0200
From: Cosmin Truta <ctruta@...il.com>
To: oss-security@...ts.openwall.com
Subject: libpng 1.6.51: Four buffer overflow vulnerabilities fixed:
 CVE-2025-64505, CVE-2025-64506, CVE-2025-64720, CVE-2025-65018

Hello, everyone,

libpng 1.6.51 has been released to address four buffer overflow
vulnerabilities discovered through fuzzing and security research.
This release fixes two high-severity and two moderate-severity CVEs
affecting libpng 1.6.0 through 1.6.50.

CVE-2025-64505 (CVSS 6.1, Moderate): Heap buffer over-read in
png_do_quantize via malformed palette index.

CVE-2025-64506 (CVSS 6.1, Moderate): Heap buffer over-read in
png_write_image_8bit with 8-bit input and convert_to_8bit enabled.

CVE-2025-64720 (CVSS 7.1, High): Out-of-bounds read in
png_image_read_composite via palette premultiplication with
PNG_FLAG_OPTIMIZE_ALPHA.

CVE-2025-65018 (CVSS 7.1, High): Heap buffer overflow in
png_combine_row triggered via png_image_finish_read when processing
16-bit interlaced PNGs with 8-bit output format.

All vulnerabilities require user interaction (processing a malicious
PNG file) and can result in information disclosure and/or denial of
service. CVE-2025-65018 may enable arbitrary code execution via heap
corruption in certain heap configurations.

GitHub Security Advisories:
- CVE-2025-64505:
https://github.com/pnggroup/libpng/security/advisories/GHSA-4952-h5wq-4m42
- CVE-2025-64506:
https://github.com/pnggroup/libpng/security/advisories/GHSA-qpr4-xm66-hww6
- CVE-2025-64720:
https://github.com/pnggroup/libpng/security/advisories/GHSA-hfc7-ph9c-wcww
- CVE-2025-65018:
https://github.com/pnggroup/libpng/security/advisories/GHSA-7wv6-48j4-hj3g

Fixes:
- CVE-2025-64505:
https://github.com/pnggroup/libpng/commit/6a528eb5fd0dd7f6de1c39d30de0e41473431c37
- CVE-2025-64506:
https://github.com/pnggroup/libpng/commit/2bd84c019c300b78e811743fbcddb67c9d9bf821
- CVE-2025-64720:
https://github.com/pnggroup/libpng/commit/08da33b4c88cfcd36e5a706558a8d7e0e4773643
- CVE-2025-65018:
https://github.com/pnggroup/libpng/commit/16b5e3823918840aae65c0a6da57c78a5a496a4d

https://github.com/pnggroup/libpng/commit/218612ddd6b17944e21eda56caf8b4bf7779d1ea

Note: CVE-2025-65018 requires both commits for correct remediation.

Release: https://github.com/pnggroup/libpng/releases/tag/v1.6.51

Credit: Samsung-PENTEST (CVE-2025-64505, CVE-2025-64506, CVE-2025-64720),
weijinjinnihao (CVE-2025-64506), yosiimich (CVE-2025-65018), with analysis
by Fabio Gritti and John Bowler.

Users should upgrade to libpng 1.6.51 immediately.

Cosmin Truta
libpng maintainer

Powered by blists - more mailing lists

Please check out the Open Source Software Security Wiki , which is counterpart to this mailing list .

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages .

Automatic Syntax Error Recovery (2020)

Lobsters
tratt.net
2025-11-22 10:02:45
Comments...

Roblox CEO Makes a Fool of Himself in Car-Crash Interview

Hacker News
kotaku.com
2025-11-22 09:41:29
Comments...
Original Article

As “pedophile hellscape” Roblox finally adds a rudimentary measure to try to prevent children from being exploited via its network of games and chatrooms used by 151 million people, the company’s CEO spoke to the New York Times podcast Hard Fork about child safety. And it didn’t go great. It really didn’t go great.

Roblox is coming under increasing scrutiny after decades of failing to implement even the most basic of protective measures to keep its pre-teen players away from the networks of pedophiles who use the game to find victims. Described last year as “a pedophile hellscape for kids” by Hindenburg Research , the game quickly introduced a new assortment of measures last October that did next to nothing, leading this year to a great deal of legal interest. Three months back, the state of Louisiana announced its intentions to sue Roblox for the dangers it was allowing, joined since by Kentucky and Texas. These actions come alongside 35 other ongoing lawsuits , including one from only yesterday by a family in Cuyaho County following Roblox ‘s alleged use in the tragic and too common abuse of their 13-year-old son.

On Wednesday of this week, Roblox announced it was beginning to roll out a new method of checking player ages before they could access chat, this time using facial recognition to try to restrict players to only talking to others in their age range. Such facial ID checks have become commonplace in the UK following the disastrously poor UK Online Safety Act, by which porn sites and other age-restricted outlets are required to verify users’ ages. This has led to sites like X, Bluesky and Discord also requiring British users to prove their age, usually by showing their face on camera in a measure that entirely thwarts all seven people who haven’t heard of a VPN.

Brainrot indeed.
© Roblox

Regarding this rollout, the New York Times ‘ Casey Newton spoke to Roblox co-founder and CEO David Baszucki about the new plans , and whether they can really help. It doesn’t begin well. When asked about the “scope of the problem” of predators in the application, Baszucki came in shoulders first saying, “We think of it not necessarily just as a problem, but an opportunity as well.” Ah, the good ol’ opportunities available when your company’s product is rife with pedophilia. He continued, outlining how wonderful it is that young people can build and communicate together, how they have “150 million daily actives, 11 billion hours a month, like what is the best way to keep pushing this forward.” It is the most astonishingly tone-deaf response.

Newton attempts to get some sensible answers from Baszucki about how the ID checks will work, and why they won’t be as easily evaded as so many others , and amidst Baszucki’s corporate waffle he’s told that Roblox is apparently always looking out for “weird signals” and will ask for further age verification should these appear, although he didn’t explain what those signals might be, nor what these further checks would be. But then Newton goes on to ask the painfully necessary question: why has it taken 19 years to even try to stop adults from being able to speak to children?

Baszucki responds by talking about the evolution of Roblox ‘s text filtering tech for inappropriate language and personally identifying information over the last two decades, and how it’s always improving, which is great but clearly not relevant to the question. Newton is having none of it, responding, “Yeah, I mean, I don’t know. When I read these lawsuits and these investigations into the company, it does not seem like predators are having that hard of a time getting around your filters…So I’m curious what’s made you so confident that things are working?” Baszucki’s response is incoherent.

“I don’t want to comment on it. We do see the chat logs of those. And we can see interesting and, many times, ways of people trying to—I’d say, many times people who are fairly sophisticated, and I’m not going to say all of them, many times kids who are over 13, who actually in any other app are not running in a filtered situation, unfortunately, figuring out how to jump to some other platform where they can communicate unfiltered, where they can share images, all of it. It’s one of the primary things we’re doing is trying to keep people on our platform. It’s an interesting industry situation. I would say, we’re not waiting for the rest of the industry. We’re like, we’re always trying to stay ahead of it. On the California age-appropriate design code, for example, we’re like, we are already doing all of that stuff. And the same thing with age estimation: We’re not doing this because of any laws that are coming, we think it’s the right thing to do.”

Newton very impressively keeps his cool, and rather than pointing out that this answer had nothing to do with the situation, nor indeed how unbelievable it is that the CEO of the company would say “I don’t want to comment on it” when asked why he’s confident in age tech that clearly doesn’t work, he instead points out that the lawsuits are demonstrating that “ Roblox is kind of, you know, where predators go to find kids.”

Things become increasingly tense, as Baszucki tries to claim this is a misrepresentation, and when pressed on whether he truly doesn’t believe Roblox has a predator problem replies, “I think we’re doing an incredible job at innovating relative to the number of people on our platform and the hours, in really leaning in to the future of how this is going to work.”

Roblox being used on a phone.
© Roblox

What becomes so grimly apparent is that even now, even after a year of extraordinary scrutiny and legal pressure, Baszucki still doesn’t have a grip on the situation at all. To be so ill-prepared for such obvious questions, to have no coherent responses for why the new tech will be effective, and to go so out of his way to appear so uninterested in the subject, is astonishing. As he’s pressed further on the ease with which predators can suggest children speak to them on another platform, Baszucki eventually unravels into literal gibberish:

“I would actually say that is a very simple red flag. Like, that sounds like text filter many prior generations. So I would say the techniques are much more sophisticated than that. We’re constantly getting better than that and sometimes it’s adversarial. Like, we’re getting into, you know, if we cryptographically were going to try to have an exchange of how to share your Snap handle or whatever handle. We see some of that starting, like things we’ll have to essentially prevent in the future.”

The CEO seems to believe that the scale of Roblox is somehow an excuse to justify its problems, repeatedly coming back to its 150 million users and 11 billion hours a month, as if this explains it all away. But more problematically, as Newton points out that the company wants those numbers to grow, Baszucki immediately switches back to talking about what an amazing financial opportunity this is. Given Roblox is really struggling to turn those users into money , it reads like he’s only speaking to investors and analysts who are increasingly concerned about Roblox ‘s lack of profits. So many responses begin with an infuriatingly irrelevant attempt to avoid the question, and then end with words like “And you could imagine Roblox at 20-X this scale, having 50 percent of the gaming market space all in a single place.” It’s so crass!

Baszucki wants kids to gamble in Roblox
© The New York Times / Kotaku

But not nearly as crass as Baszucki’s response to a question over the Hindenburg Research report into the scale of the issue with pedophiles . Hindenburg was an activist short-selling research firm that would investigate companies for fraud, malfeasance and indeed rampant use by predators, until its creator chose to disband the group in January this year . At the time of the report, Baszucki said that it was wrong to claim that Roblox had reduced its spend on trust and safety, so Newton asked for specifics.

“Fun,” says Baszucki, a man who appears pathologically incapable of reading the room. “Let’s keep going down this. And so, first off, Hindenburg is not longer in existence, correct? So, you should report on that. They went out of business for some reason…” He then demanded to know if Newton had researched the answer for himself, before saying that it’s because they’ve moved so much of the safety regulation onto AI, all while sounding absolutely furious that he was being asked. He then demands that Newton agree that if AI is more effective, it’s better to use it, and when Newton does, Baszucki starts to behave incredibly immaturely. “Good, so you’re aligning with what we did. High-five.” Then when Newton tries to ask a question, he interrupts to say, “Thank you for supporting our Roblox decision matrix .” Then interrupts yet again to say, “I’m so glad you guys are aligned with the way we run Roblox . High-five.” Think he’s done? Nope. Yet again he interrupts the question with, “Is this a stealth interview where actually you love everything we’re doing and you’re here to stealthily support it?”

And he doesn’t stop. When co-host Kevin Roose tried to get things back on the rails, Baszucki kept going with the same pathetic line. Challenged on how AI has proved very ineffective for social media companies, he just carries on saying it. The only thing that stopped this tantrum was allowing the CEO to talk about Polymarket, a cryptoscam-based prediction market, letting people place bets on things as ridiculous as awards ceremonies and future weather patterns. Some believe it’s becoming a useful way to predict markets, and that group includes Baszucki who is…and I cannot believe I’m typing this…looking to put it into Roblox so children can gamble.

He wants to find a way to do this that’s “legal” and “educational,” at which point the podcast hosts begin openly mocking the stupidity. And then, thank god, they run out of time.

Chris McCausland: Seeing into the Future – an astonishing look at how tech is changing disabled people’s lives

Guardian
www.theguardian.com
2025-11-22 07:00:32
Prepare to have your perspective shattered by the comedian’s visits to our US tech overlords. The upcoming advancements for those with disabilities are life-changing Washing machines liberated women to get soul-crushing jobs that ate up their free time. Social media gave the world one revolution – b...
Original Article

W ashing machines liberated women to get soul-crushing jobs that ate up their free time. Social media gave the world one revolution – before it destabilised democracies everywhere else. Now AI is here, and its main job seems to be replacing screenwriters. It’s easy to fall into techno-pessimism, but new documentary Seeing into the Future (Sunday 23 November, 8pm, BBC Two) has a different angle. For disabled people, tech has already brought about life-changing advancements. And we haven’t seen anything yet.

It is presented by comedian and Strictly winner Chris McCausland, who is blind. Some of the most casually astonishing scenes occur early on, showing how he uses his phone – essentially, an eye with a mouth. “What T-shirt is this?” he asks, holding up a garment. “A grey T-shirt with a graphic logo of Deftones,” his phone obliges. It can even tell him if the shirt needs ironing. But it’s where all this is going that fascinates McCausland, so he heads to the US, to see what’s in development at the houses of our tech overlords.

He swings by a facility belonging to Meta to try out some smart glasses. To my mind, he may as well be entering the lair of the White Worm, or popping round for macarons at Dracula’s castle. But that’s partly because I’m not in direct need of such technology, and the documentary’s job is to highlight possibility not jump on pitfalls. It’s not like Zuckerberg is personally in the lab, stroking a cat and spinning round on an egg chair.

I love having my perspective shaken up. A glass screen with no buttons sounds like the most excluding device imaginable, McCausland acknowledges, yet his phone became the most accessible tool he’s ever used. He’s similarly excited by the Meta Specs – I don’t think that’s what they’re actually called – which are always on and offer live video interpretation, telling you what you’re looking at. Like a phone but, crucially, wearable. “The one thing blind people never have is two hands free,” he observes.

McCausland with Maxine Williams, VP of accessibility and engagement at Meta, trying out their smart glasses.
McCausland with Maxine Williams, VP of accessibility and engagement at Meta, trying out their smart glasses. Photograph: BBC/Open Mike Productions

At MIT, a nanotechnologist tells him how molecular devices could repair cells inside our bodies. He tries bionic gait assistance – a device that straps on to the calf, giving the wearer added power. It looks like the knee brace Bruce Wayne uses in The Dark Knight Rises to kick through a brick wall when he learns he’s got no cartilage in his knee. Most moving, in every sense, he takes a trip in a driverless car. It’s the first time McCausland has taken a car journey alone.

Driverless cars will arrive in the UK next spring. (That’s a long journey.) They are what I would call an instinctive NOPE. But “It’s not massively different to trusting a driver I don’t know,” McCausland reflects. They are extraordinary: mounted with spinning radars, making calculations involving the speed of light to 3D-model the environment in real time. They may as well have gullwing doors. The fact the steering wheel moves on its own is McCausland’s favourite thing about them, which is charming. Coolness is certainly the second best thing technologists can pursue, after equality of access to lives of dignity and independence. In my defence, it’s not just that I don’t trust technology. It’s that I don’t trust profit-driven Big Tech companies to behave for the public good, or with any accountability.

There’s a parallel pleasure in the documentary – transatlantic cultural difference. These are not just Americans, remember. These are San Franciscan Futurists. The inadvertent comedy is amplified by the addition of the dry McCausland. A man so British that, even when he’s interviewing a nanotechnologist about blood-borne computers that could potentially restore his sight, he sounds as if he’d hand you thirty English notes right now if you could teleport him to the pub instead.

Even the tech is unmistakably American. “I can hear a plane?” prompts McCausland, trialling Zuckerberg’s glasses. “Yes, a plane is visible in the clear blue sky,” responds the earnest spectacles. Later, our presenter wryly looks back toward his own camera crew. “Do they look like they know what they’re doing?” he provokes. “Judging by their equipment, yes, they are professionals.” Go-go-gadget-missing-the-joke. Computers may increasingly be able to play God, but irony remains a step beyond. Even with a Batman leg brace.

Bro boost: women find LinkedIn traffic ‘drives’ if they pretend to be men

Guardian
www.theguardian.com
2025-11-22 07:00:32
Collective experiment found switching profile to ‘male’ and ‘bro-coding’ text led to big increase in reach, though site denies favouring posts by men Do your LinkedIn followers consider you a “thought leader”? Do hordes of commenters applaud your tips on how to “scale” your startup? Do recruiters sl...
Original Article

D o your LinkedIn followers consider you a “thought leader”? Do hordes of commenters applaud your tips on how to “scale” your startup? Do recruiters slide into your DMs to “explore potential synergies”?

If not, it could be because you’re not a man.

Dozens of women joined a collective LinkedIn experiment this week after a series of viral posts suggested that, for some, changing their gender to “male” boosted their visibility on the network.

Others rewrote their profiles to be, as they put it, “bro-coded” – inserting action-oriented online business buzzwords such as “drive”, “transform” and “accelerate”. Anecdotally, their visibility also increased.

The uptick in engagement has led some to speculate that an in-built sexism in LinkedIn’s algorithm means that men who speak in online business jargon are more visible on its platform.

Like most large social media platforms, LinkedIn uses an algorithm to determine which posts it shows to which users – boosting some, and downgrading others.

In a blog post on Thursday , LinkedIn acknowledged the trend, but said it did not consider “demographic information” in deciding who gets attention. Instead, it said, “hundreds of signals” factor into how a given post performs.

“Changing gender on your profile does not affect how your content appears in search or feed,” a spokesperson said. Be that as it may, the anecdotes are piling up.

“It has certainly been exciting,” said Simone Bonnett, an Oxford-based social media consultant who changed her pronouns to “he/him” and her name to “Simon E” on LinkedIn earlier this week.

“The kind of stats that I’m seeing at the moment are a 1,600% increase in profile views, which is wild if you think about what social media views look like at the moment, and a 1,300% increase in impressions. Also wild reach stats.”

Megan Cornish, a communications strategist for mental health tech companies, said she started experimenting with her LinkedIn settings after seeing her reach on the platform decline precipitously earlier this year.

First she changed her gender to “male”. Then she told ChatGPT to rewrite her profile in “male-coded” language, based on a LinkedIn post suggesting the platform favours “agentic” words such as “strategic” and “leader”.

Finally, she asked ChatGPT to rewrite old, badly performing posts from several months ago in similarly “agentic” language, figuring that recycling old, reworked content would help her isolate what effect “bro-coding” was having on her reach.

Things went great. Almost immediately, Cornish’s reach on LinkedIn spiked, increasing 415% in the week after she trialled the changes. She wrote a post about the experience, and it went viral , racking up nearly 5,000 reactions.

The problem was, she hated it. Before, her posts had been “soft”, she said. “Concise and clever, but also like warm and human.” Now, bro-Megan was assertive and self-assured – “like a white male swaggering around”.

She gave up after a week. “I was going to do it for a full month. But every day I did it, and things got better and better, I got madder and madder.”

Not everyone had the same experience as Cornish and Bonnett. Cass Cooper, a writer on technology and social media algorithms, said she changed her gender to “male” – and then her race to “white” (Cooper is Black). The overall result, she said, was a decline in her profile’s reach and engagement – an experience other women of colour on the platform have also discussed .

“We know there’s algorithmic bias, but it’s really hard to know how it works in a particular case or why,” she said.

While the LinkedIn experiments were “frustrating”, she said she believed they were a reflection of broader society-wide biases. “I’m not frustrated with the platform. I’m more frustrated with the lack of progress [in society].”

Users have been rumbling about LinkedIn’s weird position as a quasi-business, quasi-social network for some time, ever since the pandemic blurred professional boundaries and injected more oversharing into work. LinkedIn’s occasional tendency to elevate extreme “bro-coding” is best illustrated by social media accounts recording the excesses of the platform.

These latest “bro-coding” experiments, however, have their origins in what Cornish, Bonnett and others describe as algorithm changes in recent months that have caused female creators in particular to have markedly less visibility. This led to a series of informal experiments earlier this year, in which women and men in parallel industries posted the same content – and the men got drastically more reach.

LinkedIn uses an AI system to classify posts to its feed, it says, deciding how to disseminate them based on their content, as well as the poster’s professional identity and skills. It evaluates its algorithms regularly, it says, including “checks for gender-related disparities”.

A spokesperson for LinkedIn suggested that a recent decline in certain users’ reach came from a far higher volume of content on the network, adding that there had been a 24% increase in comments and a commensurate spike in video uploads in the past quarter.

Bonnett said the “bro-coding,” in her experience, was on the rise. “You always think of LinkedIn as being more genteel, more businesslike. It’s not like that any more. It’s starting to become the wild west.”

Building a deep-space logistics startup

Hacker News
ionmicrosys.com
2025-11-22 06:13:05
Comments...
Original Article

Satellite

Delivering the Future of Spaceflight

Rocket

Fast Deployment

Get your payload to the Moon, Mars, and other Deep Space destinations - soon. With rideshare missions launching as soon as 2028 , we can get your payload to where you need it, fast.

Dollar

Cost Efficient

Traditional chemical rockets are inefficient and expensive. Ion Thrusters provide a cost-efficient method to get your payload to the Deep Space destination of your choice.

Mars

Mars Ready

Our vehicles are designed to be able to deliver payloads to the red planet - on your schedule, with no need to wait for orbital alignment periods.

Will pay-per-mile raise Reeves money or drive people away from electric vehicles?

Guardian
www.theguardian.com
2025-11-22 06:00:33
Need for new road taxes is clear – but there are concerns that pricing plan could stall transition away from petrol Three pence: a small charge per mile for an electric vehicle, but a giant conceptual leap for Britain. Chancellors of the exchequer have long resisted any form of road pricing as polit...
Original Article

Three pence: a small charge per mile for an electric vehicle, but a giant conceptual leap for Britain.

Chancellors of the exchequer have long resisted any form of road pricing as politically toxic. That may be about to change next week: Rachel Reeves, perhaps inured to being pilloried for any money-raising proposal, is expected to introduce a charge explicitly linked to how far EVs drive.

The Treasury has all but confirmed some kind of charge will be announced at next week’s budget, but the details have not been revealed. According to an initial report in the Telegraph, EV drivers could from 2028 pay a supplement based on how far they had driven that year on top of their annual road tax, or vehicle excise duty (VED). That could be a self-declared estimate of distance or a check on the odometer at an MOT.

According to Department for Transport (DfT) figures, battery electric cars – with lower running costs than petrol – are used more: clocking up about 8,900 miles on average in 2024. At 3p a mile, that would bring in £267 a car from the 1.4m EVs currently on the road – about £375m a year in total.

Rachel Reeves leaves from 11 Downing Street in central London on Wednesday.
The Treasury has all but confirmed that some kind of charge on EVs will be announced when Rachel Reeves delivers the budget. Photograph: Carlos Jasso/AFP/Getty Images

The transport secretary, Heidi Alexander, was at pains to rule out a national road pricing scheme in the face of Commons attacks on Thursday – although a later “clarification” made clear that the EV pay-per-mile was still on the table.

The long-term picture is a looming shortfall in motoring tax revenues, as income from fuel duty evaporates in the transition to EVs. Petrol and diesel cars effectively pay a charge linked to how far they drive – but via fuel consumption at the pump.

Fuel duty of 52.95p a litre (roughly 5p a mile in average cars) will bring in £24.4bn this financial year, according to the latest forecast from the Office for Budget Responsibility, but the billions will dwindle away from 2030, when the ban on new pure petrol and diesel cars comes in.

The challenge is to find a fair replacement for an unsustainable system – and overcome longstanding resistance on the right to any form of road charging, bundled up in the culture wars around London’s ultra-low emission zone (Ulez) and low-traffic neighbourhoods with their claims of curtailed freedoms and increased surveillance.

A Ulez sign in central London.
Last year London’s mayor, Sadiq Khan, ruled out considering a pricing scheme after being battered by anti-Ulez hostility. Photograph: PA Images/Alamy

Some economists have championed schemes that would price roads by time and congestion – potentially fairer and a better tool to manage road use, but bringing in another level of tracking.

Any scheme should be kept simple, says Steve Gooding, the director of the RAC Foundation motoring thinktank. Although, when it comes to privacy, he adds: “The amount of data being generated by the modern car is phenomenal. If the DfT or DVLA start tracking their movements, people think Big Brother is watching. But Elon [Musk] – they’re not that fussed.”

A wider concern is that pay-per-mile would discourage drivers from switching to electric vehicles, crucial for cutting carbon emissions. Manufacturers, businesses and motoring groups such as Ford, AutoTrader and the AA have all spoken out on the timing of new charges at this point in the transition. Carmakers must, under Britain’s ZEV mandate , ensure one in three cars sold next year are zero-emission, rising to 80% by 2030 (with hybrids allowed to make up the remaining 20%).

While grants remain of up to £3,750 on new electric vehicles and – for some – the running costs remain much cheaper, some discounts or tax and charge exemptions have already ended. Transport for London recently confirmed EVs would be liable for the capital’s congestion charge from next year , and zero-emission cars started paying VED in April.

According to a report for the Social Market Foundation (SMF) thinktank, New Zealand provides a cautionary tale. EVs were made liable last year for its road-user charge, which previously only applied to diesel vehicles, whereby drivers buy paper permits in units of 1,000km (621 miles). The move, allied to the end of buyer grants and tax exemptions, led to a sharp drop in new EV sales – now just 4% of the market, having peak 19%.

Electric vehicles at a charging point in Auckland, New Zealand
Electric vehicles at a charging point in Auckland, New Zealand, where EVs were made liable last year for its road-user charge. Photograph: MIchael Craig/AP

SMF says that Iceland, which also brought EVs into pay-per-mile schemes last year, maintained incentives and differentials in pricing and had a much smaller decline in market share.

Advocates for the new technology are alarmed. The Electric Vehicle Association England, a group representing drivers, warned in a letter to the chancellor that consumer sentiment was still sceptical about EVs.

For many, running costs are no longer the incentive they once were – particularly for those reliant on public charging points, usually in poorer areas and without a driveway. Ginny Buckley, the chief executive of Electrifying.com, an EV reviews platform and marketplace, says: “If you can’t rely on off-peak, affordable home charging and you’re reliant on the public charging network, for many people it will cost you more per mile to run your EV than it will a petrol car.”

Graham Parkhurst, a professor of sustainable mobility at the University of the West of England, describes the vast difference between domestic chargers and public charging points – which attract VAT at 20% on top – as a “political timebomb”, further dividing the haves and have-nots.

Even long-term proponents of pay-per-mile such as Parkhurst warn of the need to tread carefully: “Charging according to how much a vehicle moves makes sense. Fuel duty does that. But we need time to work out how to do this in the context of wider transport taxation. To the extent we need cars, it’s much better that they are electric,” he says.

An electric vehicle charging at a charge point on a street in the UK
Long-term champions of pay-per-mile warn of the need to tread carefully. Photograph: nrqemi/Getty Images/iStockphoto

The thinktank the Resolution Foundation recommends a charge based on miles driven and weight be brought in only for future EV sales, as part of VED.

Tanya Sinclair, the chief executive of the industry group Electric Vehicles UK, agrees that motoring taxes need fundamental reform – but the government needs to be absolutely clear it wants people to switch to EVs. “Anything that muddies that message – such as giving a grant with one hand and introducing pay-per-mile with the other – undermines that clarity for the consumer,” she says.

A government spokesperson says it would “look at further support measures” for EVs, but adds: “Fuel duty covers petrol and diesel, but there’s no equivalent for electric vehicles. We want a fairer system for all drivers whilst backing the transition to electric vehicles.”

For Gooding “the best time to introduce road pricing would have been a while ago – but politics has been an interesting place”. The cross-party transport select committee recommended the urgent introduction of road pricing – replacing all motoring taxes for every type of vehicle – in 2022. But no ministers have fancied it; London’s mayor, Sadiq Khan, battered by anti-Ulez hostility, last year had to rule out even considering a pricing scheme , despite once talking of it as the sensible option .

Piloting a new policy is, Gooding says, best done “with the smallest number you can get away with – and if it’s only EVs, that’s better than trying to introduce some kind of complicated charge for the 34m cars we’ve already got”.

For some, including Buckley and the Campaign for Better Transport, an obvious, if also politically contentious, answer remains: end the 15-year freeze on fuel duty and the temporary 5p cut in place since 2022 .

Had the levy stayed the same in real terms, almost £150bn would have accrued to the public purse, according to SMF. Whatever pay-per-mile scheme evolves, Reeves “must ensure operating taxes on EVs remain lower than on petrol”, it says. “The simplest way to maintain that difference is to raise fuel duty.”

Superman copy found in mum's attic is most valuable comic ever at $9.12M

Hacker News
www.bbc.com
2025-11-22 05:21:53
Comments...
Original Article

While cleaning out their late mother's California attic last Christmas, three brothers made a life-changing discovery under a pile of faded newspapers: one of the first Superman comics ever made.

An original copy of the June 1939 first edition on the Man of Steel's adventures, it was in a remarkably pristine condition.

Now it has become the highest-priced comic book ever sold, fetching $9.12m (£7m) at auction.

Texas-based Heritage Auctions, which hosted Thursday's sale, called it the "pinnacle of comic collecting".

The brothers found six comic books, including Superman #1, in the attic underneath a stack of newspapers inside a cardboard box and surrounded by cobwebs in 2024, Heritage said in a press release.

They waited a few months before contacting the auction house, but once they did, Heritage Auctions vice-president Lon Allen visited them in San Francisco within days, according to the auction house.

The brothers, who have chosen to withhold their names, are "in their 50s and 60s, and their mom had always told them she had an expensive comics collection but never showed them", Mr Allen said in Heritage's press release.

"It's a twist on the old 'Mom threw away my comics' story."

Their mother had held on to the comic books since she and her brother bought them between the Great Depression and the beginning of World War Two, Heritage said.

Mr Allen added that the cool northern California climate was perfect for preserving old paper.

"If it had been in an attic here in Texas, it would have been ruined," he said.

That helped CGC, a large third-party comics grading service, give this copy of Superman #1 a 9.0 rating on a 10-point scale, topping the previous record of 8.5.

And at its sale price of over $9m, including buyer's premium, Superman #1 easily beat the previous highest-priced comic book ever sold by $3m.

Action Comics No. 1, the 1938 work that first introduced Superman, sold for $6m last year.

The youngest brother said in Heritage's press release that the box had remained forgotten in the back of attic.

"As the years unfolded, life brought about a series of losses and changes," he said. "The demands of everyday survival took centre stage, and the box of comics, once set aside with care and intention, was forgotten. Until last Christmas."

He added: "This isn't simply a story about old paper and ink. This was never just about a collectible.

"This is a testament to memory, family and the unexpected ways the past finds its way back to us."

Spectral rendering, part 3: Spectral vs. RGB

Lobsters
momentsingraphics.de
2025-11-22 04:53:16
Comments...
Original Article

Published 2025-11-20

Now that we have a spectral renderer, it is time to see if it was worth the effort. I hope you agree with my reasoning in part 1 , that RGB rendering does not match the physical reality, but does it really matter? We will now compare RGB rendering to spectral rendering using my spectral path tracer . In the process, we will look at various types of light sources. RGB rendering simply multiplies RGB triples of illuminants and textures component-wise to arrive at the linear RGB color displayed on screen. Spectral rendering uses proper illuminant spectra (mostly from LSPDD.org ) and reflectance spectra that are upsampled from sRGB using Fourier sRGB .

Throughout this blog post, I will be using figures that allow you to compare results in an interactive fashion. Clicking on the tabs or pressing number keys 1, 2, 3 selects an image to display (RGB rendering, spectral rendering or the illuminant spectrum). The mouse wheel lets you zoom in/out and you can pan by clicking and dragging. At the bottom you see magnified insets of all three images for the cursor position (I am well aware that this part is not useful for the spectrum).

Illuminant E and D65

In Figure 1 we use illuminant E, which is just a constant spectrum. Based on how we have constructed the Fourier sRGB lookup table , we expect RGB rendering and spectral rendering to give nearly the same result here. The reason for that is that we simply used the CIE XYZ color matching functions without multiplying by an illuminant. Thus, our illuminant is implicitly constant (i.e. illuminant E) and we expect that directly reflected illuminant-E light will have exactly the sRGB color from the original texture. The only reason why the results between RGB or spectral rendering might differ slightly is that we account for indirect illumination in these renderings. Indeed, the results are nearly indistinguishable.

Figure 1: Our test scene under illuminant E. RGB and spectral rendering produce nearly identical results.

Some might argue that we should have used illuminant D65 instead of illuminant E to construct the lookup table, because that has been defined to be white light. Most spectral upsampling techniques in the literature do not do so. The best argument for this design decision that I have heard is that sRGB (255, 255, 255) should correspond to a constant reflectance spectrum with value one. Anything else would be counterintuitive: Reflection on perfect white surfaces would change the color of incoming light. If a non-constant illuminant is baked into the upsampling technique, this criterion may be violated.

Figure 2 shows results for illuminant D65. Indeed, there are visible differences in the colors, especially in the reds. Though, they are relatively minor.

Figure 2: Our test scene under illuminant D65. There are minor but visible differences between RGB and spectral rendering.

Neither of these two results is inherently better or more correct than the other. RGB rendering relies on the rather unphysical notion that all light has one of only three wavelengths. Strictly speaking, even this interpretation is not entirely compatible with how the sRGB color space is defined since its primaries are not monochromatic spectra. Though, under the set of definitions used to define the scene in RGB rendering, the result of RGB rendering is (tautologically) correct.

For the spectral renderer, we utilize a more complete representation of the illuminant in the form of an illuminant spectrum. And we have used Fourier sRGB to come up with smooth reflectance spectra that match our surface sRGB colors. Thus, the renderer models the physical reality more accurately, but that does not automatically mean that we get a more accurate rendition of the scene. The only information that we have about the reflectance spectra is their sRGB color, so the upsampled reflectance spectra may not match those that the material intended by the artist (or the captured sample) would have. We get an interpretation that is compatible with everything we know about the scene and with physical light transport, but nothing more than that.

Other smooth illuminant spectra

In Figure 3 , we use the spectrum of a warm-white light-emitting diode (LED). Like most LEDs, this one has a relatively smooth illuminant spectrum composed of two lobes that mostly cover visible wavelengths (hence the good energy efficiency). Once again, the most visible differences are in the reds. While spectral rendering made them slightly brighter for illuminant D65, this time it is making them darker. Greens also shift a bit, but overall the differences are still relatively minor. One of the design goals for white illuminants is good color rendering , i.e. making sure that colors look like they would under illuminant D65. LEDs generally perform relatively well in this regard, so it is unsurprising that the differences are minor here.

Figure 3: Our test scene under an LED spectrum. All colors shift a bit, especially the reds and greens.

Figure 4 uses an incandescent spectrum, i.e. what you would get out of an old light bulb with a filament that heats up. In the visible spectrum, the incandescent illuminant spectrum is essentially a smooth ramp that keeps going up as you move into infrared. In other words, these light sources mostly emit invisible lights, which is why they are so inefficient that they have been prohibited in most countries . Here, the differences between RGB and spectral rendering start to become more interesting. With RGB rendering, blue becomes quite saturated and slightly green, while bright red turns slightly orange. With spectral rendering, blue surfaces become less saturated and get a yellow tint from the illuminant. RGB rendering is not capable of reducing the saturation of colors in this manner. With spectral rendering, illuminant spectra with strong maxima around certain colors will always do so.

Figure 4: Our test scene under an incandescent spectrum. Compared to RGB rendering, spectral rendering desaturates surface colors somewhat.

Spiky illuminant spectra

Now we move on to illuminant spectra with many distinct peaks. The most common examples are coated fluorescent lamps (CFL), as shown in Figure 5 . The highly saturated balloons become a bit brighter with spectral rendering, especially the green ones. Blues and reds also shift a bit, but overall, the changes here are not that big.

Figure 5: Our test scene under a CFL spectrum. This time, the greens are particularly affected.

Metal halide (MH) lamps can have even more spiky spectra as shown in Figure 5 . While the light of this lamp is relatively close to being white, its effect on surface colors in the spectral renderer differs drastically from that of other white illuminants like D65 or the LED considered above. Many saturated surface colors get a blue tint and become much less saturated. The choice of a different white illuminant has drastically altered the look of this scene in the spectral renderer. The RGB renderer reduces the illuminant spectrum to RGB before using it. Therefore, white is white and surfaces keep their colors.

Figure 6: Our test scene under a MH spectrum. With spectral rendering, the saturation of saturated surfaces is drastically reduced.

(Nearly) monochromatic spectra

The most drastic differences between RGB rendering and spectral rendering can be observed when the illuminants are nearly monochromatic, i.e. they emit most of their light close to one specific wavelength. For example, low-pressure sodium-vapor lamps emit almost all of their light near a wavelength of \(589~\mathrm{nm}\). They are commonly used as street lamps, since they have been available since the 1920s and have an efficiency that rivals modern LEDs. High-pressure sodium (HPS) lamps have slightly broader spectra and Figure 7 uses one of those (since we have another fully monochromatic example below). Light never changes its wavelength in the spectral renderer (since we do not model fluorescence). Thus, the nearly monochromatic light of the HPS lamps stays nearly monochromatic as it scatters throughout the scene. No matter what color a surface has, the reflected light will have a color very close to the incoming light, just of different brightness. In the RGB renderer, we just treat this light as a mix of red and green and thus surface colors are not overridden like that. We would be able to get such an effect for red light, green light or blue light, but not for a mixture of those. In this way, RGB rendering hands out special treatment for these three light colors, whereas a spectral renderer can deal with monochromatic light of any wavelength.

Figure 7: Our test scene under a HPS spectrum. With spectral rendering, the color of the light overrides the color of surfaces almost completely.

To drive this point home Figure 8 , uses perfectly monochromatic light at \(500~\mathrm{nm}\). Now the spectral rendering is perfectly monochromatic. Different pixels only differ in their overall brightness. The RGB renderer treats this illuminant as a mixture of blue and green and thus surface colors are retained to some extent.

Figure 8: Our test scene under monochromatic light at \(500~\mathrm{\mathrm{nm}}\). With spectral rendering, colors only differ by their brightness.

Gamut compression

Actually, it is not quite right to say that the monochromatic light at \(500~\mathrm{\mathrm{nm}}\) is a mixture of green and blue. It mixes positive amounts of green and blue and a negative amount of red. Its RGB representation is \((-1,1,0.36)\). A color with negative entries like that is called out of gamut. The sRGB gamut is relatively small, so all non-primary colors that are a bit more saturated will be out of gamut. And monochromatic spectra are the most saturated spectra that you can possibly have.

That poses a challenge for both RGB and spectral rendering, but for RGB rendering, it is more pronounced. The spectral renderer is mostly oblivious to RGB color spaces. It just models realistic light transport and estimates the spectrum of light that reaches the camera. RGB color spaces only come into play for storing the end result and displaying that on a screen. To display an RGB color that may be out of gamut on a screen, gamut compression should be used and there is an ACES standard for that. I considered implementing that, but there are limits to how much work I want to put into a blog post. For the results shown here, I simply clamped RGB vectors to the interval \([0,1]\) before converting to sRGB. Thus, you should keep in mind that some of the images above have an “invisible third channel” with negative values.

Conclusions

In spite of being extremely wide-spread, RGB rendering clashes quite badly with principles of physically-based rendering. While we invest a lot of effort to model or measure various materials or light sources, we simultaneously put an overly simplistic model of color at the foundation of it all. RGB color spaces are designed to be used for display devices and they are perfectly fine for this purpose. However, they are not meant to be used to simulate light scattering and doing so in RGB requires assumptions that are far from being physically-based.

Spectral rendering enables accurate color reproduction under all sorts of illuminants. While the primary colors red, green and blue play a special role in RGB rendering, spectral rendering can handle monochromatic illuminants of all colors and arbitrary mixtures thereof. That opens up new possibilities in lighting design, where the spectra of light sources can influence the colors of surfaces in ways that would not be possible with RGB rendering. It also makes it much easier to reproduce colors seen in real scenes: A major advantage of spectral rendering in production rendering is that it becomes easier to combine rendered and real footage, e.g. for virtual makeup. And it nicely decouples aspects of the camera such as its spectral sensitivity curves or color space from all aspects of light transport.

In real-time rendering, many of the effects mentioned above would typically be faked using per-scene color grading. That works reasonably well if the lighting is dominated by a single type of light sources, but when there is a mixture of different light sources, this approach quickly hits a wall. Spectral rendering handles these situations accurately (although importance sampling may be a challenge).

With this blog post series, I am hoping to combat some misconceptions about RGB and spectral rendering, but I have only scratched the surface. The SIGGRAPH 2021 course on the subject [Weidlich21] is recommended further reading and makes similar points. There are many benefits of spectral rendering that I have not mentioned: Metamerism can be handled and modeling effects such as fluorescence or dispersion becomes more natural. My main point here is that spectral rendering is more principled, enables accurate color reproduction and is affordable right now, even in real-time rendering.

The current reality is that even offline rendering is usually based on RGB. The most notable exception is the work of Wētā FX . They have been using spectral rendering throughout their pipeline for years and ever since Avatar: The Way of Water, they have relied on my method for spectral upsampling of reflectance. They also indicated that they would probably use this technique for all spectra if they were to start from scratch now [Weidlich21] . So while this blog post series has only covered one of many approaches, it is one that has proven itself in production rendering. And I have demonstrated that its overhead is low enough, even for real-time rendering.

References

Weidlich, Andrea and Forsythe, Alex and Dyer, Scott and Mansencal, Thomas and Hanika, Johannes and Wilkie, Alexander and Emrose, Luke and Langlands, Anders (2021). Spectral imaging in production. In ACM SIGGRAPH 2021 Courses. Official version | Author's version

Openring-rs: a webring for static site generators written in Rust

Hacker News
github.com
2025-11-22 04:12:48
Comments...
Original Article


openring-rs

A tool for generating a webring from Atom/RSS feeds.

Build Status Version License

openring-rs is a tool for generating a webring from Atom/RSS feeds, so you can populate a template with articles from those feeds and embed them in your own blog. An example template is provided in in.html .

This is a Rust-port of Drew DeVault's openring , with the primary differences being:

  • we respect throttling and send conditional requests when using --cache (recommended!)
  • the template is written using Tera and is provided as an argument, not read from stdin
  • we show a little progress bar
  • we fetch all feeds concurrently
  • we provide better error messages (via miette )
  • we allow filtering feeds with --before

Demo

To see this in action, you can look at the footer of this blog post.

https://luke.hsiao.dev/blog/openring-rs/

Install

cargo install --locked openring

Usage

A webring for static site generators written in Rust

Usage: openring [OPTIONS] --template-file <FILE>

Options:
  -n, --num-articles <NUM_ARTICLES>    Total number of articles to fetch [default: 3]
  -p, --per-source <PER_SOURCE>        Number of most recent articles to get from each feed [default: 1]
  -S, --url-file <FILE>                File with URLs of Atom/RSS feeds to read (one URL per line, lines starting with '#' or "//" are ignored)
  -t, --template-file <FILE>           Tera template file
  -s, --url <URL>                      A single URL to consider (can be repeated to specify multiple)
  -b, --before <BEFORE>                Only include articles before this date (in YYYY-MM-DD format)
  -c, --cache                          Use request cache stored on disk at `.openringcache`
      --max-cache-age <MAX_CACHE_AGE>  Discard all cached requests older than this duration [default: 14d]
  -v, --verbose...                     Increase logging verbosity
  -q, --quiet...                       Decrease logging verbosity
  -h, --help                           Print help (see more with '--help')
  -V, --version                        Print version

Using Tera templates

The templates supported by openring-rs are written using Tera . Please refer to the Tera documentation for details.

Why a Rust port?

Just for fun.

Moss Survives 9 Months in Space Vacuum

Hacker News
scienceclock.com
2025-11-22 03:57:29
Comments...
Original Article

Mosses are already known for coping with harsh radiation, dehydration, and long freezes . Now scientists have pushed them even further by exposing their spore capsules to open space for nine months, and most of them survived.

The team worked with spreading earthmoss ( Physcomitrium patens ), a small moss species used widely as a plant model by researchers. Its spore-containing capsules were mounted on the outside of the International Space Station (ISS), where they experienced direct solar radiation, vacuum conditions, and sharp temperature swings during each orbit.

Under those conditions, cells usually break down quickly. So the researchers were surprised by what came back. “We expected almost zero survival, but the result was the opposite,” says Hokkaido University biologist Tomomichi Fujita. More than 80 percent of the spores still germinated once they returned to Earth.

Also Read: Microbe That Could Turn Martian Dust into Oxygen

The team detected a small drop in chlorophyll a, but the other pigments remained stable. The spores grew normally in follow-up tests, showing no signs of major stress from their time in orbit.

This kind of toughness fits with the evolutionary history of mosses. Bryophytes — the group that includes mosses, liverworts, and hornworts — were among the first plants to move from water onto land about 500 million years ago. Their spores had to withstand drying and direct sunlight long before soils existed, which may explain why their protective structures still hold up so well today.

Low-Res_Germinated moss spores after space exposure CREDIT Dr. Chang-hyun Maeng and Maika Kobayashi
Germinated moss spores after their time in open space (Image: Dr. Chang-hyun Maeng and Maika Kobayashi)

The results place moss spores alongside the few organisms known to tolerate direct space exposure, including tardigrades and certain microbes. Their survival also adds to ongoing discussions about what types of life might endure extreme environments beyond Earth.

According to the researchers, this durability could matter for future experiments on the Moon or Mars. Mosses need very little soil and can pull nutrients directly from rock, making them candidates for early ecosystem tests in extraterrestrial settings.

“Ultimately, we hope this work opens a new frontier toward constructing ecosystems in extraterrestrial environments such as the Moon and Mars,” says Fujita. “I hope that our moss research will serve as a starting point.”

The research was published in iScience . Read the study here .


How to use views for debugging tools to save time

Lobsters
chrispenner.ca
2025-11-22 03:34:48
Comments...
Original Article

This one will be quick.

Imagine this, you get a report from your bug tracker:

Sophie got an error when viewing the diff after her most recent push to her contribution to the @unison/cloud project on Unison Share

(BTW, contributions are like pull requests, but for Unison code)

Okay, this is great, we have something to start with, let's go look up that contribution and see if any of the data there is suspicious.

Uhhh, okay, I know the error is related to one of Sophie's contributions, but how do I actually find it ?

I know Sophie's username from the bug report, that helps, but I don't know which project she was working on, or what the contribution ID is, which branches are involved, etc. Okay no problem, our data is relational, so I can dive in and figure it out with a query:

> SELECT 
  contribution.* 
  FROM contributions AS contribution
  JOIN projects AS project 
    ON contribution.project_id = project.id
  JOIN users AS unison_user 
    ON project.owner = unison_user.id
  JOIN users AS contribution_author 
    ON contribution.author_id = contribution_author.id
  JOIN branches AS source_branch 
    ON contribution.source_branch = source_branch.id
  WHERE contribution_author.username = 'sophie'
    AND project.name = 'cloud'
    AND unison_user.username = 'unison'
  ORDER BY source_branch.updated_at DESC

-[ RECORD 1 ]--------+----------------------------------------------------
id                   | C-4567
project_id           | P-9999
contribution_number  | 21
title                | Fix bug
description          | Prevent the app from deleting the User's hard drive
status               | open
source_branch        | B-1111
target_branch        | B-2222
created_at           | 2025-05-28 13:06:09.532103+00
updated_at           | 2025-05-28 13:54:23.954913+00
author_id            | U-1234

It's not the worst query I've ever had to write out, but if you're doing this a couple times a day on a couple different tables, writing out the joins gets pretty old real fast . Especially so if you're writing it in a CLI interface where's it's a royal pain to edit the middle of a query.

Even after we get the data we get a very ID heavy view of what's going on, what's the actual project name? What are the branch names? Etc.

We can solve both of these problems by writing a bunch of joins ONCE by creating a debugging view over the table we're interested in. Something like this:

CREATE VIEW debug_contributions AS
SELECT 
  contribution.id AS contribution_id,
  contribution.project_id,
  contribution.contribution_number,
  contribution.title,
  contribution.description,
  contribution.status,
  contribution.source_branch as source_branch_id,
  source_branch.name AS source_branch_name,
  source_branch.updated_at AS source_branch_updated_at,
  contribution.target_branch as target_branch_id,
  target_branch.name AS target_branch_name,
  target_branch.updated_at AS target_branch_updated_at,
  contribution.created_at,
  contribution.updated_at,
  contribution.author_id,
  author.username AS author_username,
  author.display_name AS author_name,
  project.name AS project_name,
  '@'|| project_owner.username || '/' || project.name AS project_shorthand,
  project.owner AS project_owner_id,
  project_owner.username AS project_owner_username
FROM contributions AS contribution
JOIN projects AS project ON contribution.project_id = project.id
JOIN users AS author ON contribution.author_id = author.id
JOIN users AS project_owner ON project.owner = project_owner.id
JOIN branches AS source_branch ON contribution.source_branch = source_branch.id
JOIN branches AS target_branch ON contribution.target_branch = target_branch.id;

Okay, that's a lot to write out at once, but we never need to write that again. Now if we need to answer the same question we did above we do:

SELECT * from debug_contributions 
  WHERE author.username = 'sophie'
    AND project_shorthand = '@unison/cloud'
    ORDER BY source_branch_updated_at DESC;

Which is considerably easier on both my brain and my fingers. I also get all the information I could possibly want in the result!

You can craft one of these debug tables for whatever your needs are for each and every table you work with, and since it's just a view, it's trivial to update or delete, and doesn't take any space in the DB itself.

Obviously querying over project_shorthand = '@unison/cloud' isn't going to be able to use an index, so isn't going to be the most performant query; but these are one off queries, so it's not a concern (to me at least). If you care about that sort of thing you can leave out the computed columns so you won't have to worry about that.

Anyways, that's it, that's the whole trick. Go make some debugging views and save your future self some time.

Hopefully you learned something 🤞! If you did, please consider joining my Patreon to keep up with my projects, or check out my book: It teaches the principles of using optics in Haskell and other functional programming languages and takes you all the way from an beginner to wizard in all types of optics! You can get it here . Every sale helps me justify more time writing blog posts like this one and helps me to continue writing educational functional programming content. Cheers!

What Now? Handling Errors in Large Systems

Lobsters
brooker.co.za
2025-11-22 03:22:45
Comments...
Original Article

More options means more choices.

Cloudflare’s deep postmortem for their November 18 outage triggered a ton of online chatter about error handling, caused by a single line in the postmortem:

.unwrap()

If you’re not familiar with Rust, you need to know about Result , a kind of struct that can contain either a succesful result, or an error. unwrap says basically “return the successful results if their is one, otherwise crash the program” 1 . You can think of it like an assert .

There’s a ton of debate about whether assert s are good in production 2 , but most are missing the point. Quite simply, this isn’t a question about a single program. It’s not a local property. Whether assert s are appropriate for a given component is a global property of the system, and the way it handles data.

Let’s play a little error handling game. Click the ✅ if you think crashing the process or server is appropriate, and the ❌ if you don’t. Then you’ll see my vote and justification.

  • One of ten web servers behind a load balancer encounters uncorrectable memory errors, and takes itself out of service.
  • One of ten multi-threaded application servers behind a load balancer encounters a null pointer in business logic while processing a customer request.
  • One database replica receives a logical replication record from the primary that it doesn't know how to process
  • One web server receives a global configuration file from the control plane that appears malformed.
  • One web server fails to write its log file because of a full disk.

If you don’t want to play, and just see my answers, click here: .

There are three unifying principles behind my answers here.

Are failures correlated? If the decision is a local one that’s highly likely to be uncorrelated between machines, then crashing is the cleanest thing to do. Crashing has the advantage of reducing the complexity of the system, by removing the working in degraded mode state. On the other hand, if failures can be correlated (including by adversarial user behavior), its best to design the system to reject the cause of the errors and continue.

Can they be handled at a higher layer? This is where you need to understand your architecture. Traditional web service architectures can handle low rates of errors at a higher layer (e.g. by replacing instances or containers as they fail load balancer health checks using AWS Autoscaling ), but can’t handle high rates of crashes (because they are limited in how quickly instances or containers can be replaced). Fine-grained architectures, starting with Lambda-style serverless all the way to Erlang’s approach, are designed to handle higher rates of errors, and crashing rather the continuing is appropriate in more cases.

Is it possible to meaningfully continue? This is where you need to understand your business logic. In most cases with configuration, and some cases with data, its possible to continue with the last-known good version. This adds complexity, by introducing the behavior mode of running with that version, but that complexity may be worth the additional resilience. On the other hand, in a database that handles updates via operations (e.g. x = x + 1 ) or conditional operations ( if x == 1 then y = y + x ) then continuing after skipping some records could cause arbitrary state corruption. In the latter case, the system must be designed (including its operational practices) to ensure the invariant that replicas only get records they understand. These kinds of invariants make the system less resilient, but are needed to avoid state divergence.

The bottom line is that error handling in systems isn’t a local property. The right way to handle errors is a global property of the system, and error handling needs to be built into the system from the beginning.

Getting this right is hard, and that’s where blast radius reduction techniques like cell-based architectures, independent regions, and shuffle sharding come in. Blast radius reduction means that if you do the wrong thing you affect less than all your traffic - ideally a small percentage of traffic. Blast radius reduction is humility in the face of complexity.

Footnotes

  1. Yes, I know a panic isn’t necessarily a crash , but it’s close enough for our purposes here. If you’d like to explain the difference to me, feel free.
  2. And a ton of debate about whether Rust helped here. I think Rust does two things very well in this case: it makes the unwrap case explicit in the code (the programmer can see that this line has “succeed or die behavior”, entirely locally on this one line of code), and prevents action-at-a-distance behavior (which silently continuing with a NULL pointer could cause). What Rust doesn’t do perfectly here is make this explicit enough. Some suggested that unwrap should be called or_panic , which I like. Others suggested lints like clippy should be more explicit about requiring unwrap to come with some justification, which may be helpful in some code bases. Overall, I’d rather be writing Rust than C here.

Friday Nite Videos | November 21, 2025

Portside
portside.org
2025-11-22 03:21:08
Friday Nite Videos | November 21, 2025 barry Fri, 11/21/2025 - 22:21 ...
Original Article

Friday Nite Videos | November 21, 2025

Kimmel Wraps Up Trump's Most Horrible Week. How To Spot a Fascist | Song Parody. Darializa Avila Chevalier for Congress in NY-13. Deep Dive on Trump and Bannon in the Epstein Files. Tech Billionaires' Shocking Plan For West Virginia.

Portside Portside

Microsoft Will Preload Windows 11 File Explorer to Fix Bad Performance

Hacker News
blogs.windows.com
2025-11-22 02:38:42
Comments...
Original Article

Written By published November 21, 2025

Hello Windows Insiders, today we are releasing Windows 11 Insider Preview Build 26220.7271 (KB5070307) to the Dev & Beta Channels.

As a reminder we are offering the same builds to both the Dev & Beta Channels on Windows 11, version 25H2.

If you are an Insider in the Dev Channel, you now have a window to switch from the Dev Channel to the Beta Channel if you would like. This window will only be open for as long as we’re releasing the same 25H2-based updates across both the Dev and Beta Channels. After we move Dev Channel forward to a higher build number, the opportunity to switch between these channels will close. When the Dev Channel jumps ahead, things might not be as stable as the Dev Channel is today, so we highly encourage you to evaluate which channel you would like to be in during the time in which the window to switch is open.

Changes in Dev & Beta Channel builds and updates are documented in two buckets: new features, improvements, and fixes that are being gradually rolled out for Insiders who have turned on the toggle to get the latest updates as they are available (via Settings > Windows Update*) and then new features, improvements, and fixes rolling out to everyone in the Dev & Beta Channels. For more information, see the Reminders section at the bottom of this blog post.

Introducing the Xbox full screen experience for PC

Alongside today’s general availability of the Xbox full screen experience (FSE) on more Windows handhelds, today’s preview to Windows Insiders also expands availability of FSE to additional Windows 11 PC form factors .

Designed with console-style navigation in mind, the Xbox full screen experience delivers a clean, distraction-free interface for controller-first gaming. Pair a controller to your PC for smooth task switching and a streamlined gaming experience on your desktop, laptop, or tablet.

UI showing the Task View in Xbox full screen experience on PC. 
UI showing the Task View in Xbox full screen experience on PC.

How to enter Xbox full screen experience:

You can access Xbox full screen experience from Task View, Game Bar settings , or use Win + F11 hotkey to toggle FSE.

UI of the task view interface showing the option to access Xbox full screen experience.
UI of the task view interface showing the option to access Xbox full screen experience.

The Xbox full screen experience begins as a gradual rollout to Windows Insiders on the Dev & Beta Channels, who are also registered Xbox Insiders. We expect to expand this later to all Insiders on the Dev & Beta Channels without requiring Xbox program registration. If you want to be among the first to try out these new features on your PC, join the Xbox Insiders Program and opt into the PC gaming preview through the Xbox Insiders Hub.

For more information about the Xbox full screen experience, visit Full screen experience expands to more Windows 11 PC form factors .

Feedback: Share your thoughts in Feedback Hub (WIN + F) under Gaming and Xbox > Gaming Full Screen Experience .

New features gradually being rolled out with toggle on*

Point-in-time restore for Windows

We’re excited to introduce point-in-time restore for Windows, now available to Insiders in the Beta and Dev Channels! This flexible recovery feature empowers you to quickly roll your device back to a previous state—helping minimize downtime and simplify troubleshooting when disruptions strike. Whether you’re dealing with a widespread outage or a one-off issue, point-in-time restore helps recover your system (including apps, settings, and user files) to get you back to productivity faster. For more details, check out our documentation .

Point-in-time restore shown in the Troubleshoot menu for Windows RE.
Point-in-time restore shown in the Troubleshoot menu for Windows RE.

Feedback: Share your thoughts in Feedback Hub (WIN + F) under Recovery and Uninstall > Point-in-time restore .

Introducing Fluid Dictation in Voice Typing

Following the introduction of Fluid dictation for voice access users, we’re also now introducing it for voice typing users on NPU devices. Fluid dictation makes voice-based dictation smoother and smarter by automatically correcting grammar, punctuation, and filler words as you speak, reducing the need for manual editing. Powered by on-device small language models (SLMs), it ensures fast and private processing.

To use it, set focus to a text field and launch voice typing by pressing the Windows key plus H and complete setup if you’re a first-time user. Fluid Dictation is enabled by default—you can check or toggle it via the settings flyout—so all you need to do is start talking!

UI showing the Fluid Dictation toggle in the voice typing launcher.
UI showing the Fluid Dictation toggle in the voice typing launcher.

Feedback: Share your thoughts in Feedback Hub (WIN + F) under Input and Language > Voice Typing (Windows key plus H).

Changes and Improvements gradually being rolled out with toggle on*

[Seamlessly resume more apps from your Android phone to your PC]

  • Following the ability to resume Spotify tracks from your phone onto your PC, we’re excited to share that:
    • vivo Android phone users can also now continue your browsing activity from vivo Browser on your phone, onto your default browser on your PC.
    • Honor, Huawei, Oppo, Samsung and vivo Android phone users can also now continue online files opened on M365 Copilot app from your phone onto your PC. Word, Excel, and PowerPoint files will open in the respective app on your PC if you have it installed, or if you don’t they’ll open in the default browser on your PC. Note – offline files (stored locally on the phone) are not currently supported.

FEEDBACK: Please file feedback in Feedback Hub (WIN + F) under Devices and Drivers > Linked Phone.

[Click to Do]

  • We’re testing and refining the Click-to-Do top bar to determine the best experience for future updates. Functionality will vary by device and market.

[File Explorer]

  • We’re making a few refinements to the context menu aimed at reducing the space taken by less commonly used actions, while keeping them easy to access. We’ve also updated the ordering of actions to group similar tasks. This includes:
    • We’ve moved Compress to ZIP file, Copy as Path, Set as Desktop Background, and Rotate Right, and Rotate Left into a new Manage file flyout.
    • We’ve moved cloud provider options, like Always Keep on this Device and Free Up Space, into their relevant cloud provider flyout.
    • We’ve moved Send to My Phone next to the cloud provider options.
    • We’ve moved Open Folder Location to now be next to Open and Open with.
The image on the left shows the “before” experience before and the image on the right shows the “after” experience.
The image on the left shows the “before” experience before and the image on the right shows the “after” experience.

Note, the name Manage file may change in a future Insider update. If you have feedback, please file it in the Feedback Hub under Desktop Environment > Right-Click Context Menu

  • We’re exploring preloading File Explorer in the background to help improve File Explorer launch performance. This shouldn’t be visible to you, outside of File Explorer hopefully launching faster when you need to use it. If you have the change, if needed there is an option you can uncheck to disable this called “Enable window preloading for faster launch times” in File Explorer’s Folder Options, under View. Looking forward to your feedback! If you do encounter any issues, please file them in the Feedback Hub under Files Folders and Online Storage > File Explorer Performance, or Files Folders and Online Storage > File Explorer.

[Microsoft Store ]

Based on user feedback, we have added support for uninstalling Store-managed apps from the Store’s library page. Simply find an installed app in your library, click the three-dot menu, and click uninstall. Please let us know what you think!

UI showing the uninstall functionality for Store-managed apps from the Store's library page.
UI showing the uninstall functionality for Store-managed apps from the Store’s library page.

Windows Insiders across all channels running Microsoft Store version 22510.1401.x.x and higher will see this improvement.

FEEDBACK: Please file feedback in Feedback Hub (WIN + F) under Microsoft Store.

Fixes gradually being rolled out with toggle on*

[Taskbar and System Tray]

  • Fixed an issue which could cause the taskbar to hang after receiving certain notifications.
  • Fixed an issue where the battery icon in the taskbar might unexpectedly show its own backplate when hovering over the icon in the system tray (instead of combined with wi-fi and volume).

[Internet]

  • Made some underlying improvements to help address an issue which could lead to not having internet after resuming from disconnected standby. Please don’t hesitate to file feedback under Network and Internet in the Feedback Hub if you continue experiencing issues.

[File Explorer]

  • Fixed an issue where if you opened the Recycle bin and had “Empty recycle bin” visible in the command bar, it might stay showing after you navigated away.

[Settings]

  • Fixed an issue where Settings might crash when navigating to Privacy & Security > Camera, Location, or Microphone.

[Display and Graphics]

  • Fixed an issue where recently certain games might show a message saying “Unsupported graphics card detected”, although a supported graphics card was being used.

[ Task Manager]

  • If you’re using Die or CAMM memory form factor, Task Manager will now show that in Performance under Memory > Form Factor, instead of a blank.

[.NET Framework and Visual Studio]

  • The issue causing Insiders with ARM64 PCs to potentially observe crashes with Visual Studio or applications that depend on .NET Framework should be resolved if you have installed the latest .NET Framework update.

Known issues

[Xbox full screen experience for PC]

  • [ NEW ] The virtual keyboard is not shown for controller users on devices without a touch screen. Please use the physical keyboard as a workaround for now.
  • [ NEW ] Some apps may behave unexpectedly when using FSE, particularly those that expect to be locked to a given size or launch additional windows.

[Taskbar & System Tray]

  • We’re investigating an issue which is causing the Start menu to not open for some Insiders on click, although it will open if you press the Windows key. It’s believed this issue may also potentially impact the notification center (which you can open with WIN + N).
  • We’re investigating an issue where for some Insiders apps aren’t showing in the system tray when they should be.

[File Explorer]

  • Scrollbar and footer are missing and showing a white block instead when text is scaled in dark mode version of the copy dialog.
  • [ NEW ] We’re investigating an issue where File Explorer has started showing a white flash when navigating between pages after the previous flight.

[Bluetooth]

  • [ NEW ] We’re investigating an issue causing Bluetooth device battery level to not show for some Insiders.

Reminders for Windows Insiders in the Dev & Beta Channels

  • Updates are based on Windows 11, version 25H2 via an enablement package (Build 26220.xxxx).
  • Many features are rolled out using Controlled Feature Rollout technology , starting with a subset of Insiders and ramping up over time as we monitor feedback to see how they land before pushing them out to everyone in this channel.
  • For Windows Insiders who want to be the first to get features gradually rolled out to you , you can turn ON the toggle to get the latest updates as they are available via Settings > Windows Update*. Over time, we will increase the rollouts of features to everyone with the toggle turned on. Should you keep this toggle off, new features will gradually be rolled out to your PC over time once they are ready.
  • Features and experiences included in these builds may never get released as we try out different concepts and get feedback. Features may change over time, be removed, or replaced and never get released beyond Windows Insiders. Some of these features and experiences could show up in future Windows releases when they’re ready .
  • Some features in active development we preview with Windows Insiders may not be fully localized and localization will happen over time as features are finalized. As you see issues with localization in your language, please report those issues to us via Feedback Hub.
  • Check out Flight Hub for a complete look at what build is in which Insider channel.

Thanks,
Windows Insider Program Team

The Facts About the Military Disobeying Illegal Orders

Portside
portside.org
2025-11-22 02:32:20
The Facts About the Military Disobeying Illegal Orders barry Fri, 11/21/2025 - 21:32 ...
Original Article

WHEN SIX MEMBERS OF CONGRESS released a short video on Tuesday emphatically reminding military personnel 1 that they must not obey illegal orders, the message ricocheted through the political world and the media like a rifle shot. Reactions split along predictable lines. Some saw the video as a necessary civic reminder in a volatile moment. Others attacked it as inappropriate political rhetoric directed at the armed forces. Still others lied about what was said, or mocked the message as condescending. As the controversy escalated, the lawmakers who appeared in the video began receiving death threats, while the president himself suggested—astonishingly—that their message constituted “ sedition ” and that they should be imprisoned or executed .

I want to address a fundamental point revealed by the video and the debate surrounding it: Most Americans do not understand what is in the oaths sworn by our service members. Confusion about that, combined with an understandable desire to keep the military a nonpartisan institution, fuels both the alarm that motivated the video’s creation and the backlash against the video. A clearer understanding on this subject will help reveal the aspects of our constitutional structure that protect the nation from unlawful uses of the military.

Here’s the truth, learned on the first day of service by every enlisted soldier, sailor, airman, Marine, guardian, and coast guardsman, and learned but sometimes not recognized by the young officers who first take the oath: There is not one military oath. There are two. And the differences between them explain exactly who is responsible for refusing illegal orders, why the system was designed that way, and what it means for this moment.

One reason the debate keeps going sideways is that the public keeps talking about “the military” as if it were a single, undifferentiated mass of people with identical obligations. It isn’t. The Constitution and Congress deliberately created two different oaths—one for enlisted personnel, and one for officers. That structure is not bureaucratic trivia; it is grounded on the bedrock American civil–military relations. Ignoring it leads to the misleading assumption that everyone in uniform bears equal responsibility when confronted with an unlawful command.

They don’t. And that distinction matters.

Enlisted members swear to support and defend the Constitution, and to “obey the orders of the President of the United States and the orders of the officers appointed over me, according to regulations and the Uniform Code of Military Justice.” And the UCMJ makes crystal clear that the service member’s obligation is to obey “lawful” orders , and that no enlisted member is permitted to carry out an unlawful order. But the enlisted oath is also intentionally anchored in obedience of the chain of command. The accountability lies one level up.

Which brings us to the officer oath —shorter in words, heavier in weight. Officers swear to “support and defend” the Constitution; to “bear true faith and allegiance” to it; and to “well and faithfully discharge the duties” of their office. They also affirm that they “take this obligation freely, without any mental reservation or purpose of evasion.” What they do not swear to do is equally important: Officers make no promise to obey the president and the officers above them.

That omission is not an oversight. Officers give orders, evaluate legality, and act as the constitutional circuit breakers the Founders intended. They are expected—by law, by professional ethic, and by centuries of tradition—to exercise independent judgment when presented with a questionable directive. Officers are duty-bound to refuse an unlawful order. It is not optional. It is not situational. It is their job.

When the members of Congress in their video urge what seems to be the entire military not to follow illegal orders, they may unintentionally blur the very lines that keep the system functioning. Enlisted personnel obey lawful orders; officers ensure the orders that reach them are lawful. The real constitutional failsafe is not a general broadcast to every rank. It is the officer corps, obligated by oath to the Constitution alone.

This matters in a moment when Americans are hearing loud claims about using the military to solve political disputes, intervene in elections, or take actions beyond statutory authority. People are right to worry. But they should also understand the guardrails already in place. The military has been here before—they have already, at times in our history, faced unlawful pressure, political manipulation, or attempts to turn the armed forces into a tool of personal power.

Also worth remembering: No one in the American military swears allegiance to any individual. The oaths are not pledges of loyalty to a party, a personality, or a political movement. Loyalty is pledged to the Constitution—and officers further take that obligation “without mental reservation,” knowing full well it may someday require them to stand with courage between unlawful authority and the people they serve.

So while pundits and politicians continue fighting over the optics of the lawmakers’ video, the core reality remains: The safeguards are already built into the structure. The oaths already distribute responsibility. The law already forbids what some fear. And the officer corps already knows that they bear the constitutional duty to ensure that unlawful orders never reach the young men and women who follow them, and who, in effect, they also serve.

This is not a moment for panic. It is a moment for clarity.

If Americans understood the difference between the two oaths—one grounded in obedience, the other grounded in constitutional discernment—they would see that the republic’s defenses against unlawful orders are not theoretical. They exist. They function. They don’t depend on the whims of political actors on either side of the aisle, but on the integrity of those who swear to uphold them.

1 The video is directed not only at military service members but also at members of the intelligence community—but in this article, I’m focusing exclusively on the former.

Lt. Gen. Mark Hertling (Ret.) (@MarkHertling) was commander of U.S. Army Europe from 2011 to 2012. He also commanded 1st Armored Division in Germany and Multinational Division-North during the surge in Iraq from 2007 to 2009. @markhertling

You may have noticed that sh*t has gotten weird the last few years. The Bulwark was founded to provide analysis and reporting in defense of America’s liberal democracy. That’s it. That’s the mission. The Bulwark was founded in 2019 by Sarah Longwell, Charlie Sykes, and Bill Kristol.

The death of tech idealism and rise of the homeless in Northern California

Hacker News
lithub.com
2025-11-22 02:28:28
Comments...
Original Article

Fuckers. I couldn’t get the word out of my head, because he wouldn’t stop saying it. I was sitting in the tiled courtyard of the Mediterranean-style home of an old acquaintance, a venture capitalist and serial tech entrepreneur, who lived a few blocks from Zuckerberg in Palo Alto. Next to us was a massive stone slab over which water dribbled into a reflecting pool. A Buddha sat on the stone, contemplating the flow. Above us sprawled the canopy of a century-old olive tree, which had been raining its fruit onto the courtyard.

Article continues after advertisement

It would have been an idyllic scene, were it not for the presence of my acquaintance, who kept smacking and berating his dog, a puffy, pure-white Alaskan-looking thing, who wouldn’t stop eating the olives.

In my previous career, I was a landscape designer, and this person was my client. I’d lived in Santa Cruz then, a hippie-surfer town about an hour away on the other side of the mountains that separate the Valley from the ocean. I was not alone in commuting over those mountains—many of Santa Cruz’s hippies and surfers make the trek to stick their straws where the wealth is. I went to college at the University of California in Santa Cruz—home of the fighting Banana Slugs!—and spent the entirety of my twenties there.

When I drove over after arriving in Cupertino, however, a camp lined the main road into town; hundreds of unhoused residents inhabited another area along the river.

When thirty approached, I began to think of things like owning a home, which even an hour from the Valley’s gravitational center was out of reach with my income at the time. So a few months after the Great Recession hit, I moved back to Georgia, where I’d grown up. I bought a house on seven acres for $90,000.

I’d been away from California for twelve years. Much had changed. The real estate costs I’d fled had tripled; 2008 prices now seem quaintly affordable. I don’t remember ever seeing a tent on the streets of Santa Cruz back then. It was known as a place with a lot of panhandlers and drug users, but not so many that they made their dwellings in places obvious to the casual observer. When I drove over after arriving in Cupertino, however, a camp lined the main road into town; hundreds of unhoused residents inhabited another area along the river.

Article continues after advertisement

My client had also changed. I remembered him as a charming, progressive guy, but he’d grown older, crankier, and more libertarian in the decade since I last saw him. Apparently he’d become fond of calling people fuckers , and when I broached the topic of homelessness, he erupted in a lava flow. Employees of NGOs who concoct idealistic plans to address the housing crisis? Fuckers. Activists who valiantly defend the less fortunate among us? Fuckers. He couldn’t stand to go to San Francisco anymore because of the hordes sleeping and shitting right there on the sidewalk, in front of businesses run by people who actually contribute to the economy.

“If we can figure out how to get a package from China to your doorstep in two days, we can figure this out,” he said. Whether it’s houses made of shipping containers or building artificial islands in the Bay to house the homeless, he assured me that “innovators” like himself could come up with a solution—if only the incompetent, na.ve, and corrupt fuckers in the public sector would get out of the way.

In fact, he would personally love to dig his entrepreneurial hands into the issue. But the poverty space was dominated by inefficient nonprofits and he wouldn’t waste his time consorting with them—the profit motive is what drives efficiency, after all, and efficiency paves the way to viable solutions. Nonprofits are in the business of self-congratulation, not getting things done, he said. His evidence: They hadn’t fixed the problem yet. “It’s like a car or your phone,” he said. “Either it works or it doesn’t.”

The last time I’d seen my client he was plotting his first trip to Burning Man. He’d shown me some of his paintings and we’d chatted about organic farming and his time in a kibbutz. Though he worked sixteen hours a day (he claimed to sleep no more than a few hours a night), he nevertheless found time to feed his omnivorous intellect, which snacked on cybernetics and chewed on transcendentalism after dinner. He was the archetypal boomer tech entrepreneur, kissed by the antiestablishment, but in the business of re-establishing the establishment in his own image.

The Valley overlaps geographically with the hippie homeland of San Francisco, Berkeley, and their environs, and there’s long been cross-pollination, if not orgiastic copulation, between the two spheres. As a barefoot college dropout in the early seventies, Steve Jobs’s interests included Ram Dass, Hare Krishnas, and fruitarianism; his connection to apples stemmed from a stint at the All One Farm, a commune where he worked in a Gravenstein orchard.

Article continues after advertisement

The commune fell apart as residents realized they were being conned by the spiritual leader, a close friend of Jobs, into providing free labor for his apple cider business. The apple cider guru later became a billionaire mining magnate notorious for labor and environmental abuses. Jobs, however, sought to bring his spiritual values with him in founding a company to disseminate what he considered the ultimate tool of enlightenment—the personal computer—to the masses.

This trajectory is such a prominent feature among the Valley’s founding fathers that it has spawned a minor field of academic study. “To pursue the development of individualized, interactive computing technology was to pursue the New Communalist dream of social change,” writes Fred Turner, a Stanford historian, in his book From Counterculture to Cyberculture: Stewart Brand, the Whole Earth Network, and the Rise of Digital Utopianism .

Turner’s book focuses on Stewart Brand, a Merry Prankster turned evangelist of enlightened capitalism, who once roamed from commune to commune selling back-to-the-land supplies out of his 1963 Dodge truck. The Whole Earth Truck Store, as he called it, morphed into the Whole Earth Catalog magazine, which begat the Whole Earth Software Catalog , which begat Wired magazine.

Unhoused communities don’t randomly burble up from the sidewalk. They are born of the housed communities around them, which in the Valley’s case is a particularly curious one.

Brand, writes Turner, “brokered a long-running encounter between San Francisco flower power and the emerging technological hub of Silicon Valley,” in which “counterculturalists and technologists alike joined together to reimagine computers as tools for personal liberation, the building of virtual and decidedly alternative communities, and the exploration of bold new social frontiers.”

One can imagine a young Steve Jobs digging the communalism of today’s Bay Area camps, whose countercultural idealism shares many threads with that of the Valley’s early hippie-nerds—ironic given the bulldozing of camps in the shadows of contemporary tech campuses and their tightly conformist corporate cultures. The commonalities don’t stretch very far: A rather thick thread in the hippie-techie braid is individualism, a whole lot of which hid behind the Me generation’s “New Communalist” movement. The marriage of these Bay Area cultures is alive and well, but today has more of a New Age–Burning Man vibe.

Article continues after advertisement

Brand, now in his eighties, is an ardent Burner. He’s gone from libertarian to the even-harder-to-define “post-libertarian” and is building a five-hundred-foot clock inside a Texas mountain owned by Jeff Bezos, which is designed to tick once per year for ten thousand years. A cuckoo will emerge at the end of each millennium.

Brand shares a certain intellectual hubris with my acquaintance, who asked if I would like to read an eighty-two-page white paper he wrote regarding the human brain and why Darwin’s survival-of-the-fittest theory applies not just to biological evolution, but to the optimization of social structure. I stared at the Buddha and tried to think of a way to change the subject.

Unhoused communities don’t randomly burble up from the sidewalk. They are born of the housed communities around them, which in the Valley’s case is a particularly curious one. The Valley’s valley is wide and smoggy enough that some days you can’t see the mountain ranges that form it. The scorching Diablo Range, where cattle roam oceans of desiccated grass, lies to the east.

On the other side, the lusher Santa Cruz Mountains, a place of dank redwood forests, organic farming communes, and uppity vineyards, form a verdant curtain between the Valley and the ocean. Here the tech elite build their villas and take to the fog-kissed ravines for athleisure-clad recreation.

The valley started to become the Valley in 1943 when IBM opened a factory to manufacture punch cards in San José. At the time, orchards carpeted much of the region. When the trees blossomed in early spring, the honey-scented flowers intoxicated bees and lovers alike. During the late summer harvest, the air was a punch bowl. Maps referred to it then as the Santa Clara Valley, but romantic minds of the day christened it the Valley of Heart’s Delight, after a 1927 poem by a local writer with Wordsworthian sensibilities, named Clara Louise Lawrence.

Article continues after advertisement

No brush can paint the picture
No pen describe the sight
That one can find in April
In “The Valley of Heart’s Delight.”

Cupertino did not exist back then. The Glendenning family farmed the land where the Apple Spaceship now sits. Prunes were their specialty. The farm was on Pruneridge Avenue—the valley was considered the prune capital of the world, supplying 30 percent of the global market—which passed through their orchards near the present location of Steve Jobs Theater, a smaller circular building next to the mothership.

But Apple bought the road from the city—$23,814,257 for a half mile—so you can’t drive through there anymore. Between the steel bars of the fence you can still catch a glimpse of the Glendennings’ old fruit-drying barn, which has been renovated and is now storage for landscaping equipment. The new orchards and the old barn help soften the Pentagon vibe with a little farm-to-table ambience.

The Valley’s valley is not a stereotypical one because it lacks a mighty river meandering between the mountain ranges. Instead, there is the southern leg of San Francisco Bay, a shallow, brackish estuary fed by measly creeks that barely run in the dry season. It’s a bird and crustacean paradise, but the lack of fresh water and ocean currents make for a putrid aroma that’s further intensified by the landfills, wastewater treatment plants, and commercial salt-harvesting operations clustered around the waterfront.

The smell is so intense that it’s spawned a South Bay Odor Stakeholders Group “dedicated to identifying and resolving odor issues.” One finds Reddit threads with titles like South Bay Fucking Smell: “south bay people, you know what i mean. where the fuck is this rancid ass smell coming from. it’s pretty common for it to smell like shit here, i’ve smelled it my whole life, but i just want to know where it’s comin from. My guess is the shitty salty shallow south bay water spewing out smelly air, but idk.”

“That, or else it’s your mom,” replied another user, who referred to the odor as “the ass cloud.” The poetics of the region have shifted since Lawrence’s day.

The military forefathers of the Valley must have been horrified at the hippies their children became, though by the eighties the arc of flower power had bent toward the common ground of Wall Street.

The ass cloud did not dissuade the early tech settlers, who followed the money flowing from the patron saint of the Valley’s venture capitalists: DARPA, the Department of Defense’s secretive research agency, which commissioned much of the basic science from which the IT revolution sprang. While farms like the Glendennings’ continued to pump out prunes on the arable land between the Bay and the mountains, the military-industrial complex set up along the mud flats. The Navy built an eight-acre dirigible hangar in Mountain View, still one of the largest freestanding structures ever erected. The CIA quietly rooted itself among the reeds and spread rhizomatically. During the Cold War, aerospace companies blossomed between DOD installations. Lockheed was the Valley’s biggest employer when Kent and Steve Jobs were growing up in the suburbs that slowly consumed the orchards.

The American tech industry was born in the Bay Area because its defense industry parents came here to ward off the Japanese—during World War II, this was the gateway to the “Pacific Theater,” as the Asian front of the war was euphemistically referred to. This first generation of the Valley “seeded companies that repurposed technologies built for war to everyday life,” writes Margaret O’Mara, a tech industry historian. “Today’s tech giants all contain some defense-industry DNA.”

Jeff Bezos’s grandfather, for instance, was a high-ranking official at the US Atomic Energy Commission and at ARPA, the precursor to DARPA. Jerry Wozniak, father of Apple’s other Steve—Steve “The Woz” Wozniak, the company cofounder and part of the gang tweaking on computers in the Jobs’ garage—was an engineer at Lockheed. The military forefathers of the Valley must have been horrified at the hippies their children became, though by the eighties the arc of flower power had bent toward the common ground of Wall Street.

The Navy’s dirigible hangar still looms over the Bay, but Google now rents the property from the government for the parking of private jets. The company dominates the neighborhood to the west of the hangar, a spread of dull office buildings revolving around the central Googleplex, with its employee swimming pools, volleyball courts, and eighteen cafeterias. There are no houses or apartments in the neighborhood, though there are residential districts—of a sort. These are surprisingly affordable, which means that some of the folks who smear avocado on the techies’ toast and stock the kombucha taps have the good fortune to live nearby.

It’s easy to miss their humble abodes, however. An out-of-towner who gets off at the Google exit to take a leak could be forgiven for thinking they’d stumbled across some sort of RV convention. But those aren’t recreational vehicles lining the backstreets of the Google-burbs—those are homes on wheels.

RVs parked on the side of the road are the new desirable real estate, and like the old industrial cores of American cities that have evolved from roughshod hangouts for unemployed artists to haute loft developments for upwardly mobile professionals, their inhabitants aren’t immune to class stratification. Most of the rigs are older, ramshackle models, but here and there shiny coaches broadcast the relative wealth of their inhabitants—techies who could afford an apartment but don’t want to waste their money on rent.

They roll out of bed, hop on a company bike, and are at the office in three minutes, in the meantime saving up for a big house in the outer, outer, outer burbs, where you can still get a McMansion for under $3 million. Some already have the McMansion and use their RV as a workweek crash pad.

The more-rickety RVs belong to the avocado smearers and lawn mower operators. Crisanto Avenue, five minutes from the Googleplex, is the Latin America of Mountain View’s homes-on-wheels community. It’s like a museum of 1980s RVs—Toyota Escapers, Winnebago Braves, Chevy Lindys, Fleetwood Jamborees—most of them emanating Spanish banter, many with blue tarps over the roof, and some leaking unmentionable juices from onboard septic tanks. Apartments line one side of Crisanto, but the side with the RVs fronts onto train tracks. A shaded strip of earth along the tracks, maybe twelve feet wide, serves as a communal front yard, complete with potted plants and patio furniture, for pets and kids to play.

She’d held America in high esteem before living here. “La vida en los Estados Unidos es terrible,” she said.

An older Peruvian woman named Ida invited me into her RV, where a half-eaten pineapple sat serenely on an otherwise empty table. She used to live in a two-bedroom apartment with sixteen other people—“Fue imposible!” she said—until she learned of the RV scene. She couldn’t afford to purchase one, but there’s a growing industry in the Valley for old-school RV rentals; residents on Crisanto told me they pay between $500 and $1,000 per month, depending on the RV, plus a $75 fee to pump sewage.

Since Ida arrived in the US in 2003, she has worked mainly as a nanny, often for around six dollars per hour. Work was sparse during the pandemic, so she accepted whatever pay she was offered. One family gave her twenty dollars for taking care of their two children for twelve hours. She’d held America in high esteem before living here. “La vida en los Estados Unidos es terrible,” she said.

My visual experience of the Valley began to shift. My eyes had once flashed at views of the water, clever billboards (“Hey Facebook, our planet doesn’t like your climate posts”), and homes with the billowy, buff-colored grasses and scrawny wildflowers that signify the aesthetics of people who can afford expensive landscaping designed to look feral.

But the more time I spent with the Valley’s have-nots, the more my focus became trained on the visual language of the income inequality ecosystem: the camouflage patterns of desiccated vegetation pocked with blue tarps and plastic bags flapping in the branches; the hulking silhouettes of recreational vehicles parked in non-recreational environments; the bodies splayed out on the sidewalk.

Here and there, artistic aberrations emerge in the motif. I met a thirty-year old man named Ariginal who lived with his family and dogs in a 1983 Chevy camper van that he’d hand-painted marine blue with school-bus-yellow trim. A blue neon light mounted to the undercarriage illuminated the street in a cool glow as they motored around in their Scooby-Doo mobile at night. Ariginal went to school to be a fireman but became an Uber driver. He’s also a rapper, fashion model, and inventor—there are a few things he’s hoping to patent, and he wanted to show me the drawings, but his daughter was napping in the van. “I have a lot of dreams,” he said.

Within twelve minutes of meeting Ariginal I learned that he recently “discovered a couple of lumps . . . uh, in my testicles.” They were cancerous. He’d just had the tumors removed and would soon be undergoing radiation to make sure they don’t come back. “Just another obstacle,” he sighed.

“Vanlife has become the norm here,” a veteran gig worker named Chase, who’s driven for Uber, Instacart, and Amazon Flex, told me. He was not talking about hipsters who move into a home on wheels because it sounds like a fun and Instagrammable lifestyle. He was referring to his colleagues who have no other choice.

I found there is significant overlap between the gig work community and the unhoused community. Some full-time gig workers end up living in their vehicles; some camp residents become part-time gig workers because it’s a way to make a buck that doesn’t require a home address or the scrutiny of a human boss, only a working phone. Rudy, for instance, began delivering for food apps—using Lime scooters he rents by the hour—after he became homeless.

Their camps keep getting razed, but like the marshland reeds, they sprout right back.

The mobile communities stretch along the Bay up to Meta’s headquarters at 1 Hacker Way, on the outskirts of Palo Alto. East Palo Alto, the historically Black community surrounding the Meta campus, is one of the least gentrified, most impoverished places in the Valley—a 2017 study found that 42 percent of students in the local school district were homeless. A sixty-acre nature preserve across the street from the Meta campus is home to endangered species such as the salt marsh harvest mouse and Ridgway’s rail, a chicken-sized bird with a long, pointy beak and special glands that allow it to drink salt water.

A local variety of Homo sapiens lives there too, who are endangered in a different sort of way. The authorities want them out because their “presence is compromising the health of the estuary,” according to Palo Alto Weekly . Their poop and trash are considered pollution under the federal Clean Water Act—grounds for eviction. “The welfare of wildlife and the health of Baylands ecosystems is pitted against the very real human needs of people,” the paper opined. Their camps keep getting razed, but like the marshland reeds, they sprout right back.

Different regions of the Valley lend themselves to different expressions of homelessness. In the suburban areas, there are lots of vehicle dwellers because it’s (relatively) easy to find a place to park. In densely developed San Francisco, homelessness is largely centered along sidewalks, pushing the lives of unhoused individuals up close and personal, but keeping camps small and dispersed. Golden Gate Park is a would-be homeless haven, but local authorities have managed to keep camps from proliferating.

In San José, however, local green spaces have been commandeered by the unhoused, with camps that have developed into villages, especially along the Guadalupe River, where the Crash Zone was located, and its tributaries.

San José’s waterways are hideously un-scenic—views are of rubble and trash; the vegetation appears to be in a fight for its life. And although the Guadalupe is called a river, it’s more like a creek that bulges into a torrent on the rare occasion of a multiday rainstorm. Its abused hydrology forms the armature of a shadow city—a new form of urban infrastructure that is unplanned and unwelcome—within a city that does not acknowledge its shadow.

At a community meeting in 2017 to solicit input on a homeless shelter the city wanted to build, a horde of angry residents expressed their discontent over efforts to accommodate the unhoused: “Build a wall,” they chanted.

The Guadalupe River and its camps pierce downtown San José, meandering past the Zoom campus and Adobe’s four-tower headquarters to the Children’s Discovery Museum on Woz Way (as in Apple’s Wozniak, the Apple cofounder), where a community of Latinx campers have chiseled terraced gardens into the riverbank to grow food. People call it the Shelves.

Downstream from the Crash Zone, the Guadalupe flows north along the edge of the airport on its way to the Bay, passing dozens of camps and acres of office parks populated by household names—PayPal, Google, Hewlett-Packard, Roku, Cisco, Intel. One of the biggest camps in this part of town emerged on vacant land owned by Apple, just north of the airport, where the company plans to build yet another campus. Located on Component Drive, it was known as Component Camp.

The Jungle, they said, was “a crime syndicate ruled by gangs, where police do not enter.”

As word spread that displaced Crash Zone residents might soon inundate the place, the company announced they would spend “several million dollars” on a high-end sweep—evicted residents were given vouchers for nine months in a hotel—just weeks before the Crash Zone sweep began.

The Guadalupe’s tributaries tell further stories. Los Gatos Creek leads to the headquarters of eBay and Netflix, as well as Downtown West, a neighborhood being built from the ground up by Google. The city approved a development proposal that included office space for twenty-five thousand folks, but only four thousand units of housing—the sort of job-supply-to-housing-demand ratio that helps put a $3 million sticker on a bungalow.

A report by an economic research firm found that the project’s job-to-housing imbalance would translate to a $765-per month increase for San José renters over a decade—to offset upward pressure on rents, they said more than four times as many units would be needed, a third of them at subsidized rates. The San José Chamber of Commerce declared the report “fundamentally flawed” and dismissed findings like the $765 rent increase. “I don’t think that the stark reality presented in the report is realistic,” a representative told San José Spotlight , “nor something we can expect to happen in the next 8 to 10 years.” In the previous decade, however, median rent in the area had gone up by exactly $763.

Coyote Creek is home to a camp called the Jungle—I never deduced whether this was a reference to hobo jungles or the dense vegetation, or both—on which national media descended in 2014 as it was being swept. It was similar to the Crash Zone in scale, and headlines touting it as the “nation’s largest homeless camp” became a mantra. It was a feast of poverty porn.

“Living in The Jungle means learning to live in fear,” said The Atlantic , quoting a resident who displayed “a machete that he carries up his sleeve at night.” For the UK’s Daily Mail , it was an opportunity to get Dickensian. “Dilapidated, muddy and squalid though it was, it was all they had to call home—a shantytown in the heart of Silicon Valley,” the reporter lamented. “In the last month, one resident tried to strangle another with a cord of wire and another was nearly beaten to death with a hammer.” The Jungle, they said, was “a crime syndicate ruled by gangs, where police do not enter.”

The New York Times was more restrained, striking a valiant tone, with a photo of the mayor himself helping a resident to “wheel his belongings up a muddy embankment.” The local CBS station reported that displaced residents immediately formed a “New Jungle” a half mile away. Before long, they recolonized the original Jungle.

The Crash Zone had grown to the size of the original Jungle, if not larger, by the time I first visited. The fields outside the airport fence technically belong to a city park, but when driving by they appeared like a cross between a junkyard and a refugee camp, in which explosives were periodically detonated. RVs in various states of disrepair butted up to tarp compounds that overflowed with debris, from bottles and cans to appliances and vehicles.

This visual buffet was a mix of freshly prepared, putrefied, and charred—one resident had a tidily landscaped yard with a pair of pink plastic flamingos, while other homesteads were a mix of rotting garbage, blackened grass, melted tarps, and burnt-out vehicles. My eyes flowed over suitcases and furniture and school buses to unexpected items, such as a piano, several boats, and a limousine. The first residents I met cautioned me against wandering into certain sections, where I might be mistaken for an undercover cop—the confetti labyrinth of structures left many a hidden nook where bad things might happen.

One guy had cobbled together a two-story wood cabin; around it were huge piles of wood chips and logs. I wanted to knock but was told the owner wielded an axe and did not like visitors. They called him the Woodchucker.

It was midsummer when I first visited the Crash Zone, height of the dry season in California. Large portions of the landscape were barren earth, and the powder-dry soil coated the skin of its residents. Here and there, people sifted through the loose dirt with their hands; occasionally someone held up a small trinket they’d discovered, inspecting it in the harsh light of the sun.

A woman walked by with a mixing bowl and a toy unicorn, stooping to extract a scrap of blue tarp from the earth, before she continued on. A minimally dressed man pulled clothes from a dumpster and tried them on, not necessarily in the way they were designed to be worn, and quickly took them off again. He spoke incomprehensibly to himself as he did this, tsking and looking annoyed, as though he just couldn’t find the outfit he was looking for. He was thin, barefoot; I wondered how he stayed alive.

I saw a man thrashing his body in anger as he crossed the street. A dreadlocked white guy in a hoodie wandered by with a baseball bat in one hand and a small, sweet-looking dog in the other. The wind picked up; a dust devil spun. A car crawled out of one of the fields with a flat tire, its rim scraping the asphalt as it entered the street. Every five minutes or so, a plane roared overhead like an angry avian dinosaur.

The Crash Zone spilled from its gills, extending beyond the end-of-the-runway fields and into the surrounding cityscape. One end spilled into a series of baseball diamonds, the dugout now housing, the clubhouse a drug den, the bathrooms given over to the sex trade—“People pull through in $100,000 cars trolling for people to blow them in the car,” a resident told me.

On an adjacent street, a man on crutches lived on a bench next to what was once a demonstration garden for drought-tolerant plants, according to a small sign, which had devolved into a garden of rocks and bare earth. The street proceeds across a large bridge where a solitary tent occupied one of the little nooks designed as a place for pedestrians to linger and look out over the Guadalupe. The bike and pedestrian paths that weave through the riparian corridor below provided access to a neighborhood of tents and shacks, a leafy suburb of the Crash Zone known as the Enchanted Forest. Its inhabitants pulled their cars over the curb, using the paths as driveways. Joggers and cyclists and parents pushing strollers paraded through nonetheless.

As San José’s camps have spread, the Guadalupe River parklands have become the frontlines of a local culture war.

The tents flowed along the river to the San José Heritage Rose Garden, where thousands of rare and forgotten varieties have been arranged in concentric rings of paths and beds. Some of those varieties disappeared following the Crash Zone’s pandemic-era population explosion, when incidents of arson and “rose rustling”—horticultural theft—were reported by garden volunteers on the site’s Facebook page, the insinuation of who was responsible clearly legible between the lines of the posts. The tents trickled past the roses and collected in clumps along the edges of the Historic Orchard, whose few remaining trees appear murdered and maimed, where they bumped into the six-foot fence that protects the children at the Rotary PlayGarden, a gift to the city from the local Rotary Club. A gate attendant keeps the you-know-who from wandering in to the $6 million playscape.

As San José’s camps have spread, the Guadalupe River parklands have become the frontlines of a local culture war. “The city’s homeless problem is becoming a PR problem,” a CBS anchor said in a 2019 segment. “From their airplane windows, arriving visitors are greeted by a shantytown of tents, blue tarps, and RVs,” they said, describing the trail network that parallels the river as “an eleven-mile stretch of human misery and suffering.”

The campers swim in the animosity that drenches the airwaves and cyberspaces around them. I wondered how my new friends on the street felt when they heard these things. How much does the angst directed toward them undermine their prospects of recovery? I found myself reading the Tripadvisor reviews of Guadalupe River Park, which encompasses much of the trail system. They felt like a beating.

“The once beautiful walking, running and biking trail has been taken over by homeless, garbage, rats,” wrote Robin G. “Really bad,” wrote hob0525. “It was basically . . . a tour of homeless camps. We walked for over an hour thinking it would get better. . . . It did not.”

In a 2019 survey by the Guadalupe River Park Conservancy, 77 percent of respondents did not feel “welcome and safe” in the park. “It’s something that I’ve never seen before, honestly,” Jason Su, the conservancy’s director, told The Mercury News . “This is just on a scale that’s just so, so large.”

It’s as though the city feels it’s been invaded by the unhoused. But turn San José inside out and it’s a giant homeless camp being invaded by a city.

__________________________________

From Front Street: Resistance and Rebirth in the Tent Cities of Techlandia by Brian Barth. Used with the permission of the publisher, Astra Publishing House. Copyright © 2025 by Brian Barth.

Is Trump a Lame Duck Yet?

Portside
portside.org
2025-11-22 02:18:28
Is Trump a Lame Duck Yet? barry Fri, 11/21/2025 - 21:18 ...
Original Article

The appalled magistrate wrote, “If this procedure did not take place, then the court is in uncharted legal territory in that the indictment returned in open court was not the same charging document presented to and deliberated upon by the grand jury.” The whole case may be thrown out.

And Trump’s crony, Federal Housing Finance Agency Director Bill Pulte, who has combed mortgage records to find ways to prosecute prominent Democrats, is now on the defensive himself, as lawyers challenge his flagrant conflicts of interest. Trump’s minions even managed to bungle a slam dunk by admitting the sheer racism in the Texas redistricting plan, which was then overruled by a three-judge panel, with the most indignant comments coming from a Trump appointee.

Appellate Judge Jeffrey Brown wrote that “it’s challenging to unpack the DOJ Letter because it contains so many factual, legal, and typographical errors. Indeed, even attorneys employed by the Texas Attorney General—who professes to be a political ally of the Trump Administration—describe the DOJ Letter as ‘legally[] unsound,’ ‘baseless,’ ‘erroneous,’ ‘ham-fisted,’ and ‘a mess.’”

Meanwhile, the Republican victory in forcing Democrats to reopen the government with no concessions on health care is looking more and more like a defeat because it keeps the issue of unaffordable health insurance front and center. In the most recent polls, approval of Trump is underwater by 17 points. Even among Republicans, his approval is 68 percent, sharply down from 92 percent in March. As we head into an election year, with Democrats flipping both Houses a distinct possibility, more and more Republican legislators are looking to save their own skins—which gives them more reason to distance themselves from Trump, and the process keeps intensifying.

So are we out of the woods yet? No, we are not.

The more Trump is on the defensive, the more hysterical he becomes. The latest example is his call to execute Democrats who pointed out that the professional military has an obligation to defy illegal commands. Even the White House press office had to walk that back. But Trump is continuing to use carrots and sticks with the corporate parents of media organizations to destroy a free and critical press.

And as an increasingly desperate Trump tries to keep changing the subject and the headlines, watch out for even more reckless foreign-policy adventures.

However, something fundamental has shifted. Trump is not a dead duck yet, but he is increasingly a lame one. And the more he proves impotent to punish defiant Republicans, the more they will keep acting to distance themselves and to weaken Trump.

We may yet redeem our democracy. That seemed a long shot just a few months ago. Not a bad cause for Thanksgiving.\

Robert Kuttner is co-founder and co-editor of The American Prospect, and professor at Brandeis University’s Heller School. His latest book is .  Follow Bob at his site, robertkuttner.com , and on Twitter. rkuttner@prospect.org


Used with the permission. The American Prospect, Prospect.org, 2024. All rights reserved. Click here [use the current article's link] to read the original article at Prospect.org.

Click here to support The American Prospect's brand of independent impact journalism.

Pledge to support fearlessly independent journalism by joining the Prospect as a member today.

Every level includes an opt-in to receive our print magazine by mail, or a renewal of your current print subscription.

Youth Vote Surged in NYC. Was It a Paradigm Shift?

Portside
portside.org
2025-11-22 02:07:05
Youth Vote Surged in NYC. Was It a Paradigm Shift? barry Fri, 11/21/2025 - 21:07 ...
Original Article

As the 2026 election cycle gets underway, advisers to a new crop of candidates are drawing lessons from Zohran Mamdani on how to activate young voters and potentially change the electorate.

A Gothamist analysis of turnout data shows voters between ages 30 and 39 made up the largest share of a record-setting 2 million voters. Turnout among 18- to 29-year-old voters nearly tripled compared to four years ago, the largest increase of any age group.

Mamdani has said he persuaded those groups by not patronizing them. But now advisers are wrestling with whether Mamdani’s success getting young people to the polls represents a paradigm shift or a moment that can’t be recreated.

“I don’t think this is a Mamdani-specific moment,” said Alyssa Cass, a Democratic strategist and partner at Slingshot Strategies.

"I think what you're seeing in New York City is the emergence of what I like to call the 'Precarity Coalition,'" Cass said. “If you are under 40 here in New York City, it's not working for you anymore.”

Cass said young voters are facing daily challenges that make the city sometimes feel unworkable, including the cost of living and childcare. She is currently advising Alex Bores, a state assemblymember representing Manhattan’s East Side who is one of the nearly dozen candidates vying for U.S. Rep. Jerry Nadler’s congressional seat.

“Increasingly, the idea of having a good life is out of reach, and that is for people who are poor, working class, middle class and even upper middle class,” she said.

Other experts say drawing conclusions about local, state legislative or congressional district battles from a citywide race is risky.

“I do think that people need to take a beat because a district race is very different than a mayoral,” said Lupe Todd-Medina, a Democratic political consultant at Effective Media Strategies and the former spokesperson for City Council Speaker Adrienne Adams’ mayoral campaign.

Still, the response to Mamdani’s affordability message and the spike in turnout among younger voters, Cass says, is reconstituting the city’s electorate and should change how candidates campaign going forward.

Traditionally in New York City, candidates often begin their outreach by appealing to insiders, activating Democratic political clubs and interest groups. “I don’t think that does that job anymore,” Cass said.

She said candidates need to meet voters where they are, in person or online, with a consistent message that taps into voters' gut feelings about life in the city.

But Todd-Medina noted that the candidates and their ability to appeal to voters vary at the local level.

She considered Councilmember Chi Ossé's potential Democratic primary bid against Democratic House Minority Leader Hakeem Jeffries, who represents the 8th Congressional District in Brooklyn. Ossé's Council district overlaps with a portion of Jeffries' congressional district.

“Ossé represents a sliver of the 8th Congressional District. So maybe he plays better in brownstone Brooklyn,” said Todd-Medina, citing a left-leaning portion of the district. “But how is he going to play in Seagate? How does he play in Coney Island?” she added, referring to more conservative neighborhoods.

Todd-Medina is currently advising Marlon Rice, who is challenging state Sen. Jabari Brisport, a democratic socialist.

She credited Mamdani for running a hopeful campaign that expanded the electorate. Mamdani, she said, effectively contrasted with what she described as Andrew Cuomo’s “spastic reign of terror” that painted a grim picture of New York City that did not align with most New Yorkers’ day-to-day lives.

But she was reluctant to say the shifts in the electorate were a sign of permanent changes.

“Mamdani might just be the outlier case,” said Todd-Medina. “We don’t know yet because we’re about to start the next electoral cycle.”

Brigid Bergin is an award-winning senior reporter on the People and Power desk. She is fiercely committed to telling stories that help people engage and support democracy. In 2022, she hosted a live, ten-week call-in politics show ahead of the midterm elections called The People's Guide to Power. Brigid's reporting in 2017 included some of the first coverage of a political newcomer now known as AOC. In 2016, she broke the news of a voter purge in Brooklyn just days before New York’s presidential primary, triggering city, state and federal investigations. Brigid also guest hosts The Brian Lehrer Show and All Of It. She graduated from the University at Albany and the CUNY Newmark School of Journalism. Got a tip? Email bbergin@wnyc.org or Signal 917-723-4719.

Joe Hong is the investigative data reporter for WNYC and Gothamist. He previously covered K-12 education for several newsrooms across California. His reporting has led to local reforms in juvenile justice system as well as a state law requiring universal screening for dyslexia. Got a tip? Email jhong@wnyc.org

Gothamist is a website about New York City news, arts, events and food, brought to you by New York Public Radio. Donate. Newsletters.

Infinibay LXD Container

Hacker News
github.com
2025-11-22 01:54:45
Comments...
Original Article

Infinibay LXD Deployment

LXD-based containerization for the Infinibay VDI management platform.

Status

Production Ready - Automated provisioning with intelligent orchestration and multi-distro support

Quick Links

Why LXD?

LXD provides native support for KVM/libvirt, making it ideal for running VMs inside containers without privileged mode or complex workarounds.

Key advantages:

  • ✅ Native KVM device access - no --privileged mode needed
  • ✅ Full systemd support inside containers
  • ✅ Designed for nested virtualization
  • ✅ YAML-based configuration (lxd-compose)
  • ✅ Better security isolation for VM workloads
  • ✅ Minimal performance overhead (~5%)

Supported Operating Systems

Infinibay's LXD deployment supports multiple Linux distributions with automatic package manager detection:

  • Debian/Ubuntu - Uses apt-get (auto-detected)
  • RHEL/CentOS/Fedora/Rocky/AlmaLinux - Uses dnf or yum (auto-detected)
  • openSUSE/SLES - Uses zypper (auto-detected)
  • Arch/Manjaro/EndeavourOS - Uses pacman (auto-detected)

The setup script automatically detects your distribution and uses the appropriate package manager. LXD installation path (snap vs native package) is also auto-detected.

Overview

This directory contains LXD-based containerization for Infinibay using lxd-compose .

Structure:

lxd/
├── run.sh                         # Main management script ⭐
├── .lxd-compose.yml               # Main lxd-compose config
├── envs/
│   └── infinibay.yml              # Infinibay project definition
├── profiles/
│   └── templates/                 # LXD profile templates
├── values.yml.example             # Configuration template
├── setup.sh                       # Automated installation
├── INSTALL.md                     # Complete guide
└── README.md                      # This file

Note: lxd-compose uses a different structure than docker-compose:

  • Main config: .lxd-compose.yml
  • Projects: envs/*.yml files
  • Commands: apply infinibay , destroy infinibay , stop infinibay

Architecture

The deployment creates 4 LXD containers:

  1. infinibay-postgres - PostgreSQL database
  2. infinibay-redis - Redis cache
  3. infinibay-backend - Node.js API + libvirt-node + infiniservice + KVM access
  4. infinibay-frontend - Next.js web interface

Quick Start

# 1. Clone repository and navigate to lxd directory
cd infinibay/lxd

# 2. Run setup (installs LXD, lxd-compose, detects package manager)
sudo ./setup.sh

# 3. IMPORTANT: Activate lxd group (REQUIRED!)
newgrp lxd
# This activates the group in your current session
# You need to do this after setup.sh adds you to the lxd group

# 4. Configure environment variables
# Option A: Edit the auto-generated .env (RECOMMENDED)
nano .env
# setup.sh already created .env with secure auto-generated passwords
# IMPORTANT: Change ADMIN_PASSWORD from auto-generated to your own!

# Option B: If you prefer to start from .env.example before setup.sh
# cp .env.example .env && nano .env
# Then run setup.sh, which will detect and preserve your .env

# 5. Deploy and start Infinibay (smart default - does everything!)
./run.sh
# This one command:
# - Creates containers if they don't exist
# - Starts containers if they're stopped
# - Provisions if not already done (installs PostgreSQL, Redis, Node.js, Rust, libvirt)
# - Shows access URLs when ready
# Takes 5-10 minutes on first run

# 6. Access Infinibay
# URLs will be displayed after ./run.sh completes
# Frontend: http://<frontend-ip>:3000
# Backend API: http://<backend-ip>:4000

What happens:

  • setup.sh - Installs LXD, lxd-compose, detects your distro and package manager, auto-detects LXD path, generates .env with secure passwords
  • newgrp lxd - ⚠️ REQUIRED - Activates lxd group permissions
  • .env configuration - ⚠️ IMPORTANT - Review and change ADMIN_PASSWORD (auto-generated passwords should be personalized!)
  • ./run.sh - Intelligent orchestration: creates containers, provisions software, starts everything
    • Checks if environment exists → creates if not
    • Checks if containers are running → starts if stopped
    • Checks if provisioned → provisions if not (tracked via LXD metadata)
    • Skips already-completed steps automatically
  • Containers have shared /opt/infinibay directory (your code)
  • Data persists in /data directories even if containers are destroyed

Important: Group Membership

After running setup.sh , you may need to activate the lxd group:

Option 1 (Quick - current session only):

Option 2 (Permanent - requires re-login):

logout
# Then login again

How to check if you're in the group:

groups | grep lxd
# Should show 'lxd' in the output

Common Operations

Recommended Workflow (Smart Default)

# One command does everything - creates, provisions, and starts
./run.sh              # Smart default - handles everything automatically

# Fresh start - destroy and recreate everything
./run.sh redo         # or: ./run.sh rd

# Quick status check
./run.sh status       # or: ./run.sh s

Using run.sh (All Commands)

# Smart default workflow (recommended)
./run.sh              # Does everything: create → provision → start

# Manual step-by-step (if you prefer explicit control)
./run.sh apply        # Shortcuts: a, ap - Create containers
./run.sh provision    # Shortcuts: p, pr - Install software

# Container management
./run.sh status       # Shortcuts: s, st - Check status
./run.sh destroy      # Shortcuts: d, de - Remove containers
./run.sh redo         # Shortcut: rd - Destroy and recreate (fresh start)
./run.sh restart      # Shortcuts: r, re - Legacy alias for redo

# Execute commands in containers
./run.sh exec backend bash      # Shortcuts: e, ex
./run.sh exec postgres psql -U infinibay
./run.sh exec frontend npm run dev

# Follow container logs
./run.sh logs backend           # Shortcuts: l, lo
./run.sh logs postgres

# Update profiles only (after modifying templates)
./run.sh setup-profiles         # Shortcut: sp

# Show help with all shortcuts
./run.sh help

Complete shortcut reference:

Command Shortcuts Description
apply a , ap Create and start containers
provision p , pr Install software in containers
redo rd Destroy and recreate everything
destroy d , de Stop and remove all containers
restart r , re Legacy alias for redo
status s , st Show container status
setup-profiles sp Update LXD profiles only
exec e , ex Execute command in container
logs l , lo Follow container logs

Direct LXC Commands

# View container status
sg lxd -c "lxc list"

# Execute commands
sg lxd -c "lxc exec infinibay-backend -- bash"

# Create snapshot
sg lxd -c "lxc snapshot infinibay-backend backup-$(date +%Y%m%d)"

# List snapshots
lxc info infinibay-backend

Current Status

Implemented and Working:

  • ✅ Creates 4 Ubuntu containers with resource limits
  • ✅ Mounts shared /opt/infinibay directory (your code)
  • ✅ Persistent /data directories for each service
  • ✅ Automated provisioning scripts for all containers
  • ✅ PostgreSQL installation and configuration
  • ✅ Redis installation and configuration
  • ✅ Node.js 20.x LTS + npm
  • ✅ Rust toolchain (for libvirt-node native modules)
  • ✅ libvirt + KVM with /dev/kvm device access
  • ✅ Systemd services ready for backend/frontend
  • ✅ Network connectivity between containers
  • ✅ Universal package manager support (apt/dnf/zypper/pacman)
  • ✅ Automatic LXD path detection (snap vs native)
  • ✅ Smart default orchestration with state tracking
  • ✅ Provisioning state persistence via LXD metadata

Still Manual:

  • ⏳ npm install in backend/frontend
  • ⏳ Database migrations
  • ⏳ Starting Infinibay services
  • ⏳ Application configuration

After provisioning, you need to:

  1. Install npm dependencies in backend/frontend
  2. Run database migrations
  3. Configure and start Infinibay services

See INSTALL.md for detailed instructions.

Troubleshooting

"No project selected" error

# Make sure you specify the project name
lxd-compose apply infinibay  # ✓ Correct
lxd-compose apply             # ✗ Wrong

"Unable to read the configuration file" error

# You need to be in the lxd group
newgrp lxd
# Or logout/login

"Permission denied" on LXD socket

# Check if you're in lxd group
groups | grep lxd

# If not, the setup script should have added you
# Just run:
newgrp lxd

Smart default fails at provisioning step

# Check individual container status
./run.sh status

# Use redo to start fresh (destroys and recreates everything)
./run.sh redo

Want to force re-provisioning

# Option 1: Use redo command (destroys and recreates everything)
./run.sh redo

# Option 2: Manually clear provisioning state for specific container
lxc config unset infinibay-backend user.provisioned
lxc config unset infinibay-frontend user.provisioned
lxc config unset infinibay-postgres user.provisioned
lxc config unset infinibay-redis user.provisioned
# Then run: ./run.sh

vs Native Installer

Aspect LXD (Current) Native Installer
Status 🚧 In Development ✅ Production Ready
Provisioning Manual for now ✅ Fully automated
Isolation ✅ Full container isolation ❌ System-wide
Rollback ✅ Snapshots ❌ Manual
Complexity Medium Low

Recommendation: Use the native installer for production deployments until LXD provisioning is complete.

Contributing

See INSTALL.md for development workflows.

References


Last Updated : 2025-11-21 Status : Production Ready

Rula Daood Wanted To Change the Most Important Arab Political Body in Israel

Portside
portside.org
2025-11-22 01:54:01
Rula Daood Wanted To Change the Most Important Arab Political Body in Israel barry Fri, 11/21/2025 - 20:54 ...
Original Article

On November 4, Rula Daood became the first woman to apply to lead the Higher Arab Monitoring Committee, an umbrella organization that advocates for Arab communities within Israel at the national level.

For Daood, the national co-director of the grassroots movement Standing Together, this would have been a step from activism into politics. "I wanted to bring a new face and a new agenda to the committee," she tells Haaretz. A change, she says, that is badly needed.

The chairman of the committee sets the political agenda for Arab Israeli citizens. It is the only elected position on the committee, which comprises of Arab Knesset members, local council heads and representatives of different streams in the Arab community. Until this weekend, the post was held by former Hadash MK Mohammed Barakeh for a decade. The election, which took place on Saturday, was won by Jamal Zahalka , former chairman of the Balad party.

"The Higher Committee is supposed to mobilize and organize the Arab minority in Israel," Daood, 40, said in an interview with Haaretz before Saturday's election. "It is a powerful place that could bring Palestinian rights forward, but it hasn't been using its power for the past 20 years."

The committee was founded when protests among Palestinians against then-Prime Minister Menachem Begin's right-wing government grew stronger – ultimately leading to the first intifada in the eighties. The Higher Arab Monitoring Committee was meant to function as a unifying independent political organization that would coordinate the political activities of various Israeli Arab nonprofits, advocacy groups and other organizations, leading to change at a national level for Arab and Palestinian citizens of Israel.

"227 Arab Israeli citizens were killed this year, and the feeling is that nobody really cares," Daood says, referring to the record-high rates of homicides and gun violence devastating Arab communities across Israel. Her candidacy was an expression of a wider change in the Arab communities and political establishment, struggling to become more diverse and inclusive to women.

Daood recalls that she made her decision to put in her candidacy during a protest in her hometown Kafr Yasif. "Many people were around me," she says. "Many young people and many women." In contrast, on the stage were only men. They were the only ones who would speak. "That was the moment I decided we need a change."

Some of the thousands of Palestinian citizens of Israel that protested rising murder rates and the government's inaction, in Arabeh, northern Israel, earlier this month.

Some of the thousands of Palestinian citizens of Israel that protested rising murder rates and the government's inaction, in Arabeh, northern Israel, earlier this month.

Some of the thousands of Palestinian citizens of Israel that protested rising murder rates and the government's inaction, in Arabeh, northern Israel, earlier this month.

During the past months, anger within the Arab community about the committees' lack of action against rising crime and murder rates, which continues to rise, especially among young people. At a demonstration on November 1 against the violence in the northern town of Arabeh, protesters reportedly tried to prevent the committees' officials, including then chairman Barakeh, from speaking.

The old guard

Since Daood has not been part of the committee, she needed to be endorsed by at least six municipalities. "I went to the mayors of different municipalities, and I got more than six supporters," Daood explains. But the day before the mayors had to finalize their choice of candidates, Daood got a call from a committee member telling her that one mayor withdrew his support for her.

At this point, Daood had six supporters left, still enough for her candidacy. But in the evening, her phone rang again – another mayor changed his mind. Later that night, she received a text message informing her that another mayor had withdrawn his endorsing signature, making it certain that Daood would not have enough votes.

To Daood it was clear that the mayors had been pressured to draw their votes. "The committee is very much controlled by old politics, people who have been holding their positions for decades." It seems they were afraid of the change, she speculates. "They feel threatened by a new face and a new agenda, speaking a different language, that can really stir things up. And they didn't want me to be there."

To her, the problem of leadership is not limited to the Arab political establishment in Israel. "There is a lack of leadership on the left that speaks about the change we need," Daood explains. "About the day after [the war], about real partnership between Israelis and Palestinians, about how we can change our reality." To her, it is clear that leaders must focus on what can be done instead of what cannot.

"Many people don't believe in what they can do," she says. "They don't believe they have the power." To her, the old political establishment represented by the committee cannot bring about change. "They don't have the most progressive kind of ideas. They don't believe in organizing, in working with communities and with people. This is what I wanted to change with my candidacy."

Standing Together co-directors Rula Daood, left, and Alon-Lee Green, holding signs that read, "We refuse to occupy Gaza."

'I was able to make some noise'

When the committee opened the registration for the elections and announced the names of the election committee, all eight members were men. While this was business as usual in the years before, this time, the committee faced backlash. Women's rights organizations and feminist activists spoke out against it.

"When I put my candidacy first, it made a lot of fuss. Nobody really expected it, and it moved many things," Daood says. "I was able to make some noise." Four more candidates entered the race, among them another woman: former MK and feminist activist Neveen Abu Rahmoun.

For Daood, who together with her Standing Together co-director Alon-Lee Green became international faces of joint Palestinian and Israeli resistance to the war in Gaza, leading the committee would have been the next step in her career. With a national election on the horizon, questions about her future have mounted. Today, she is still unsure about where she will go from here.

"I honestly don't know," she says. But there are things she is sure of: "I want to change politics. I want to make a change for my own people, but I also want the whole Israeli society to understand that I am a legitimate leader for both Jews and Palestinians in this land." She would love if this is possible to do through an organization like Standing Together, she says. "If your question is about the Knesset – Maybe. Probably. Really, I don't know."

The Higher Arab Monitoring Committee did not respond to Haaretz's request for comment.

Haaretz is an independent daily newspaper with a broadly liberal outlook both on domestic issues and on international affairs. It has a journalistic staff of some 330 reporters, writers and editors. The paper is perhaps best known for its Op-ed page, where its senior columnists - among them some of Israel's leading commentators and analysts - reflect on current events. Haaretz plays an important role in the shaping of public opinion and is read with care in government and decision-making circles. Get a digital subscription to Haaretz.

Installing FreeBSD 15 on my desktop

Lobsters
yorickpeterse.com
2025-11-22 01:26:34
Comments...
Original Article

This week I've been working on a script and slides for a YouTube video about Inko . After a week of doing that I needed a bit of a break. Last week I wrote a bit about FreeBSD , and specifically about wanting to try it out on my yet to be delivered Framework 16 laptop. This got me thinking: why don't I try FreeBSD on my desktop first, then see if it's still worth trying out on a laptop? After all, my desktop has a spare SSD that I don't use much, so I could move its data elsewhere temporarily and install FreeBSD on this SSD, leaving my main system untouched.

What follows is a sort of transcript (with some editing) of doing just that, a process that took a total of some three hours. Because I wrote most of this while actually performing the work, it may feel a little chaotic at times, but I hope it gives a bit of insight into the process.

The hardware

The desktop in question uses an AMD Ryzen 5600X CPU, with an Intel Arc A380 GPU. The SSD FreeBSD will be installed on is a Samsung Evo 860 with 256 GiB of storage. The WiFi card is an Intel AX200, which is supported by FreeBSD.

Preparing a USB drive

I downloaded the latest FreeBSD 15 snapshot ISO format, then wrote it to a random USB drive using dd . I initially tried to use GNOME Disks to restore the ISO to the USB drive, but for some reasons this results in it not being a bootable drive. I vaguely recall having had similar issues in the past with Linux distributions, so this isn't FreeBSD specific.

Installing FreeBSD

Booting up worked and the installer detects the AX200, but then seemingly got stuck for a good minute or so, after which it moved on. I'm not sure why, but it didn't seem to matter much as the rest of the installer worked fine.

Using the installer I went with a ZFS on root setup and enabled disk encryption. In particular I enabled the "local_unbound" service to cache DNS lookups. Knowing this won't work with my router (which runs a local DNS server/cache) because it doesn't support DNSSEC, I was a bit surprised to see the installer not consider this at all, i.e. there's no "use local_unbound but without DNSSEC" option.

First boot

After installing FreeBSD I rebooted into the target SSD. The first thing I noticed was a bunch of error messages from the ntp daemon (which I enabled) saying it couldn't resolve a bunch of DNS names. This is because my router doesn't support DNSSEC. I fixed this by creating /var/unbound/conf.d/disable-dnssec.conf with the following contents:

server:
    module-config: "iterator"

Because FreeBSD ships vi by default (not vim, actual vi) this was a little annoying as vi works a little different compared to vim. After saving the file I restarted the local_unbound service, and all was well again.

FreeBSD offers both doas and sudo . I figured I'd give doas a try, mainly because I wanted to give it a try. This requires you to copy /usr/local/etc/doas.conf.sample to /usr/local/etc/doas.conf and edit it accordingly. I just used it with the permit :wheel rule, which is enough for most people. I then found out that doas doesn't support password persistence outside of OpenBSD, meaning you have to enter your password again for every doas command. While there appears to be a fork available called opendoas that does support it, it in turn doesn't appear to be actively maintained (judging by the GitHub repository ). I ended up going back to sudo instead.

I then installed Fish and made it the default shell as follows:

chsh -s /usr/local/bin/fish yorickpeterse

I then logged out and back in again, and Fish works as expected.

FreeBSD shows a message-of-the-day when you log in, which I don't want as it's rather long. To disable this, I emptied /etc/motd.template then ran sudo service motd restart to re-generate the message, then disabled the service using sudo sysrc update_motd="NO" . We also need to remove /var/run/motd . I think in hindsight editing the template wasn't required as I could've just disabled the service then remove /var/run/motd file. Ah well, lessons learned I guess.

Fighting the GPU

Now it's time to make sure the GPU is set up. The reason my desktop is using an Intel GPU is because it used to use an aging AMD RX 550, but after dealing with AMD driver bugs for a few months I got fed up and decided to replace it. I picked the A380 because it was the cheapest GPU with support for hardware decoding that I could find.

To do this we have to install drm-kmod , which pulls in about 130 driver related packages (yikes). Next we need to make sure the driver is loaded upon startup by adding it to /etc/rc.conf like so:

sudo sysrc kld_list+=i915kms

This doesn't affect the existing system though, so we also have to load the module using kldload ( modprobe but for FreeBSD), so I ran this:

This crashed my system. Brilliant. Worse, because I added the module to /etc/rc.conf it keeps crashing when you reboot. The error shown when FreeBSD tries to load the module says to run pkg install gpu-firmware-kmod to install the necessary firmware, but because the module is now loaded at startup we first have to figure out how to get back into a working system. I found a forum post that offered some suggestions, but they didn't work.

I ended up booting into the installation USB and mounted the host drive following the instructions from this article , using /dev/ada0p3 as the drive name. I then opened /mnt/etc/rc.conf and commented out the line that loads the i915kms driver, then rebooted. We have a working system again!

Now to do what that error said: install the missing package. Well, except it wasn't missing because when I installed it pkg said it was already installed. This is fine, I guess?

A bit of searching reveals this issue , first reported in August 2024. There's a pull request that should fix this , but I'm not going to compile a custom kernel just to get a working system. It also seems the PR has just been sitting around for a while, which doesn't bode well.

Most people would give up at this point, but I have one final trick up my sleeve: when I replaced my AMD RX 550 I didn't throw it away in case I ever needed it again, so I can temporarily use it instead of the A380. It shouldn't be necessary, but at this point I want to try and get a working desktop environment just so I can say I at least tried.

So after trying a few different screwdrivers to unscrew the GPU bracket screws and some cursing later, the A380 is replaced with the RX 550. I booted up the system and edited /etc/rc.conf to load the amdgpu driver instead of i915kms . I then decided to reboot the system for good measure, though this isn't strictly necessary. I am now presented with a system that works, except the console font is tiny for some reason. This article suggests using vidcontrol -f terminus-b32 which did the trick.

Installing a desktop environment

Where were we again? Oh yes, I was going to install a desktop environment. I'd use GNOME, but GNOME recently announced they were going to depend on systemd more and more, and the GNOME version provided by FreeBSD is a bit old at this point (GNOME 47). KDE seems better supported, so I'll give that a try. The FreeBSD installer is supposed to come with an option to install KDE for you, though the ISO I used for FreeBSD 15 didn't have that option. Either way, from what I found it uses X11 and I want to use Wayland, so I wouldn't have used it anyway. This article lists some steps to enable KDE. The socket options it suggests to apply seem a bit suspicious, as in, they look like the kind of setting people just copy-paste without thinking, so we'll skip those unless they turn out to be required after all.

Let's install the necessary packages:

sudo pkg install seatd kde sddm

This ends up installing close to 700 packages. This took a while since pkg downloads packages one at a time. Support for concurrent downloads was first requested back in 2017 , but isn't implemented as of November 2025. This wouldn't be a huge problem if it wasn't for the FreeBSD mirrors only supporting speeds in the range of 5-20 Mib/sec, while my internet connection's maximum speed is 100 MiB/sec.

Upon the installation finishing, I realized I hadn't explicitly stated or switched to the latest branch for the FreeBSD ports, so I edited /usr/local/etc/pkg/repos/FreeBSD.conf to be as follows:

FreeBSD-base {
  url: "pkg+https://pkg.freebsd.org/${ABI}/base_latest",
  enabled: yes
}

I then ran sudo pkg update followed by sudo pkg upgrade and there was nothing to update, so I guess we're all good.

Configuring the desktop environment

Now to enable the service we need for KDE. The linked article doesn't mention enabling SDDM but it seems to me like that would be required for it to start, so we'll give that a go as well:

sudo sysrc dbus_enable="YES"
sudo sysrc seatd_enable="YES"
sudo sysrc sddm_enable="YES"
sudo service dbus start
sudo service seatd start
sudo service sddm start

This results in SDDM starting and showing the login screen. The default session is set to Wayland already, and logging in works fine. Neat!

Moving the mouse around I'm noticing some weird artifacts on the desktop wallpaper:

Artifacts on the
desktop

Looking at the display settings I noticed scaling is set to 170%, while for this display it should be 200%. Changing this removed the artifacts, so I guess this is some sort of KDE bug?

Another thing I'm noticing when moving the cursor around or when window animations play is that it isn't as smooth as GNOME, as if the display's refresh rate is lower than it should be, though it's in fact set to 60hz. I vaguely recall having this issue on GNOME when I was still using the AMD RX 550, so maybe it's a GPU issue. Or maybe it's those socket options I decided not to enable initially, so let's give that a try, just in case, though it's a bit of a stretch. First I ran the following to apply the settings to the existing system:

sudo sysctl net.local.stream.recvspace=65536
sudo sysctl net.local.stream.sendspace=65536

The resulting output suggests this is already the default value, so I guess that's not the reason, and the settings might not be necessary at all.

Now let's get rid of some software I don't need such as Konqueror and Kate:

sudo pkg remove konqueror kate

Initially this gave me a bit of a heart attack as it tells you that the kde package will also be removed, but it turns out to be fine and not actually uninstall your entire KDE setup.

Audio

Audio works fine with no configuration necessary. Neat.

Network

While the network itself works, there's no GUI application of any kind to manage it, as NetworkManager isn't available on FreeBSD. I found networkmanager-shim which is required by kf6-networkmanager-qt . I installed the latter in case I'd also need that, logged out and back in again and...nothing. Searching a bit more lead to me finding networkmgr which is available as a FreeBSD package, so let's try that:

sudo pkg install networkmgr

Logging out and in again and there's now a network icon in the Plasma panel. Unfortunately, it seems to be an X11/Xwayland application and looks horrible:

networkmgr on FreeBSD

Apologies for the poor quality! I hadn't set up a screenshot application of some sort and didn't want to also deal with that, so I just took a photo with my phone.

It also doesn't appear to show anything related to WiFi. ifconfig doesn't list anything WiFi related either. I guess I have to set up wpa_supplicant or something along those lines, but I'd prefer it if my desktop environment could manage it for me.

Bluetooth doesn't appear to work either, probably for the same reasons because it's handled by the same AX200 chip. I found that wifimgr can be used to manage WiFi, but starting it results in it complaining I have to first configure a device in /etc/rc.conf . Ugh.

It's at this point that not only was it getting late, I also had enough. I can see the appeal of FreeBSD, and it's impressive how much up to date software there is in the ports repository, but there's a reason I moved away from Arch Linux several years ago: I just don't have the patience nor interest to endlessly fiddle with configuration files just to get a basic system up and running.

Conclusion

If you have enough patience and time I think you can set up a decent KDE desktop environment using FreeBSD, assuming your hardware is properly supported. That is also the biggest challenge though: FreeBSD developers have limited resources and port the Linux GPU drivers to FreeBSD, instead of using bespoke drivers. This means it will always lag behind Linux, ranging from maybe only a few weeks to months or even years.

Based on the challenges I ran into while trying to install FreeBSD on my desktop, I'm not going to try and install it on a laptop any time soon. I just don't have the patience or interest. If I did, I'd go back to using Arch Linux.

There are also some choices that FreeBSD makes that I don't agree with or don't want to deal with, such as the archaic way of writing service files or setting up log rotation, or the fact that the output of the -h option for the FreeBSD userspace utilities (e.g. ls ) is as good as useless.

On the bright side, if the FreeBSD foundation continues focusing on improving the laptop and desktop experience of FreeBSD then all this could be different 1-2 years from now, so maybe I'll try again in the future.

Letter from Codeberg: Onwards and upwards

Lobsters
blog.codeberg.org
2025-11-22 00:31:52
Comments...
Original Article

(This is a stripped-down version of the newsletter sent out to Codeberg e. V. members as an email. The version emailed to our members contains some additional details concerning the association's Annual Assembly. If you are interested in helping us shape Codeberg, consider contributing or participating our non-profit association ! Strength comes in numbers; we could always use your support!)

Dear Codeberg e.V. members and supporters!

It's time to share some news about what has happened in the past months around Codeberg.

Highlights

  • Codeberg e. V. held its Annual Assembly, and elected a new Presidium, which in turn appointed a new Executive Board.
  • We reworked our Privacy Policy and clarified our policies on repository licensing in our Terms of Use.
  • Intra-association meetings are now held regularly.
  • We now have a second part-time employee.
  • Hardware updates!

A brief status update

Codeberg has experienced immense growth over the past few months. Here's a rundown of our numbers:

  • Codeberg e.V. now has more than 1000 members (1208, to be exact).
    • 786 members have active voting rights, 415 of them are supporting members and the remaining 7 are honorary members.
    • Out of the 1208 members, 61 of them are corporations (which can only have a supporting membership without voting rights).
  • We now have more than 300k repositories and recently crossed 200k registered user accounts.
  • Some more well-established projects now have a presence on Codeberg.
  • As of September 2025, Anja joined us as a part-time employee to help us with administrative matters. We now have two part-time employees.

Annual Assembly & Elections

Once every year, the entire member body of Codeberg e. V. members meets in what we call the Annual Assembly. It guarantees that the matters of the association are in the hands of its users. Codeberg e. V. has more than 1000 individuals backing it.

Once every two years, the Assembly elects up to eight members to the Presidium. The Presidium meets a few times every month to—as the name suggests—preside over matters involving Codeberg's direction. Such tasks may involve implementing proposals that were accepted by the Assembly, answering emails and responding to media inquiries and organizing teams of volunteers. The following people were elected for the 2025-2027 term in alphabetical order:

  • Andreas Reindl (@crapStone)
  • Andreas Shimokawa (@ashimokawa)
  • Daniele Gobbetti (@daniele)
  • Daphne Preston-Kendal (@dpk)
  • Moritz Marquardt (@momar)
  • Otto Richter (@fnetX)
  • Panagiotis Vasilopoulos (@n0toose)
  • William Zijl (@gusted)

Additionally, the Presidium is responsible for appointing an Executive Board, which is responsible and liable for Codeberg's day-to-day operations. With that being said, the Presidium has appointed the following people for the 2025-2026 term:

  • Otto Richter (@fnetX)
  • Moritz Marquardt (@momar)
  • William Zijl (@gusted)

Both bodies were previously exclusively German, whereas the Presidium now comprises of members residing in three European countries and the Executive Board will have a member from the Netherlands. This also marks the first time that our Executive Board has had three members — that being the maximum amount possible.

We strive to be more internationally oriented, as well as adjust to the immense growth we've been experiencing. As such, we have been making efforts in documenting our structures better to help with onboarding.

New privacy policy

We sent out an email to all our users to inform them about our new Privacy Policy that went into force on October 2nd, 2025. In case you missed it, you can check out the commit that we linked here: https://codeberg.org/Codeberg/org/src/commit/2bad44ef35bfc2e154d2c3168bf44d1be739528d/PrivacyPolicy.md

Terms of use

Codeberg is a non-profit organization with the explicit mission of advancing the development of free, libre and open-source projects. It was founded for that purpose, and its continued operation is made possible by all the volunteers, the donors and the members that share said mission. We offer a free service, and what we want to ask from our users is simple: To give back something to the community by letting others reuse, adapt and extend what they create.

In principle, we asked people to make their works "free and open-source". But what is "free and open-source", exactly?

Earlier, our Terms of Use avoided answering that question. Instead, it stipulated that people had to use a license approved by the Free Software Foundation and/or the Open Source Initiative . Therefore, blogs and personal websites licensed under the copyleft Creative Commons Attribution-ShareAlike 4.0 International were technically against the rules, despite said license being more appropriate than, say, the software-oriented 3-Clause BSD License ! We found that our Terms confused or scared away users and use cases that we view as aligned with the Codeberg goals and values: Book authors and artists, conference presenters that wanted to upload their slides, people wanting to use/host public domain content or people that wanted to use copyleft assets for their libre game. Those are cases that we previously quietly tolerated (and were already happy to host despite them not being technically allowed).

We made an effort to provide such projects clarity and make them officially allowed as well. After long discussions with Codeberg e. V. members as well as members of our wider community, we came up with the following proposal, which was formally agreed upon by Codeberg e. V.'s General Assembly and will be implemented soon after publication: https://codeberg.org/Codeberg/org/pulls/1219

An additional benefit of the changes made is that they reinforce our own governance and independence, as we would not rely on third-party organizations and their governance structures as much as we have. Trying to define what is "free and open-source" was a rather lengthy endeavour. However, we find that this was necessary—especially given the context of our recent growth—and we truly hope that the greater community will be rather pleased with the result.

Regular Codeberg Meetings

As previously discussed among members, we have now established weekly meetings of the Codeberg community. The meetings are open to everyone, but we expect that most people are Codeberg e. V. members.

After a quick round of introduction and some small talk, we usually dedicate our time towards a topic announced in advance, often "open end" and until very late. During the first meetings, topics have been mostly technical, but we aim at addressing non-technical areas in the future, such as community building, the documentation or public relations.

If you have a question or a matter that you'd like to discuss with other Codeberg contributors, you can always join and present the matter during the first minutes of the session. When no urgent and spontaneous topics need to be discussed, we move to the scheduled topic of the session.

The meetings are scheduled every Tuesday evening, 18.00 Berlin time or 17.00 UTC time. We meet at https://lecture.senfcall.de/ott-zml-1vs-qcc . The meeting is open to everyone interested in joining, but we mostly target Codeberg e. V. members

New employee for administrative tasks

Since September 15, 2025, Anja Hänel joined our virtual office team. Codeberg is growing rapidly, and so are the administrative chores required to keep the project running. We are grateful for the expertise she brings into our team, which helped us clarify, simplify and accelerate internal processes around member management and accounting.

Her help relieves Andreas Shimokawa (our current part-time system administrator and developer) from some of the office tasks. Together, they improved the member management and accounting tools. For example, some of you who have outstanding membership fees have likely received friendly reminders recently.

Infrastructure status

Since the last update, there have been some changes to the infrastructure powering Codeberg. We are running on 3 servers, one Gigabyte and 2 Dell servers (R730 and R740). We have bought and added 2 new SATA SSDs (8TB capacity) to the Dell servers to address growing storage demands. This finally balanced storage capacity of the 3 Ceph nodes which was unequal for historical reasons and resulted in inefficient storage usage on the first node.

One of our servers, Achtermann (R730), was running with only 1 CPU (as the second slot had been damaged). While the server was reliable, some workarounds were necessary (such as cramming some hardware into limited PCIe slots, as most of them are bound to the defunct slot). Recently, we received two donated Dell R740s and deployed one of them to replace the damaged R730. Waxenstein (our newly deployed R740) performs much faster than Achtermann (1 CPU, 160GB RAM) and has 384GB of RAM (more than twice than that of Achtermann!). We repurposed the RAM of the now-decommissioned Achtermann and used it to double the RAM of Aburayama, which is the name of the R730 that is still in service. This boost let us allocate much more resources to our various containers, resulting in performance improvements for Codeberg. If you are interested in more detailed and up-to-date information about our hardware, we maintain an overview in the following repository: https://codeberg.org/Codeberg-Infrastructure/meta/src/branch/main/hardware

Hardware donations allow us to get access to high quality hardware. Although aged, the performance (and even energy efficiency) is often not much worse than with new hardware that we could afford. In the interest of saving embodied carbon emissions from hardware manufacturing, we believe that used hardware is the more sustainable path.

We are considering to repurpose older generations of hardware to offsite CI/CD runners. While the hardware is less energy efficient than newer, we hope to use direct solar power to operate CI/CD nodes only during sunshine hours. Using efficient machines for 24/7 operation and less efficient machines for about 4 to 8 hours a day is likely a reasonable approach.

Some users indicated interest in CI runners using the ARM CPU architecture. Currently, Apple's M1 and M2 series have outstanding energy efficiency. We are investigating how broken Apple laptops could be repurposed into CI runners. After all, automated CI usage doesn't depend on the same factors that human beings depend on when using a computer (functioning screen, speakers, keyboard, battery, etc.). If you own a broken M1/M2 device or know someone who does, and believe that it is not worth a conventional repair, we would be happy to receive your hardware donation and give it a try! (Sidenote: There are also non-profit organizations that may be willing to accept your working devices and repurpose them for those in need. For Germany, we'd recommend checking out Computertruhe e. V. .)

On a software level, we are currently struggling with recurring performance degradation of our Galera database cluster (MariaDB). While it usually holds up nicely, we see sudden drop in performance every few days. It can usually be "fixed" with a simple restart of Forgejo to clear the backlog of queries. We are still investigating potential issues with our database cluster. In the meantime, work is ongoing to optimize database queries that were observed to trigger the behaviour in Forgejo.

Community Spotlight

To end this letter, we'll share a few (of the many) cool repositories on Codeberg that caught our eye:

  • git-pages is an alternative to Codeberg Pages , it uses a different approach to serving static pages that is more efficient. Codeberg is planning to gradually migrate to it.
  • Readeck ( Codeberg Link ) allows you to preserve web content locally to read it later and find it back easily.
  • µcad ( Codeberg Link ) is a description language for creating geometric objects that can be used for 3D printing or CNC milling. Although it is in "an early stage of development" at time of writing, we are definitely intrigued!
  • ly is a display manager for Linux and BSD (i.e. it provides a "login screen"). It comes with a wave-y animation that may make your laptop look cooler.
  • GoToSocial ( Codeberg Link ) is a self-hostable social networking service that uses the ActivityPub protocol.
  • Typesetter ( Codeberg Link ) is a local editor for editing Typst documents.

Thank you for your trust and continued support, as well as for reading our updates.

--

Codeberg.org
Codeberg e.V. – Arminiusstraße 2 - 4 – 10551 Berlin – Germany
Registered at registration court Amtsgericht Charlottenburg VR36929.

Sharper MRI scans may be on horizon thanks to new physics-based model

Hacker News
news.rice.edu
2025-11-22 00:30:26
Comments...
Original Article
Nov. 17, 2025
Walter Chapman and research team
Walter Chapman and Thiago Pinheiro discuss a molecular model of their research.

Researchers at Rice University and Oak Ridge National Laboratory have unveiled a physics-based model of magnetic resonance relaxation that bridges molecular-scale dynamics with macroscopic magnetic resonance imaging (MRI) signals, promising new insight into how contrast agents interact with water molecules. This advancement paves the way for sharper medical imaging and safer diagnostics using MRI. The study was published in The Journal of Chemical Physics Nov. 12.

This new approach, known as the NMR eigenmodes framework, solves the full physical equations that can be used to interpret how water molecules relax around metal-based imaging agents, a task that previous models approximated. These findings could alter the development and application of new contrast agents in both medicine and materials science.

“By better modeling the physics of nuclear magnetic resonance relaxation in liquids, we gain a tool that doesn’t just predict but also explains the phenomenon,” said Walter Chapman , the William W. Akers Professor of Chemical and Biomolecular Engineering. “That is crucial when lives and technologies depend on accurate scientific understanding.”

Modeling a molecular process

During an MRI scan, contrast agents are often used to enhance image clarity. These agents, typically based on a gadolinium ion encased in an organic shell, alter the way nearby water molecules respond to magnetic fields. This alteration, known as relaxation, enhances the contrast in tissue images.

Until now, most scientific models describing this process have relied on significant simplifications, treating complex molecular motions with limited fidelity to the real system’s behavior, which limited their predictive accuracy. The researchers sought to improve upon this.

“Our previous work used detailed simulations to study how water molecules interact with these contrast agents,” said Dilipkumar Asthagiri, a senior computational biomedical scientist in the National Center for Computational Sciences at Oak Ridge National Laboratory. “In the present paper, we developed a comprehensive theory to interpret those previous molecular dynamics simulations and experimental findings. The theory, however, is general and can be used to understand NMR relaxation in liquids broadly.”

A framework rooted in physics

To create a more effective approach, the research team turned to the Fokker-Planck equation, a master equation that describes how the probabilities of molecular positions and velocities evolve. By solving this equation, they were able to capture the full spectrum of molecular motion and relaxation.

Essentially, the eigenmodes framework identifies the “natural modes” of how water molecules respond to contrast agents at the microscopic level. These modes provide a more detailed and accurate picture to interpret the relaxation process than earlier models could offer.

MRI image

“The concept is similar to how a musical chord consists of many notes,” said Thiago Pinheiro , the study’s first author, a Rice doctoral graduate in chemical and biomolecular engineering and postdoctoral researcher in the chemical sciences division at Oak Ridge National Laboratory. “Previous models only captured one or two notes, while ours picks up the full harmony.”

This framework not only reproduces experimental measurements at clinical MRI frequencies with high precision, but it also demonstrates that widely used simplified models are specific instances of a broader, more comprehensive theory.

Broader impacts beyond imaging

The implications of this research extend beyond medical imaging. Because NMR relaxation is used to study the behavior of liquids in various scientific and industrial applications, the framework could also be applied in areas such as battery design and subsurface fluid flow.

“This kind of detailed modeling can help us understand how fluids behave in confined spaces like porous rocks or biological cells,” said Philip Singer, assistant research professor in chemical and biomolecular engineering at Rice. “It’s a fundamental tool that links molecular-scale dynamics to observable effects.”

The research team has made its code available as open source to encourage broader adoption and further development. Co-authors of the study also include Betul Orcan-Ekmekci from Rice’s Department of Mathematics, who contributed significant insights into the mathematical modeling.

The Ken Kennedy Institute, Rice Creative Ventures Fund, Robert A. Welch Foundation (No. C-1241) and Oak Ridge Leadership Computing Facility at Oak Ridge National Laboratory (No. DE-AC05-00OR22725 with the U.S. Department of Energy) supported this study.

A bug caused by a door in a game you may have heard of called "Half Life 2"

Lobsters
mastodon.gamedev.place
2025-11-22 00:15:29
Comments...

How to get started with the ed text editor (2022)

Lobsters
www.redhat.com
2025-11-22 00:04:16
Comments...
Original Article

For as well-loved as the vi command is, it's the ed command that's considered the standard Unix text editor. It was the very first text editor for Unix, and it's available on even the most modern Linux systems.

Unlike text editors you may be used to on Linux or another system, ed doesn't open a window or even a screen of its own. That's because it's a functional editor that you can control either interactively or with a script. If you're already familiar with sed , then you'll find ed easy to learn. If you're new to both, ed can give you a different perspective on how you can process and modify data on your system.

Launch ed

Launching ed is easy; just enter the command at the prompt:

$ ed

When you first launch ed , you get no feedback or prompt. That's the expected behavior, so don't panic. Your system hasn't crashed, ed is just waiting for your instructions.

To get ed to be a little more visual, use the p command to create a prompt. Type the letter p followed by the Return or Enter key:

$ ed
p
?

The question mark ( ? ) is the default ed prompt.

Use the ed buffer

While ed is active, it uses a place in memory to store data. This location is called a buffer . Such storage is significant because you're not editing a file directly. You're editing a copy of file data placed into the buffer. As long as you save the buffer when you're done, ed preserves any changes you make to the data.

If you exit ed without writing changes to a file on disk, it loses all changes because they only existed in the buffer. It's no different than closing any application without saving changes, but ed doesn't warn you, so keep this in mind.

Generate text with ed

Similar to the vi editor, ed starts in command mode . This means you can issue commands to the editor, as you did to display a prompt, but you can't write or edit text without issuing a command first.

You can append text to the current buffer using the a command followed by the Return or Enter key. Whatever text you type into the terminal now will be appended to the buffer. Stop ed from appending text to the buffer by typing a solitary dot ( . ) on its own line.

This example adds two lines ( [myconfig] and widget=True ) to the buffer:

?
a
[myconfig]
widget=True
.

After a terminating dot, ed returns to command mode.

Save the buffer to disk

Once you're happy with your text, you can write the buffer to a file using the w command followed by the destination file's name:

?
w myconfig.txt
23

As confirmation, it outputs the number of characters written to the file.

Read an ed file

You will probably use ed to edit existing config files more often than you use it to write new text files from scratch. To load a file into the buffer, enter ed followed by the name of the file you want to load:

$ ed myfile.txt

From within ed , you can open an existing file into the buffer using the r command:

?
r /etc/myconfig.txt

[ Download the Linux commands cheat sheet . ]

View the buffer

To see all lines in the buffer, type ,p and then press Return :

?
,p
[myconfig]
widget=True

To see just a specific line, type the line number:

?
1
[myconfig]
2
widget=True

Edit the buffer

To edit a file, first load it in the buffer:

$ ed myconfig.txt
,p
[myconfig]
widget=True
foo=bar
openssl=libssl

To change the word "True" to "False" in the first setting of this file, select the line you want to target (2) and then invoke the search function by entering s followed by the replacement term:

?
2
widget=True
s/True/False/
2
widget=False

To target another line, use a different line number and search terms:

? 
4
openssl=libssl
s/libssl/libgnutls/
s/openssl/ssl/

View the edits you've made to the buffer using the ,p command:

[myconfig]
widget=True
foo=bar
ssl=libgnutls

You haven't written the buffer back to the file yet, so the altered lines exist only in memory. To save your changes back into the file, use the w command:

w myfile.txt
45

Clear the buffer

To start a new document or load one into a clean environment, you must clear out the buffer. The c command clears the buffer, which you can verify using the print command ( ,p ):

c
,p

Quit ed

There are two common ways to end an ed session: you can press Ctrl+D or you can type the q command. This doesn't give you a chance to save your buffer, so make sure you've written data you want to keep out to a file!

Get to know ed

If nothing else, learning ed is a powerful safeguard against getting left without a text editor when your system is in a state of recovery and you're left with only the most basic toolset. This happened to me once, and I was able to fix an errant configuration file only because I had just enough recollection of using ed in a Linux course I'd taken at a community center long ago.

It's true that ed might be the last resort, but it's nice to know what to do with the command when it's your one and only choice. And even if you don't anticipate needing ed (even in an emergency) it's a fun command to explore and gives you a good understanding of how tools like vim and sed came about. Use info ed to view the full manual to learn more.

CISA warns Oracle Identity Manager RCE flaw is being actively exploited

Bleeping Computer
www.bleepingcomputer.com
2025-11-21 23:50:27
The U.S. Cybersecurity & Infrastructure Security Agency (CISA) is warning government agencies to patch an Oracle Identity Manager tracked as CVE-2025-61757 that has been exploited in attacks, potentially as a zero-day. [...]...
Original Article

Oracle

The U.S. Cybersecurity & Infrastructure Security Agency (CISA) is warning government agencies to patch an Oracle Identity Manager tracked as CVE-2025-61757 that has been exploited in attacks, potentially as a zero-day.

CVE-2025-61757 is a pre-authentication RCE vulnerability in Oracle Identity Manager, discovered and disclosed by Searchlight Cyber analysts Adam Kues and Shubham Shahflaw.

The flaw stems from an authentication bypass in Oracle Identity Manager's REST APIs, where a security filter can be tricked into treating protected endpoints as publicly accessible by appending parameters like ?WSDL or ;.wadl to URLpaths.

Wiz

Once unauthenticated access is gained, attackers can reach a Groovy script, which is a compilation endpoint that does not typically execute a script. However, it can be abused to run malicious code at compile time through Groovy's annotation-processing features.

This chain of flaws enabled the researchers to achieve pre-authentication remote code execution on affected Oracle Identity Manager instances.

The flaw was fixed as part of Oracle's October 2025 security updates , released on October 21.

Yesterday, Searchlight Cyber released a technical report detailing the flaw and providing all the information required to exploit it.

"Given the complexity of some previous Oracle Access Manager vulnerabilities, this one is somewhat trivial and easily exploitable by threat actors," warned the researchers .

CVE-2025-61757 exploited in attacks

Today, CISA has added the Oracle CVE-2025-61757 vulnerability to its Known Exploited Vulnerabilities (KEV) catalog and given Federal Civilian Executive Branch (FCEB) agencies until December 12 to patch the flaw as mandated by the Binding Operational Directive (BOD) 22-01.

"This type of vulnerability is a frequent attack vector for malicious cyber actors and poses significant risks to the federal enterprise," warned CISA.

While CISA has not shared details of how the flaw was exploited, Johannes Ullrich, the Dean of Research for SANS Technology Institute, warned yesterday that the flaw may have been exploited as a zero-day as early as August 30.

"This URL was accessed several times between August 30th and September 9th this year, well before Oracle patched the issue," explained Ullrich in an ISC Handler Diary .

"There are several different IP addresses scanning for it, but they all use the same user agent, which suggests that we may be dealing with a single attacker."

According to Ullrich, the threat actors issued HTTP POST requests to the following endpoints, which match the exploit shared by Searchlight Cyber.

/iam/governance/applicationmanagement/templates;.wadl
/iam/governance/applicationmanagement/api/v1/applications/groovyscriptstatus;.wadl

The researcher says the attempts came from three different IP addresses, 89.238.132[.]76, 185.245.82[.]81, 138.199.29[.]153, but all used the same browser user agent, which corresponds to Google Chrome 60 on Windows 10.

BleepingComputer contacted Oracle to ask whether they have detected the flaw exploited in attacks, and will update the story if we get a response.

Wiz

Secrets Security Cheat Sheet: From Sprawl to Control

Whether you're cleaning up old keys or setting guardrails for AI-generated code, this guide helps your team build securely from the start.

Get the cheat sheet and take the guesswork out of secrets management.

I learned Vulkan and wrote a small game engine with it

Hacker News
edw.is
2025-11-21 23:28:40
Comments...
Original Article

tl;dr : I learned some Vulkan and made a game engine with two small game demos in 3 months.

The code for the engine and the games can be found here: https://github.com/eliasdaler/edbr


This article documents my experience of learning Vulkan and writing a small game/engine with it. It took me around 3 months to do it without any previous knowledge of Vulkan (I had previous OpenGL experience and some experience with making game engines, though).

The engine wasn’t implemented as a general purpose engine, which is probably why it took me a few months (and not years) to achieve this. I started by making a small 3D game and separated reusable parts into the “engine” afterwards. I can recommend everyone to follow the same process to not get stuck in the weeds (see “Bike-shedding” section below for more advice).

Preface

I’m a professional programmer, but I’m self-taught in graphics programming. I started studying graphics programming around 1.5 years ago by learning OpenGL and writing a 3D engine in it.

The engine I wrote in Vulkan is mostly suited for smaller level-based games. I’ll explain things which worked for me, but they might not be the most efficient. My implementation would probably still be a good starting point for many people.

Hopefully, this article will help make some things about Vulkan clearer to you. But you also need to be patient. It took me months to implement what I have today and I did it by cutting corners in many places. But if a self-taught programmer like me can build something with Vulkan, then so can you!

Learning graphics programming

This is a very high level overview of how I learned some graphics programming myself. If there’s interest, I might write another article with more resources and helpful guidelines.

If you haven’t done any graphics programming before, you should start with OpenGL. It’s much easier to learn it and not get overwhelmed by all the complexity that Vulkan has. A lot of your OpenGL and graphics programming knowledge will be useful when you start doing things with Vulkan later.

Ideally, you should at least get a textured model displayed on the screen with some simple Blinn-Phong lighting. I can also recommend doing some basic shadow mapping too, so that you learn how to render your scene from a different viewpoint and to a different render target, how to sample from depth textures and so on.

I can recommend using the following resources to learn OpenGL:

Sadly, most OpenGL resources don’t teach the latest OpenGL 4.6 practices. They make writing OpenGL a lot more enjoyable. If you learn them, transitioning to Vulkan will be much easier (I only learned about OpenGL 3.3 during my previous engine development, though, so it’s not a necessity).

Here are some resources which teach you the latest OpenGL practices:

It’s also good to have some math knowledge, especially linear algebra: how to work with vectors, transformation matrices and quaternions. My favorite book about linear algebra/math is 3D Math Primer for Graphics and Game Development by F. Dunn and I. Parbery . You don’t need to read it all in one go - use it as a reference if some math in the OpenGL resources above doesn’t make sense to you.

Bike-shedding and how to avoid it

https://en.wikipedia.org/wiki/Law_of_triviality

Ah, bike-shedding… Basically, it’s a harmful pattern of overthinking and over-engineering even the simplest things. It’s easy to fall into this trap when doing graphics programming ( especially when doing Vulkan since you need to make many choices when implementing an engine with it).

  • Always ask yourself “Do I really need this?”, “Will this thing ever become a bottleneck?”.
  • Remember that you can always rewrite any part of your game/engine later.
  • Don’t implement something unless you need it right now . Don’t think “Well, a good engine needs X, right…?”.
  • Don’t try to make a general purpose game engine. It’s probably even better to not think about “the engine” at first and write a simple game.
  • Make a small game first - a Breakout clone, for example. Starting your engine development by doing a Minecraft clone with multiplayer support is probably not a good idea.
  • Be wary of people who tend to suggest complicated solutions to simple problems.
  • Don’t look too much at what other people do. I’ve seen many over-engineered engines on GitHub - sometimes they’re that complex for a good reason (and there are years of work behind them). But you probably don’t need most of that complexity, especially for simpler games.
  • Don’t try to make magical wrappers around Vulkan interfaces prematurely, especially while you’re still learning Vulkan.

Get it working first. Leave “TODO”/“FIXME” comments in some places. Then move on to the next thing. Try to fix “TODO”/“FIXME” places only when they really become problematic or bottleneck your performance. You’ll be surprised to see how many things won’t become a problem at all.

Some of this advice only applies when you’re working alone on a hobby project. Of course, it’s much harder to rewrite something from scratch when others start to depend on it and a “temp hack” becomes a fundamental part of the engine which is very hard to change without breaking many things.

Why Vulkan?

Ask yourself if you need to learn a graphics API at all. If your main goal is to make a game as soon as possible, then you might be better off using something like Godot or Unreal Engine.

However, there’s nothing wrong with reinventing the wheel or doing something from scratch. Especially if you do it just for fun, to get into graphics programming or to get an in-depth knowledge about how something works.

The situation with graphic APIs in 2024 is somewhat complicated. It all depends on the use case: DirectX seems like the most solid choice for most AAA games. WebGL or WebGPU are the only two choices for doing 3D graphics on the web. Metal is the go-to graphics API on macOS and iOS (though you can still do Vulkan there via MoltenVK).

My use case is simple: I want to make small 3D games for desktop platforms (Windows and Linux mostly). I also love open source technology and open standards. So, it was a choice between OpenGL and Vulkan for me.

OpenGL is a good enough choice for many small games. But it’s very unlikely that it’ll get new versions in the future (so you can’t use some newest GPU capabilities like ray tracing), it’s deprecated on macOS and its future is uncertain.

WebGPU was also a possible choice. Before learning Vulkan, I learned some of it . It’s a pretty solid API, but I had some problems with it:

  • It’s still not stable and there’s not a lot of tutorials and examples for it. This tutorial is fantastic, though.
  • WGSL is an okay shading language, but I just find its syntax not as pleasant as GLSL’s (note that you can write in GLSL and then load compiled SPIR-V on WebGPU native).
  • On desktop, it’s essentially a wrapper around other graphic APIs (DirectX, Vulkan, Metal).This introduces additional problems for me:
    • It can’t do things some things that Vulkan or DirectX can do.
    • It has more limitations than native graphic APIs since it needs to behave similarly between them.
    • RenderDoc captures become confusing as they differ between the platforms (you can get DirectX capture on Windows and Vulkan capture on Linux) and you don’t have 1-to-1 mapping between WebGPU calls and native API calls.
    • Using Dawn and WGPU feels like using bgfx or sokol. You don’t get the same degree of control over the GPU and some of the choices/abstractions might not be the most pleasant for you.
  • No bindless textures (WIP discussion here ).
  • No push constants (WIP discussion here ).

Still, I think that WebGPU is a better API than OpenGL/WebGL and can be more useful to you than Vulkan in some use cases:

  • Validation errors are much better than in OpenGL/WebGL and not having global state helps a lot.
  • It’s also kind of similar to Vulkan in many things, so learning a bit of it before diving into Vulkan also helped me a lot.
  • It requires a lot less boilerplate to get things on the screen (compared to Vulkan).
  • You don’t have to deal with explicit synchronization which makes things much simpler.
  • You can make your games playable inside the browser.

Learning Vulkan

Learning Vulkan seemed like an impossible thing for me previously. It felt like you needed to have many years of AAA game graphics programming experience to be able to do things in it. You also hear people saying “you’re basically writing a graphics driver when writing in Vulkan” which also made Vulkan sounds like an incredibly complicated thing.

I have also checked out some engines written in Vulkan before and was further demotivated by seeing tons of scary abstractions and files named like GPUDevice.cpp or GPUAbstraction.cpp which had thousands of lines of scary C++ code.

The situation has changed over the years. Vulkan is not as complicated as it was before. First of all, Khronos realized that some parts of Vulkan were indeed very complex and introduced some newer features which made many things much simpler (for example, dynamic rendering). Secondly, some very useful libraries which reduce boilerplate were implemented. And finally, there are a lot of fantastic resources which make learning Vulkan much easier than it was before.

The best Vulkan learning resource which helped me get started was vkguide . If you’re starting from scratch, just go through it all (you might stop at “GPU driver rendering” chapter at first - many simple games probably won’t need this level of complexity)

Vulkan Lecture Series by TU Wien also nicely teaches Vulkan basics (you can probably skip “Real-Time Ray Tracing” chapter for now). I especially found a lecture on synchronization very helpful.

Here are some more advanced Vulkan books that also helped me:

Here’s the result of my first month of learning Vulkan:

By this point I had:

  • glTF model loading
  • Compute skinning
  • Frustum culling
  • Shadow mapping and cascaded shadow maps

Of course, doing it for the 3rd time (I had it implemented it all in OpenGL and WebGPU before) certainly helped. Once you get to this point, Vulkan won’t seem as scary anymore.

Let’s see how the engine works and some useful things I learned.

Engine overview and frame analysis

https://github.com/eliasdaler/edbr

My engine is called EDBR (Elias Daler’s Bikeshed Engine) and was initially started as a project for learning Vulkan. It quickly grew into a somewhat usable engine which I’m going to use for my further projects.

At the time of writing this article, the source code line counts are as follows:

  • Engine itself: 19k lines of code
    • 6.7k LoC related to graphics,
    • 2k LoC are light abstractions around Vulkan
  • 3D cat game: 4.6k LoC
  • 2D platformer game: 1.2k LoC

I copy-pasted some non-graphics related stuff from my previous engine (e.g. input handling and audio system) but all of the graphics and many other core systems were rewritten from scratch. I feel like it was a good way to do it instead of trying to cram Vulkan into my old OpenGL abstractions.

You can follow the commit history which shows how I started from clearing the screen, drawing the first triangle, drawing a textured quad and so on. It might be easier to understand the engine when it was simpler and smaller.

Let’s see how this frame in rendered:

Most of the steps will be explained in more detail below.
  • Skinning

First, models with skeletal animations are skinned in the compute shader. The compute shader takes unskinned mesh and produces a buffer of vertices which are then used instead of the original mesh in later rendering steps. This allows me to treat static and skinned meshes similarly in shaders and not do skinning repeatedly in different rendering steps.

  • CSM (Cascaded Shadow Mapping)

I use a 4096x4096 depth texture with 3 slices for cascaded shadow mapping. The first slice looks like this:

  • Geometry + shading

All the models are drawn and shading is calculated using the shadow map and light info. I use a PBR model which is almost identical to the one described in Physically Based Rendering in Filament . The fragment shader is quite big and does calculation for all the lights affecting the drawn mesh in one draw call:

Everything is drawn into a multi-sampled texture. Here’s how it looks after resolve:

(Open the previous two screenshots in the next tab and flip between the tabs to see the difference more clearly)

  • Depth resolve

Depth resolve step is performed manually via a fragment shader. I just go through all the fragments of multi-sample depth texture and write the minimum value into the non-MS depth texture (it’ll be useful in the next step).

  • Post FX

Some post FX is applied - right now it’s only depth fog (I use “depth resolve” texture from the previous step here), afterwards tone-mapping and bloom will also be done here.

  • UI

Dialogue UI is drawn. Everything is done in one draw call (more is explained in “Drawing many sprites” section)

And that’s it! It’s pretty basic right now and would probably become much more complex in the future (see “Future work” section).

General advice

There are a couple of libraries which greatly improve the experience of writing Vulkan. Most of them are already used in vkguide, but I still want to highlight how helpful they were to me.

vk-bootstrap simplifies a lot of Vulkan boilerplate: physical device selection, swapchain creation and so on.

I don’t like big wrappers around graphic APIs because they tend to be very opinionated. Plus, you need to keep a mental map of “wrapper function vs function in the API spec” in your head at all times.

Thankfully, vk-bootstrap is not like this. It mostly affects the initialization step of your program and doesn’t attempt to be a wrapper around every Vulkan function.

When I was learning Vulkan, I started doing Vulkan from scratch, without using any 3rd party libraries. Replacing big amounts of the initialization code with vk-bootstrap was a joy. It’s really worth it.

I’ll be honest, I used VMA without even learning about how to allocate memory in Vulkan manually. I read about it in the Vulkan spec later - I’m glad that I didn’t have to do it on my own.

  • volk

Volk was very useful for me for simplifying extension function loading. For example, if you want to use very useful vkSetDebugUtilsObjectNameEXT for setting debug names for your objects (useful for RenderDoc captures and validation errors), you’ll need to do this if you don’t use volk:

// store this pointer somewhere
PFN_vkSetDebugUtilsObjectNameEXT pfnSetDebugUtilsObjectNameEXT;

// during your game init
pfnSetDebugUtilsObjectNameEXT = (PFN_vkSetDebugUtilsObjectNameEXT)
    vkGetInstanceProcAddr(instance, "vkSetDebugUtilsObjectNameEXT");

// and finally in your game code
pfnSetDebugUtilsObjectNameEXT(device, ...);

With volk, all the extensions are immediately loaded after you call volkInitialize and you don’t need to store these pointers everywhere. You just include volk.h and call vkSetDebugUtilsObjectNameEXT - beautiful!

GfxDevice abstraction

I have a GfxDevice class which encapsulates most of the commonly used functionality and stores many objects that you need for calling Vulkan functions ( VkDevice , VkQueue and so on). A single GfxDevice instance is created on the startup and then gets passed around.

It handles:

  • Vulkan context initialization.
  • Swapchain creation and management.
  • beginFrame returns a new VkCommandBuffer which is later used in all the drawing steps.
  • endFrame does drawing to the swapchain and does sync between the frames.
  • Image creation and loading textures from files.
  • Buffer creation.
  • Bindless descriptor set management (see “Bindless descriptors” section below).

That’s… a lot of things. However, it’s not that big: GfxDevice.cpp is only 714 lines at the time of writing this article. It’s more convenient to pass one object into the function instead of many ( VkDevice , VkQueue , VmaAllocator and so on).

Handling shaders

In Vulkan, you can use any shading language which compiles to SPIR-V - that means that you can use GLSL, HLSL and others. I chose GLSL because I already knew it from my OpenGL experience.

You can pre-compile your shaders during the build step or compile them on the fly. I do it during the build so that my shader loading runtime code is simpler. I also don’t have an additional runtime dependency on the shader compiler. Also, shader errors are detected during the build step and I don’t get compile errors during the runtime.

I use glslc (from shaderc project, it’s included in Vulkan SDK) which allows you to specify a DEPFILE in CMake which is incredibly useful when you use shader includes. If you change a shader file, all files which include it are recompiled automatically. Without the DEPFILE , CMake won’t be able to see which files shader files need to be recompiled and will only recompile the file which was changed.

My CMake script for building shaders looks like this:

function (target_shaders target shaders)
    set(SHADERS_BUILD_DIR "${CMAKE_CURRENT_BINARY_DIR}/shaders")
    file(MAKE_DIRECTORY "${SHADERS_BUILD_DIR}")
    foreach (SHADER_PATH ${SHADERS})
        get_filename_component(SHADER_FILENAME "${SHADER_PATH}" NAME)
        set(SHADER_SPIRV_PATH "${SHADERS_BUILD_DIR}/${SHADER_FILENAME}.spv")
        set(DEPFILE "${SHADER_SPIRV_PATH}.d")
        add_custom_command(
          COMMENT "Building ${SHADER_FILENAME}"
          OUTPUT "${SHADER_SPIRV_PATH}"
          COMMAND ${GLSLC} "${SHADER_PATH}" -o "${SHADER_SPIRV_PATH}" -MD -MF ${DEPFILE} -g
          DEPENDS "${SHADER_PATH}"
          DEPFILE "${DEPFILE}"
        )
        list(APPEND SPIRV_BINARY_FILES ${SHADER_SPIRV_PATH})
    endforeach()

    set(shaders_target_name "${target}_build_shaders")
    add_custom_target(${shaders_target_name}
      DEPENDS ${SPIRV_BINARY_FILES}
    )
    add_dependencies(${target} ${shaders_target_name})
endfunction()

and then in the main CMakeLists file:

set(SHADERS
    skybox.frag
    skinning.comp
    ... // etc
)

# prepend shaders directory path
get_target_property(EDBR_SOURCE_DIR edbr SOURCE_DIR)
set(EDBR_SHADERS_DIR "${EDBR_SOURCE_DIR}/src/shaders/")
list(TRANSFORM SHADERS PREPEND "${EDBR_SHADERS_DIR}")

target_shaders(game ${SHADERS})

Now, when you build a game target, shaders get built automatically and the resulting SPIR-V files are put into the binary directory.

Push constants, descriptor sets and bindless descriptors

Passing data to shaders in OpenGL is much simpler than it is in Vulkan. In OpenGL, you could just do this:

In shader:

uniform float someFloat;

In C++ code:

const auto loc = glGetUniformLocation(shader, "someFloat");
glUseProgram(shader);
glUniform1f(loc, 42.f);

You can also use explicit uniform location like this.

In shader:

layout(location = 20) uniform float someFloat;

In code:

const auto loc = 20;
glUniform1f(loc, 42.f);

In Vulkan, you need to group your uniforms into “descriptor sets”:

// set 0
layout (set = 0, binding = 0) uniform float someFloat;
layout (set = 0, binding = 1) uniform mat4 someMatrix;
// set 1
layout (set = 1, binding = 0) uniform float someOtherFloat;
... // etc.

Now, this makes things a lot more complicated, because you need to specify descriptor set layout beforehand, use descriptor set pools and allocate descriptor sets with them, do the whole VkWriteDescriptorSet + vkUpdateDescriptorSets thing, call vkCmdBindDescriptorSets for each descriptor set and so on.

I’ll explain later how I avoided using descriptor sets by using bindless descriptors and buffer device access. Basically, I only have one “global” descriptor set for bindless textures and samplers, and that’s it. Everything else is passed via push constants which makes everything much easier to handle.

Pipeline pattern

I separate drawing steps into “pipeline” classes.

Most of them look like this:

class PostFXPipeline {
public:
    void init(GfxDevice& gfxDevice, VkFormat drawImageFormat);
    void cleanup(VkDevice device);

    void draw(
        VkCommandBuffer cmd,
        GfxDevice& gfxDevice,
        const GPUImage& drawImage,
        const GPUImage& depthImage,
        const GPUBuffer& sceneDataBuffer);

private:
    VkPipelineLayout pipelineLayout;
    VkPipeline pipeline;

    struct PushConstants {
        VkDeviceAddress sceneDataBuffer;
        std::uint32_t drawImageId;
        std::uint32_t depthImageId;
    };
};
  • init loads needed shaders and initializes pipeline and pipelineLayout :
void PostFXPipeline::init(GfxDevice& gfxDevice, VkFormat drawImageFormat)
{
    const auto& device = gfxDevice.getDevice();

    const auto pcRange = VkPushConstantRange{
        .stageFlags = VK_SHADER_STAGE_FRAGMENT_BIT,
        .offset = 0,
        .size = sizeof(PushConstants),
    };

    const auto layouts = std::array{gfxDevice.getBindlessDescSetLayout()};
    const auto pushConstantRanges = std::array{pcRange};
    pipelineLayout = vkutil::createPipelineLayout(device, layouts, pushConstantRanges);

    const auto vertexShader =
        vkutil::loadShaderModule("shaders/fullscreen_triangle.vert.spv", device);
    const auto fragShader =
        vkutil::loadShaderModule("shaders/postfx.frag.spv", device);
    pipeline = PipelineBuilder{pipelineLayout}
                   .setShaders(vertexShader, fragShader)
                   .setInputTopology(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST)
                   .setPolygonMode(VK_POLYGON_MODE_FILL)
                   .disableCulling()
                   .setMultisamplingNone()
                   .disableBlending()
                   .setColorAttachmentFormat(drawImageFormat)
                   .disableDepthTest()
                   .build(device);
    vkutil::addDebugLabel(device, pipeline, "postFX pipeline");

    vkDestroyShaderModule(device, vertexShader, nullptr);
    vkDestroyShaderModule(device, fragShader, nullptr);
}

The init function is usually called once during the engine initialization. PipelineBuilder abstraction is described in vkguide here . I modified it a bit to use the Builder pattern to be able to chain the calls.

  • cleanup does all the needed cleanup. It usually simply destroys the pipeline and its layout:
void PostFXPipeline::cleanup(VkDevice device)
{
    vkDestroyPipeline(device, pipeline, nullptr);
    vkDestroyPipelineLayout(device, pipelineLayout, nullptr);
}
  • draw is called each frame and all the needed inputs are passed as arguments. It’s assumed that the sync is performed outside of the draw call (see “Synchronization” section below). Some pipelines are only called once per frame - some either take std::vector of objects to draw or are called like this:
for (const auto& mesh : meshes) {
    somePipeline.draw(cmd, gfxDevice, mesh, ...);
}

The typical draw function looks like this:

void PostFXPipeline::draw(
    VkCommandBuffer cmd,
    GfxDevice& gfxDevice,
    const GPUImage& drawImage,
    const GPUImage& depthImage,
    const GPUBuffer& sceneDataBuffer)
{
    // Bind the pipeline
    vkCmdBindPipeline(cmd, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);

    // Bind the bindless descriptor set
    gfxDevice.bindBindlessDescSet(cmd, pipelineLayout);

    // Handle push constants
    const auto pcs = PushConstants{
        // BDA - explained below
        .sceneDataBuffer = sceneDataBuffer.address,
        // bindless texture ids - no need for desc. sets!
        // explained below
        .drawImageId = drawImage.getBindlessId(),
        .depthImageId = depthImage.getBindlessId(),
    };
    vkCmdPushConstants(
        cmd, pipelineLayout, VK_SHADER_STAGE_FRAGMENT_BIT, 0, sizeof(PushConstants), &pcs);

    // Finally, do some drawing. Here we're drawing a fullscreen triangle
    // to do a full-screen effect.
    vkCmdDraw(cmd, 3, 1, 0, 0);
}

Note another thing: it’s assumed that draw is called between vkCmdBeginRendering and vkCmdEndRendering - the render pass itself doesn’t care what texture it renders to - the caller of draw is responsible for that. It makes things simpler and allows you to do several draws to the same render target, e.g.:

// handy wrapper for creating VkRenderingInfo
const auto renderInfo = vkutil::createRenderingInfo({
    .renderExtent = drawImage.getExtent2D(),
    .colorImageView = drawImage.imageView,
    .colorImageClearValue = glm::vec4{0.f, 0.f, 0.f, 1.f},
    .depthImageView = depthImage.imageView,
    .depthImageClearValue = 0.f,
    // for MSAA
    .resolveImageView = resolveImage.imageView,
});

vkCmdBeginRendering(cmd, &renderInfo.renderingInfo);

// draw meshes
for (const auto& mesh : meshesToDraw) {
    meshPipeline.draw(cmd, gfxDevice, mesh, ...);
}
// draw sky
skyboxPipeline.draw(cmd, gfxDevice, camera);

vkCmdEndRendering(cmd);
I use VK_KHR_dynamic_rendering everywhere. I don’t use Vulkan render passes and subpasses at all. I’ve heard that they’re more efficient on tile-based GPUs, but I don’t care about mobile support for now. VK_KHR_dynamic_rendering just makes everything much easier.

Using programmable vertex pulling (PVP) + buffer device address (BDA)

I have one vertex type for all the meshes. It looks like this:

struct Vertex {
    vec3 position;
    float uv_x;
    vec3 normal;
    float uv_y;
    vec4 tangent;
};
Of course, you can greatly optimize it using various methods, but it’s good enough for me for now. The uv_x / uv_y separation comes from vkguide - I think it’s a nice idea to get good alignment and not waste any bytes

The vertices are accessed in the shader like this:

layout (buffer_reference, std430) readonly buffer VertexBuffer {
    Vertex vertices[];
};

layout (push_constant, scalar) uniform constants
{
    VertexBuffer vertexBuffer;
    ... // other stuff
} pcs;

void main()
{
    Vertex v = pcs.vertexBuffer.vertices[gl_VertexIndex];
    ...
}

PVP frees you from having to define vertex format (no more VAOs like in OpenGL or VkVertexInputBindingDescription + VkVertexInputAttributeDescription in Vulkan). BDA also frees you from having to bind a buffer to a descriptor set - you just pass an address to your buffer which contains vertices in push constants and that’s it.

Also note the scalar layout for push constants. I use it for all the buffers too. Compared to “std430” layout, it makes alignment a lot more easy to handle - it almost works the same as in C++ and greatly reduces the need for “padding” members in C++ structs.

Bindless descriptors

Textures were painful to work with even in OpenGL - you had “texture slots” which were awkward to work with. You couldn’t just sample any texture from the shader if it wasn’t bound to a texture slot beforehand. ARB_bindless_texture changed that and made many things easier.

Vulkan doesn’t have the exact same functionality, but it has something similar. You can create big descriptor sets which look like this:

// bindless.glsl
layout (set = 0, binding = 0) uniform texture2D textures[];
...
layout (set = 0, binding = 1) uniform sampler samplers[];

You’ll need to maintain a list of all your textures using some “image manager” and when a new texture is loaded, you need to insert it into the textures array. The index at which you inserted it becomes a bindless “texture id” which then can be used to sample it in shaders. Now you can pass these ids in your push constants like this:

layout (push_constant, scalar) uniform constants
{
  uint textureId;
  ...
} pcs;

and then you can sample your texture in the fragment shader like this:

// bindless.glsl
#define NEAREST_SAMPLER_ID 0
...

vec4 sampleTexture2DNearest(uint texID, vec2 uv) {
    return texture(nonuniformEXT(sampler2D(textures[texID], samplers[NEAREST_SAMPLER_ID])), uv);
}

// shader.frag
vec4 color = sampleTexture2DNearest(pcs.textureId, inUV);

Two things to note:

  1. I chose separate image samplers so that I could sample any texture using different samplers. Common samplers (nearest, linear with anisotropy, depth texture samplers) are created and put into samplers array on the startup.
  2. The wrapper function makes the process of sampling a lot more convenient.
The placement of nonuniformEXT is somewhat tricky and is explained very well here .

I use bindless ids for the mesh material buffer which looks like this:

struct MaterialData {
    vec4 baseColor;
    vec4 metallicRoughnessEmissive;
    uint diffuseTex;
    uint normalTex;
    uint metallicRoughnessTex;
    uint emissiveTex;
};

layout (buffer_reference, std430) readonly buffer MaterialsBuffer {
    MaterialData data[];
} materialsBuffer;

Now I can only pass material ID in my push constants and then sample texture like this in the fragment shader:

MaterialData material = materials[pcs.materialID];
vec4 diffuse = sampleTexture2DLinear(material.diffuseTex, inUV);
...

Neat! No more bulky descriptor sets, just one int per material in the push constants.

You can also put different texture types into the same set like this (this is needed for being able to access textures of types other than texture2D ):

layout (set = 0, binding = 0) uniform texture2D textures[];
layout (set = 0, binding = 0) uniform texture2DMS texturesMS[];
layout (set = 0, binding = 0) uniform textureCube textureCubes[];
layout (set = 0, binding = 0) uniform texture2DArray textureArrays[];

And here’s how you can sample textureCube with a linear sampler (note that we use textureCubes here instead of textures ):

vec4 sampleTextureCubeLinear(uint texID, vec3 p) {
    return texture(nonuniformEXT(samplerCube(textureCubes[texID], samplers[NEAREST_SAMPLER_ID])), p);
}

Here’s a very good article on using bindless textures in Vulkan:

https://jorenjoestar.github.io/post/vulkan_bindless_texture/

Handling dynamic data which needs to be uploaded every frame

I find it useful to pre-allocate big arrays of things and push stuff to them in every frame. Basically, you can pre-allocate an array of N structs (or matrices) and then start at index 0 at each new frame and push things to it from the CPU. Then, you can access all these items in your shaders. For example, I have all joint matrices stored in one big mat4 array and the skinning compute shader accesses joint matrices of a particular mesh using start index passed via push constants (more about it will be explained later).

Here are two ways of doing this:

    1. Have N buffers on GPU and swap between them.

vkguide explains the concept of “in flight” frames pretty well. To handle this parallelism properly, you need to have one buffer for the “currently drawing” frame and one buffer for “currently recording new drawing commands” frame to not have races. (If you have more frames in flight, you’ll need to allocate more than 2 buffers)

This means that you need to preallocate 2 buffers on GPU. You write data from CPU to GPU to the first buffer during the first frame. While you record the second frame, GPU reads from the first buffer while you write new data to the second buffer. On the third frame, GPU reads from the second buffer and you write new info to the first buffer… and so on.

    1. One buffer on GPU and N “staging” buffers on CPU

This might be useful if you need to conserve some memory on the GPU.

Let’s see how it works in my engine:

class NBuffer {
public:
    void init(
        GfxDevice& gfxDevice,
        VkBufferUsageFlags usage,
        std::size_t dataSize,
        std::size_t numFramesInFlight,
        const char* label);

    void cleanup(GfxDevice& gfxDevice);

    void uploadNewData(
        VkCommandBuffer cmd,
        std::size_t frameIndex,
        void* newData,
        std::size_t dataSize,
        std::size_t offset = 0);

    const GPUBuffer& getBuffer() const { return gpuBuffer; }

private:
    std::size_t framesInFlight{0};
    std::size_t gpuBufferSize{0};
    std::vector<GPUBuffer> stagingBuffers;
    GPUBuffer gpuBuffer;
    bool initialized{false};
};

void NBuffer::init(
    GfxDevice& gfxDevice,
    VkBufferUsageFlags usage,
    std::size_t dataSize,
    std::size_t numFramesInFlight,
    const char* label)
{
    ...

    gpuBuffer = gfxDevice.createBuffer(
        dataSize, usage | VK_IMAGE_USAGE_TRANSFER_DST_BIT, VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE);
    vkutil::addDebugLabel(gfxDevice.getDevice(), gpuBuffer.buffer, label);

    for (std::size_t i = 0; i < numFramesInFlight; ++i) {
        stagingBuffers.push_back(gfxDevice.createBuffer(
            dataSize, usage | VK_BUFFER_USAGE_TRANSFER_SRC_BIT, VMA_MEMORY_USAGE_AUTO_PREFER_HOST));
    }

    ...
}

Note how staging buffers are created using VMA’s PREFER_HOST flag and the “main” buffer from which we read in the shader is using the PREFER_DEVICE flag.

Here’s how new data is uploaded ( full implementation ):

void NBuffer::uploadNewData(
    VkCommandBuffer cmd,
    std::size_t frameIndex,
    void* newData,
    std::size_t dataSize,
    std::size_t offset) const
{
    assert(initialized);
    assert(frameIndex < framesInFlight);
    assert(offset + dataSize <= gpuBufferSize && "NBuffer::uploadNewData: out of bounds write");

    if (dataSize == 0) {
        return;
    }

    // sync with previous read
    ... // READ BARRIER CODE HERE

    auto& staging = stagingBuffers[frameIndex];
    auto* mappedData = reinterpret_cast<std::uint8_t*>(staging.info.pMappedData);
    memcpy((void*)&mappedData[offset], newData, dataSize);

    const auto region = VkBufferCopy2{
        .sType = VK_STRUCTURE_TYPE_BUFFER_COPY_2,
        .srcOffset = (VkDeviceSize)offset,
        .dstOffset = (VkDeviceSize)offset,
        .size = dataSize,
    };
    const auto bufCopyInfo = VkCopyBufferInfo2{
        .sType = VK_STRUCTURE_TYPE_COPY_BUFFER_INFO_2,
        .srcBuffer = staging.buffer,
        .dstBuffer = gpuBuffer.buffer,
        .regionCount = 1,
        .pRegions = &region,
    };

    vkCmdCopyBuffer2(cmd, &bufCopyInfo);

    // sync with write
    ... // WRITE BARRIER CODE HERE
}

I’d go with the first approach for most cases (more data on GPU, but no need for manual sync) unless you need to conserve GPU memory for some reason. I’ve found no noticeable difference in performance between two approaches, but it might matter if you are uploading huge amounts of data to GPU on each frame.

Destructors, deletion queue and cleanup

Now, this might be somewhat controversial… but I didn’t find much use of the deletion queue pattern used in vkguide. I don’t really need to allocated/destroy new objects on every frame.

Using C++ destructors for Vulkan object cleanup is not very convenient either. You need to wrap everything in custom classes, add move constructors and move operator= … It adds an additional layer of complexity.

In most cases, the cleanup of Vulkan objects happens in one place - and you don’t want to accidentally destroy some in-use object mid-frame by accidentally destroying some wrapper object.

It’s also harder to manage lifetimes when you have cleanup in happening in the destructor. For example, suppose you have a case like this:

struct SomeClass {
    SomeOtherClass b;

    void init() {
        ...
    }

    void cleanup() {
        ...
    }
}

If you want to cleanup SomeOtherClass resources (e.g. the instance of SomeOtherClass has a VkPipeline object) during SomeClass::cleanup , you can’t do that if the cleanup of SomeOtherClass is performed in its destructor.

Of course, you can do this:

struct SomeClass {
    std::unique_ptr<SomeOtherClass> b;

    void init() {
        b = std::make_unique<SomeOtherClass>();
        ...
    }

    void cleanup() {
        b.reset();
        ...
    }
}

… but I don’t like how it introduces a dynamic allocation and requires you to do write more code (and it’s not that much different from calling a cleanup function manually).

Right now, I prefer to clean up stuff directly, e.g.

class SkyboxPipeline {
public:
    void cleanup(VkDevice device) {
        vkDestroyPipeline(device, pipeline, nullptr);
        vkDestroyPipelineLayout(device, pipelineLayout, nullptr);
    }

private:
    VkPipelineLayout pipelineLayout;
    VkPipeline pipeline;
    ...
}

// in GameRenderer.cpp:
void GameRenderer::cleanup(VkDevice device) {
    ...
    skyboxPipeline.cleanup(device);
    ...
}

This approach is not perfect - first of all, it’s easy to forget to call cleanup function, This is not a huge problem since you get a validation error in case you forget to cleanup some Vulkan resources on shutdown:

Validation Error: [ VUID-vkDestroyDevice-device-05137 ] Object 0: handle = 0x4256c1000000005d, type = VK_OBJECT_TYPE_PIPELINE_LAYOUT; | MessageID = 0x4872eaa0 | vkCreateDevice():  OBJ ERROR : For VkDevice 0x27bd530[], VkPipelineLayout 0x4256c1000000005d[] has not been destroyed. The Vulkan spec states: All child objects created on device must have been destroyed prior to destroying device (https://vulkan.lunarg.com/doc/view/1.3.280.1/linux/1.3-extensions/vkspec.html#VUID-vkDestroyDevice-device-05137)

VMA also triggers asserts if you forget to free some buffer/image allocated with it.

I find it convenient to have all the Vulkan cleanup happening explicitly in one place. It makes it easy to track when the objects get destroyed.

Synchronization

Synchronization in Vulkan is difficult. OpenGL and WebGPU do it for you - if you read from some texture/buffer, you know that it will have the correct data and you won’t get problems with data races. With Vulkan, you need to be explicit and this is usually where things tend to get complicated.

Right now I manage most of the complexities of sync manually in one place. I separate my drawing into “passes”/pipelines (as described above) and then insert barriers between them. For example, the skinning pass writes new vertex data into GPU memory. Shadow mapping pass reads this data to render skinned meshes into the shadow map. Sync in my code looks like this:

// do skinning in compute shader
for (const auto& mesh : skinnedMeshes) {
    skinningPass.doSkinning(gfxDevice, mesh);
}

{
    // Sync skinning with CSM
    // This is a "fat" barrier and you can potentially optimize it
    // by specifying all the buffers that the next pass will read from
    const auto memoryBarrier = VkMemoryBarrier2{
        .sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER_2,
        .srcStageMask = VK_PIPELINE_STAGE_2_COMPUTE_SHADER_BIT,
        .srcAccessMask = VK_ACCESS_2_SHADER_WRITE_BIT,
        .dstStageMask = VK_PIPELINE_STAGE_2_VERTEX_SHADER_BIT,
        .dstAccessMask = VK_ACCESS_2_MEMORY_READ_BIT,
    };
    const auto dependencyInfo = VkDependencyInfo{
        .sType = VK_STRUCTURE_TYPE_DEPENDENCY_INFO,
        .memoryBarrierCount = 1,
        .pMemoryBarriers = &memoryBarrier,
    };
    vkCmdPipelineBarrier2(cmd, &dependencyInfo);
}

// do shadow mapping
shadowMappingPass.draw(gfxDevice, ...);

Of course, this can be automated/simplified using render graphs. This is something that I might implement in the future. Right now I’m okay with doing manual sync. vkconfig’s “synchronization” validation layer also helps greatly in finding sync errors.

The following resources were useful for understanding synchronization:

More implementation notes

Drawing many sprites

With bindless textures, it’s easy to draw many sprites using one draw call without having to allocate vertex buffers at all.

First of all, you can emit vertex coordinates and UVs using gl_VertexIndex in your vertex shader like this:

void main()
{
    uint b = 1 << (gl_VertexIndex % 6);
    vec2 baseCoord = vec2((0x1C & b) != 0, (0xE & b) != 0);
    ...
}

This snippet produces this set of values:

gl_VertexIndex baseCoord
0 (0,0)
1 (0,1)
2 (1,1)
3 (1,1)
4 (1,0)
5 (0,0)
Two triangles form a quad

Two triangles form a quad

All the sprite draw calls are combined into SpriteDrawBuffer which looks like this in GLSL:

struct SpriteDrawCommand {
    mat4 transform; // could potentially be mat2x2...
    vec2 uv0; // top-left uv coord
    vec2 uv1; // bottom-right uv coord
    vec4 color; // color by which texture is multiplied
    uint textureID; // sprite texture
    uint shaderID; // explained below
    vec2 padding; // padding to satisfy "scalar" requirements
};

layout (buffer_reference, scalar) readonly buffer SpriteDrawBuffer {
    SpriteDrawCommand commands[];
};

On CPU/C++ side, it looks almost the same:

struct SpriteDrawCommand {
    glm::mat4 transform;
    glm::vec2 uv0; // top-left uv coordinate
    glm::vec2 uv1; // bottom-right uv coodinate
    LinearColor color; // color by which texture is multiplied by
    std::uint32_t textureId; // sprite texture
    std::uint32_t shaderId; // explained below
    glm::vec2 padding; // padding
};

std::vector<SpriteDrawCommand> spriteDrawCommands;

I create two fixed size buffers on the GPU and then upload the contents of spriteDrawCommands (using techniques described above in the “Handling dynamic data” section).

The sprite renderer is used like this:

// record commands
renderer.beginDrawing();
{
    renderer.drawSprite(sprite, pos);
    renderer.drawText(font, "Hello");
    renderer.drawRect(...);
}
renderer.endDrawing();

// do actual drawing later:
renderer.draw(cmd, gfxDevice, ...);
The same renderer also draws text, rectangles and lines in my engine. For example, the text is just N “draw sprite” commands for a string composed of N glyphs. Solid color rectangles and lines are achieved by using a 1x1 pixel white texture and multiplying it by SpriteCommand::color in the fragment shader.

And finally, here’s how the command to do the drawing looks like inside SpriteRenderer::draw :

vkCmdDraw(cmd, 6, spriteDrawCommands.size(), 0, 0);
// 6 vertices per instance, spriteDrawCommands.size() instances in total

The complete sprite.vert looks like this:

#version 460

#extension GL_GOOGLE_include_directive : require
#extension GL_EXT_buffer_reference : require

#include "sprite_commands.glsl"

layout (push_constant) uniform constants
{
    mat4 viewProj; // 2D camera matrix
    SpriteDrawBuffer drawBuffer; // where sprite draw commands are stored
} pcs;

layout (location = 0) out vec2 outUV;
layout (location = 1) out vec4 outColor;
layout (location = 2) flat out uint textureID;
layout (location = 3) flat out uint shaderID;

void main()
{
    uint b = 1 << (gl_VertexIndex % 6);
    vec2 baseCoord = vec2((0x1C & b) != 0, (0xE & b) != 0);

    SpriteDrawCommand command = pcs.drawBuffer.commands[gl_InstanceIndex];

    gl_Position = pcs.viewProj * command.transform * vec4(baseCoord, 0.f, 1.f);
    outUV = (1.f - baseCoord) * command.uv0 + baseCoord * command.uv1;
    outColor = command.color;
    textureID = command.textureID;
    shaderID = command.shaderID;
}

All the parameters of the sprite draw command are self-explanatory, but shaderID needs a bit of clarification. Currently, I use it to branch inside the fragment shader:

...

#define SPRITE_SHADER_ID 0
#define TEXT_SHADER_ID   1

void main()
{
    vec4 texColor = sampleTexture2DNearest(textureID, inUV);

    // text drawing is performed differently...
    if (shaderID == TEXT_SHADER_ID) {
        // glyph atlas uses single-channel texture
        texColor = vec4(1.0, 1.0, 1.0, texColor.r);
    }

    if (texColor.a < 0.1) {
        discard;
    }

    outColor = inColor * texColor;
}

This allows me to draw sprites differently depending on this ID without having to change pipelines. Of course, it can be potentially bad for the performance. This can be improved by drawing sprites with the same shader ID in batches. You’ll only need to switch pipelines when you encounter a draw command with a different shader ID.

The sprite renderer is very efficient: it can draw 10 thousand sprites in just 315 microseconds.

Compute skinning

I do skinning for skeletal animation in a compute shader. This allows me to have the same vertex format for all the meshes.

Basically, I just take the mesh’s vertices (not skinned) and joint matrices and produce a new buffer of vertices which are used in later rendering stages.

Suppose you spawn three cats with identical meshes:

All three of them can have different animations. They all have an identical “input” mesh. But the “output” vertex buffer will differ between them, which means that you need to pre-allocate a vertex buffer for each instance of the mesh.

Here’s how the skinning compute shader looks like:

#version 460

#extension GL_GOOGLE_include_directive : require
#extension GL_EXT_buffer_reference : require

#include "vertex.glsl"

struct SkinningDataType {
    ivec4 jointIds;
    vec4 weights;
};

layout (buffer_reference, std430) readonly buffer SkinningData {
    SkinningDataType data[];
};

layout (buffer_reference, std430) readonly buffer JointMatrices {
    mat4 matrices[];
};

layout (push_constant) uniform constants
{
    JointMatrices jointMatrices;
    uint jointMatricesStartIndex;
    uint numVertices;
    VertexBuffer inputBuffer;
    SkinningData skinningData;
    VertexBuffer outputBuffer;
} pcs;

layout (local_size_x = 256, local_size_y = 1, local_size_z = 1) in;

mat4 getJointMatrix(int jointId) {
    return pcs.jointMatrices.matrices[pcs.jointMatricesStartIndex + jointId];
}

void main()
{
    uint index = gl_GlobalInvocationID.x;
    if (index >= pcs.numVertices) {
        return;
    }

    SkinningDataType sd = pcs.skinningData.data[index];
    mat4 skinMatrix =
        sd.weights.x * getJointMatrix(sd.jointIds.x) +
        sd.weights.y * getJointMatrix(sd.jointIds.y) +
        sd.weights.z * getJointMatrix(sd.jointIds.z) +
        sd.weights.w * getJointMatrix(sd.jointIds.w);

    Vertex v = pcs.inputBuffer.vertices[index];
    v.position = vec3(skinMatrix * vec4(v.position, 1.0));

    pcs.outputBuffer.vertices[index] = v;
}
  1. I store all joint matrices in a big array and populate it every frame (and also pass the starting index in the array for each skinned mesh, jointMatricesStartIndex ).
  2. Skinning data is not stored inside each mesh vertex, a separate buffer of num_vertices elements is used.

After the skinning is performed, all the later rendering stages use this set of vertices Thee rendering process for static and skinned meshes becomes identical, thanks to that.

Anton’s OpenGL 4 Tutorials book has the best skinning implementation guide I’ve ever read. Game Engine Architecture by Jason Gregory has nice explanations about skinning/skeletal animation math as well.

Game / renderer separation

I have a game/renderer separation which uses a simple concept of “draw commands”. In the game logic, I use entt , but the renderer doesn’t know anything about entities or “game objects”. It only knows about the lights, some scene parameters (like fog, which skybox texture to use etc) and meshes it needs to draw.

The renderer’s API looks like this in action:

void Game::generateDrawList()
{
    renderer.beginDrawing();

    // Add lights
    const auto lights = ...; // get list of all active lights
    for (const auto&& [e, tc, lc] : lights.each()) {
        renderer.addLight(lc.light, tc.transform);
    }

    // Render static meshes
    const auto staticMeshes = ...; // list of entities with static meshes
    for (const auto&& [e, tc, mc] : staticMeshes.each()) {
        // Each "mesh" can have multiple submeshes similar to how
        // glTF separates each "mesh" into "primitives".
        for (std::size_t i = 0; i < mc.meshes.size(); ++i) {
            renderer.drawMesh(mc.meshes[i], tc.worldTransform, mc.castShadow);
        }
    }

    // Render meshes with skeletal animation
    const auto skinnedMeshes = ...; // list of entities with skeletal animations
    for (const auto&& [e, tc, mc, sc] : skinnedMeshes.each()) {
        renderer.drawSkinnedMesh(
            mc.meshes, sc.skinnedMeshes, tc.worldTransform,
            sc.skeletonAnimator.getJointMatrices());
    }

    renderer.endDrawing();
}

When you call drawMesh or drawSkinnedMesh , the renderer creates a mesh draw command and puts it in std::vector<MeshDrawCommand> which are then iterated through during the drawing process. The MeshDrawCommand looks like this:


struct SkinnedMesh {
    GPUBuffer skinnedVertexBuffer;
};

struct MeshDrawCommand {
    MeshId meshId;
    glm::mat4 transformMatrix;
    math::Sphere worldBoundingSphere;

    const SkinnedMesh* skinnedMesh{nullptr};
    std::uint32_t jointMatricesStartIndex;
    bool castShadow{true};
};
  • meshId is used for looking up static meshes in MeshCache - it’s a simple std::vector of references to vertex buffers on GPU.
  • If the mesh has a skeleton, jointMatricesStartIndex is used during compute skinning and skinnedMesh->skinnedVertexBuffer is used for all the rendering afterwards (instead of meshId )
  • worldBoundingSphere is used for frustum culling.

This separation is nice because the renderer is clearly separated from the game logic. You can also do something more clever as described here if sorting draw commands becomes a bottleneck.

Scene loading and entity prefabs

I use Blender as a level editor and export it as glTF. It’s easy to place objects, colliders and lights there. Here’s how it looks like:

Writing your own level editor would probably take months (years!), so using Blender instead saved me quite a lot of time.

It’s important to mention how I use node names for spawning some objects. For example, you can see an object named Interact.Sphere.Diary selected in the screenshot above. The part before the first dot is the prefab name (in this case “Interact”). The “Sphere” part is used by the physics system to create a sphere physics body for the object (“Capsule” and “Box” can also be used, otherwise the physics shape is created using mesh vertices).

Some models are pretty complex and I don’t want to place them directly into the level glTF file as it’ll greatly increase each level’s size. I just place an “Empty->Arrows” object and name it something like “Cat.NearStore”. This will spawn “Cat” prefab and attach “NearStore” tag to it for runtime identification.

Prefabs are written in JSON and look like this:

{
  "scene": {
    "scene": "assets/models/cato.gltf"
  },
  "movement": {
    "maxSpeed": [4, 4, 4]
  },
  "physics": {
    "type": "dynamic",
    "bodyType": "virtual_character",
    "bodyParams": {
        ...
    }
  }
}

During the level loading process, if the node doesn’t have a corresponding prefab, it’s loaded as-is and its mesh data is taken from the glTF file itself (this is mostly used for static geometry). If the node has a corresponding prefab loaded, it’s created instead. Its mesh data is loaded from the external glTF file - only transform is copied from the original glTF node (the one in the level glTF file).

Once glTFX is released and the support for it is added to Blender, things might be even easier to handle as you’ll be able to reference external glTF files with it.

MSAA

Using forward rendering allowed me to easily implement MSAA. Here’s a comparison of how the game looks without AA and with MSAA on:

No AA

No AA

MSAA x8

MSAA x8

MSAA is explained well here: https://vulkan-tutorial.com/Multisampling

Here’s another good article about MSAA: https://therealmjp.github.io/posts/msaa-overview/ and potential problems you can have with it (especially with HDR and tone-mapping).

UI

My UI system was inspired by Roblox’s UI API: https://create.roblox.com/docs/ui

Basically, the UI can calculate its own layout without me having to hard code each individual element’s size and position. Basically it relies on the following concepts:

  • Origin is an anchor around which the UI element is positioned. If origin is (0, 0) , setting UI element’s position to be (x,y) will make its upper-left pixel have (x,y) pixel coordinate. If the origin is (1, 1) , then the element’s bottom-right corner will be positioned at (x, y) . If the origin is (0.5, 1) then it will be positioned using bottom-center point as the reference.
  • Relative size makes the children’s be proportional to parent’s size. If (1,1) then the child element will have the same size as the parent element. If it’s (0.5, 0.5) then it’ll have half the size of the parent. If the parent uses children’s size as a guide, then if a child has (0.5, 0.25) relative size, the parent’s width will be 2x larger and the height will be 4x larger.
  • Relative position uses parent’s size as a guide for positioning. It’s useful for centering elements, for example if you have an element with (0.5, 0.5) origin and (0.5, 0.5) relative position, it’ll be centered inside its parent element.
  • You can also set pixel offsets for both position and size separately (they’re called offsetPosition and offsetSize in my codebase).
  • You can also set a fixed size for the elements if you don’t want them to ever be resized.
  • The label/image element size is determined using its content.

Here are some examples of how it can be used to position child elements:

a) The child (yellow) has relative size (0.5, 1), relative position of (0.5, 0.5) and origin (0.5, 0.5) (alternatively, the relative position can be (0.5, 0.0) and origin at (0.5, 0.0) in this case). Its parent (green) will be two times wider, but will have the same height. The child element will be centered inside the parent.

b) The child (yellow) has origin (1, 1), fixed size (w,h) and absolute offset of (x,y) - this way, the item can be positioned relative to the bottom-right corner of its parent (green)


Let’s see how sizes and positions of UI elements are calculated ( implementation in EDBR ).

First, sizes of all elements are calculated recursively. Then positions are computed based on the previously computed sizes and specified offset positions. Afterwards all elements are drawn recursively - parent element first, then its children etc.

When calculating the size, most elements either have a “fixed” size (which you can set manually, e.g. you can set some button to always be 60x60 pixels) or their size is computed based on their content. For example, for label elements, their size is computed using the text’s bounding box. For image elements, their size equals the image size and so on.

If an element has an “Auto-size” property, it needs to specify which child will be used to calculate its size. For example, the menu nine-slice can have several text labels inside the “vertical layout” element - the bounding boxes will be calculated first, then their sizes will be summed up - then, the parent’s size is calculated.

Let’s take a look at a simple menu with bounding boxes displayed:

Here, root NineSliceElement is marked as “Auto-size”. To compute its size, it first computes the size of its child ( ListLayoutElement ). This recursively computes the sizes of each button, sums them up and adds some padding ( ListLayoutElement also makes the width of each button the same based on the maximum width in the list).

Dear ImGui and sRGB issues

I love Dear ImGui. I used it to implement many useful dev and debug tools (open the image in a new tab to see them better):

It has some problems with sRGB , though. I won’t explain it in detail, but basically if you use sRGB framebuffer, Dear ImGui will look wrong in many ways, see the comparison:

Left - naive sRGB fix for Dear ImGui, right - proper fix

Left - naive sRGB fix for Dear ImGui, right - proper fix

Left - naive sRGB fix for Dear ImGui, right - proper fix

Left - naive sRGB fix for Dear ImGui, right - proper fix

Sometimes you can see people doing hacks by doing pow(col, vec4(2.2)) with Dear ImGui’s colors but it still doesn’t work properly with alpha and produces incorrect color pickers.

I ended up writing my own Dear ImGui backend and implementing DilligentEngine’s workaround which is explained in detail here and here .

Writing it wasn’t as hard as I expected. I only need to write the rendering part, while “logic/OS interaction” part (input event processing, clipboard etc.) is still handled by default Dear ImGui SDL backend in my case.

There are some additional benefits of having my own backend:

  1. It supports bindless texture ids, so I can draw images by simply calling ImGui::Image(bindlessTextureId, ...) . Dear ImGui’s Vulkan backend requires you to “register” textures by calling ImGui_ImplVulkan_AddTexture for each texture before you can call ImGui::Image .
  2. It can properly draw linear and non-linear images by passing their format into backend (so that sRGB images are not gamma corrected twice when they’re displayed)
  3. Initializing and dealing with it is easier as it does Vulkan things in the same way as the rest of my engine.

Other stuff

There are many parts of the engine not covered there because they’re not related to Vulkan. I still feel like it’s good to mention them briefly for the sake of completion.

  • I use Jolt Physics for physics.

Integrating it into the engine was pretty easy. Right now I mostly use it for collision resolution and basic character movement.

The samples are fantastic . The docs are very good too.

I especially want to point out how incredible JPH::CharacterVirtual is. It handles basic character movement so well. I remember spending days trying to get proper slope movement in Bullet to work. With Jolt, it just worked “out of the box”.

Here’s how it basically works (explaining how it works properly would probably require me to write quite a big article):

  • You add your shapes to Jolt’s world.
  • You run the simulation.
  • You get new positions of your physics objects and use these positions to render objects in their current positions.
I implemented Jolt physics shape debug renderer using im3d

I implemented Jolt physics shape debug renderer using im3d

  • I use entt for the entity-component-system part.

It has worked great for me so far. Previously I had my own ECS implementation, but decided to experiment with a 3rd party ECS library to have less code to maintain.

The audio system is mostly based on these articles: https://indiegamedev.net/2020/02/15/the-complete-guide-to-openal-with-c-part-1-playing-a-sound/

Integrating it was very easy (read the PDF doc, it’s fantastic!) and it helped me avoid tons of bike-shedding by seeing how little time something, which I thought was “inefficient”, really took.

What I gained from switching to Vulkan

There are many nice things I got after switching to Vulkan:

  • No more global state

This makes abstractions a lot easier. With OpenGL abstractions/engines, you frequently see “shader.bind()” calls, state trackers, magic RAII, which automatically binds/unbinds objects and so on. There’s no need for that in Vulkan - it’s easy to write functions which take some objects as an input and produce some output - stateless, more explicit and easier to reason about.

  • API is more pleasant to work with overall - I didn’t like “binding” things and the whole “global state machine” of OpenGL.
  • You need to write less abstractions overall.

With OpenGL, you need to write a lot of abstractions to make it all less error-prone… Vulkan’s API requires a lot less of this, in my experience. And usually the abstractions that you write map closer to Vulkan’s “raw” functions, compared to OpenGL abstractions which hide manipulation of global state and usually call several functions (and might do some stateful things for optimization).

  • Better validation errors

Validation errors are very good in Vulkan. While OpenGL has glDebugMessageCallback , it doesn’t catch that many issues and you’re left wondering why your texture looks weird, why your lighting is broken and so on. Vulkan has more extensive validation which makes the debugging process much better.

  • Debugging in RenderDoc

I can now debug shaders in RenderDoc. It looks like this:

With OpenGL I had to output the values to some texture and color-pick them… which took a lot of time. But now I can debug vertex and fragment shaders easily.

  • More consistent experience across different GPUs and OSes.

With OpenGL, drivers on different GPUs and OSes worked differently from each other which made some bugs pop up only on certain hardware configurations. It made the process of debugging them hard. I still experienced some slight differences between different GPUs in Vulkan, but it’s much less prevalent compared to OpenGL.

  • Ability to use better shading languages in the future

GLSL is a fine shading language, but there are some new shading languages which promise to be more feature-complete, convenient and readable, for example:

I might explore them in the future and see if they offer me something that GLSL lacks.

  • More control over every aspect of the graphics pipeline.
  • Second system effect, but good

My first OpenGL engine was written during the process of learning graphics programming from scratch. Many abstractions were not that good and rewriting them with some graphics programming knowledge (and some help from vkguide) helped me implement a much cleaner system.

  • Street cred

And finally, it makes me proud to be able to say “I have a custom engine written in Vulkan and it works”. Sometimes people start thinking about you as a coding wizard and it makes me happy and proud of my work. :)

Future work

There are many things that I plan to do in the future, here’s a list of some of them:

  • Sign-distance field font support ( good article about implementing them)
  • Loading many images and generating mipmaps in parallel (or use image formats which already have mipmaps stored inside of them)
  • Bloom.
  • Volumetric fog.
  • Animation blending.
  • Render graphs.
  • Ambient occlusion.
  • Finishing the game? (hopefully…)

Overall, I’m quite satisfied with what I managed to accomplish. Learning Vulkan was quite difficult, but it wasn’t as hard as I imagined. It taught me a lot about graphics programming and modern APIs and now I have a strong foundation to build my games with.

Reverse Engineering the Miele Diagnostic Interface

Lobsters
medusalix.github.io
2025-11-21 23:17:07
Comments...
Original Article

A few weeks ago, my parents’ old Miele washing machine suddenly stopped functioning. It seemed like the machine finally required some maintenance, considering that it had been in operation for almost 20 years without issues. Disassembling the appliance revealed a number of hoses connecting the different parts of the washing machine. Upon closer inspection, several of these hoses were almost completely blocked by the detergent residue that had accumulated over the past two decades. After cleaning all hoses, the appliance initially seemed to be working fine again. However, at the end of the washing cycle, the machine didn’t start to spin up. According to multiple forum posts, this fault was most likely caused by the analog pressure sensor that determines the water level inside the drum. If the residual water doesn’t fall under a certain level during the pumping cycle, the machine aborts the current washing program. The white sensor housing can be easily spotted in the bottom right corner of the machine’s electronics board:

Analog pressure sensor mounted on PCB
Analog pressure sensor mounted on PCB

Following some quick measurements with a multimeter, I determined that the sensor was functioning correctly. However, as several Miele experts pointed out, the sensor might have to be calibrated again after taking the machine apart, requiring a proprietary Miele software that is only available to registered service technicians. Fortunately, it turned out that this specific problem was not related to the calibration but could be instead fixed by clearing the machine’s fault memory.

Even though the washing machine was now working again, I was still curious about how the pressure sensor could actually be calibrated. As far as I could tell, there were no external ports on the electronics board for programming purposes. So how does the Miele software communicate with the appliance?

This article turned out to be much longer than initially expected. Feel free to jump to individual sections if you want to skip some of the theory!

Discovering the diagnostic interface #

Online repair guides and technical documentation often mention the so-called Miele Diagnostic Utility (MDU) , a proprietary tool used by technicians to diagnose common faults on all kinds of Miele devices. While every official repair business can register on Miele’s website to get access to the service utility, its use requires very costly special hardware that has to be purchased from Miele and dedicated training sessions.

At first glance, very little information can be found online about the MDU, except for a few screenshots of the software. For illustrative purposes, I came up with the following (very rough) sketch of the graphical user interface:

User interface of the MDU (simplified sketch)
User interface of the MDU (simplified sketch)

While looking for more details of the software, I discovered this presentation (in French) about an older version of Miele’s diagnostic software and hardware, offering deeper insights into the capabilities and workings of the interface.

Diagnostic capabilities #

Judging from the contents of the presentation, the MDU can be used to read various properties from a connected appliance. This includes the software ID, model and fabrication number, operating hours and fault memory. However, the number of properties that can be queried seems to vary from model to model.

While this data might be interesting for technically inclined appliance owners, the real power of the MDU lies in the monitoring features of the software. In addition to the live status of all sensors connected to the washing machine, such as the temperature sensor, water level sensor, or motor RPM, the utility also provides an overview of the actuator status , including all heating, water control and pump relays. The selected washing program and current program phase are also displayed by the software, along with the configured options, such as prewash or increased water level.

Many Miele washing machines provide a service mode that can be accessed by turning on the machine while pressing a certain button combination on the front panel. The service options offered by this mode can also be triggered by the MDU. However, the software additionally features a calibration menu that is used to calibrate internal sensors like the analog pressure sensor that measures the water level.

Finally, the MDU also provides program updates for Miele appliances. These updates were originally intended to allow changes to the built-in washing programs, such as adjusting program cycle times or the water amount. On newer appliances, the MDU can even update the full firmware of the electronics board.

Communication hardware #

These features are highly useful for diagnostic purposes, not only for professional service technicians but also for appliance owners that would like to repair their own devices. But how does the MDU communicate with a Miele appliance? Reading through the presentation slides reveals a so-called Program Correction (PC) interface that is available on all appliances manufactured since 1996. This interface is located on the front panel of the machine, usually disguised as the check inlet indicator on washing machines or the salt missing indicator on dishwashers. The following picture clearly shows the PC interface on a Miele Softtronic W 2446 washing machine:

Front panel of Miele Softtronic W 2446 washing machine
Front panel of Miele Softtronic W 2446 washing machine

While these indicator lights normally show the operating status of the machine, they are not just regular LEDs. Instead, the red PC indicator LED also includes an infrared phototransistor, enabling bidirectional communication with the MDU software using a suitable optical communication adapter. According to a public Miele presentation , this interface is not only used for field diagnostics, but also during the development phase and end-of-line factory testing. The presentation also includes a picture of the actual surface-mount LED that is used on the adapter side, which looks very similar to the OSRAM Multi TOPLED SFH 7250 infrared emitter and phototransistor at first glance. While a dual-use indicator is clever in principle, it comes with drawbacks. When the respective indicator light is actually in use, no communication via the PC interface is possible. For this reason, Miele might have decided to switch to a dedicated PC LED indicator on newer appliances, such as their coffee machines. Due to the close proximity between the emitter and phototransistor, the communication is also limited to a relatively slow half-duplex operation.

Practical use of the MDU software requires a proprietary optical communication adapter, which has to be purchased separately from Miele. This adapter, which is also referred to as the Miele Optical Interface , consists of an interface box (codename EZI 820) and a head unit (EZI 821 or EZI 821-A) that are connected via a fiber-optic cable. The interface box features a DE-9 connector for RS-232 communication with a host PC. Newer versions of the optical interface also include a USB connector for this purpose. The head unit is then attached to the appliance through a suction cup mechanism, aligning an optical fiber with the PC indicator hole. This complete assembly and communication technique has been patented by Miele in 1995, with the original intention of allowing washing program corrections for after-sales service.

Due to the proprietary nature of the optical interface, Miele does not publish any images of the adapter unit. However, given the high cost of official hardware, these adapters often surface on auction sites with detailed pictures . Some people are even looking to buy the MDU from other sources, as the adapter is pretty much useless without the software.

Building an open source MDU #

While not many details are available online about the internals of the Miele Optical Interface, this forum user claims to have bought the unit from an eBay auction. The adapter is apparently a simple serial to infrared converter, implementing the well-known Infrared Data Association (IrDA) standard, commonly used in older laptops and embedded systems. It is based on an STM32F103 microcontroller, with all upper level protocol logic implemented by the MDU software. This is excellent news, as building an adapter would therefore only require a cheap microcontroller and an infrared emitter/detector.

In contrast to the details about the adapter unit, the proprietary protocol that is used by the MDU software is completely undocumented. However, reverse engineering the protocol would allow an open source diagnostic software to be built, which would be immensely useful for the repair community. It might also allow older Miele appliances to be integrated into home automation solutions, by building a bridge between the PC interface and existing software such as Home Assistant.

Reverse engineering the electronics #

With these goals in mind, I decided to look for salvaged electronics from old Miele appliances on eBay. More specifically, I was looking for the main circuit board of a washing machine, since experimenting on a fully assembled appliance would have posed significant electrical and mechanical hazards. As luck would have it, I managed to win the bid for a brand new Miele EDPW 206 manufactured in 2010:

Front side of Miele EDPW 206 electronics board
Front side of Miele EDPW 206 electronics board
Back side of Miele EDPW 206 electronics board
Back side of Miele EDPW 206 electronics board

This board is part of the Miele W 961 washing machine series, manufactured from 1998 to 2003, according to this forum post . The EDPW 200 label on the back side of the PCB hints at the existence of further variations of this board for other washing machines. In contrast to newer Miele appliances, the power electronics are contained on a separate PCB for this machine, making the reverse engineering process much safer.

The PCB itself is a pretty simple double-layer design, without any bigger copper fills. Ground and power traces are instead routed as separate tracks, leading to the enormous number of vias that can be seen in the previous pictures. Figuring out the connections between individual components is unfortunately pretty tedious for this reason.

Key components #

One of the central components of this PCB is a large 80-pin chip marked with MIELE 6478170 M37451MC-804FP . A quick online search for M37451 suggests that this chip is part of the Mitsubishi 740 series of 8-bit microcontrollers, which are also known as MELPS 740, according to Wikipedia . These microcontrollers were originally manufactured during the 1980s and 1990s, with a relatively simple instruction set similar to the widely known WDC 65C02. Although these parts are no longer produced today, the instruction set lives on in the newer Renesas 38000/740 microcontroller series.

The M37451MC-804FP includes an integrated mask ROM, meaning the Miele firmware is embedded directly in the chip’s die and can’t be reprogrammed after the manufacturing process. As denoted by the MC suffix, the M37451MC-804FP has a total RAM size of 512 bytes with a 24 kB mask ROM. Other features include an 8-bit ADC with 8 channels, an 2-channel 8-bit DAC and three individual 16-bit timers. Serial communication is handled by a serial I/O block that can be configured for asynchronous or synchronous operation. The chip is powered by a 5 V supply, with an operating frequency of 10 MHz. More information about the Mitsubishi microcontrollers can be found in volume two of Mitsubishi’s Single-Chip 8-bit Microcomputers Data Book .

Mitsubishi M37451 microcontroller and Microchip 93LC66BI EEPROM
Mitsubishi M37451 microcontroller and Microchip 93LC66BI EEPROM

Located right next to the microcontroller is a Microchip 93LC66BI EEPROM with a capacity of only 512 bytes. The stored data is organized in 16-bit words, which can be accessed via a Microwire interface. All configuration parameters and the current state of the running wash cycle are written to the EEPROM just before the machine is powered off. This allows the machine to resume the program once it is turned back on again. In addition to this data, the EEPROM also stores any program corrections that are applied via the PC interface.

As the water inlet, detergent processing and heating cycle are controlled by individual relays that require an input voltage higher than 5 V, the PCB also includes a Texas Instruments ULN2003A Darlington transistor array.

The water level inside the washing machine’s drum is sensed by an SPX3078D analog pressure sensor manufactured by Motorola. This sensor basically consists of a silicon diaphragm, which is used to determine the applied pressure through a Wheatstone bridge circuit. The differential output voltage is then processed by an ELMOS E210.01C. Since ELMOS provides no public documentation on this component, its exact function is unclear. However, I strongly assume it contains an operational amplifier and possibly additional signal processing circuitry. One of the pins is connected to the microcontroller’s analog input port and provides a voltage proportional to the sensed pressure.

Motorola SPX3078D pressure sensor, ELMOS E210.01C IC and TI ULN2003A transistor array
Motorola SPX3078D pressure sensor, ELMOS E210.01C IC and TI ULN2003A transistor array

Most of the indicator LEDs on the PCB are multiplexed and wired to a Motorola MC14489DW LED driver, which offers an SPI interface for the microcontroller:

Motorola MC14489DW LED driver
Motorola MC14489DW LED driver

Upon detailed inspection of the LEDs on the right side of the board, one can see that the lowest LED is quite different from the rest of the group. Looking closer reveals that this is actually a combined red LED and infrared phototransistor. This is the optical PC interface, disguised as one of the indicator lights:

Optical PC interface next to normal indicator LED
Optical PC interface next to normal indicator LED

The LED is not part of the group of multiplexed indicator LEDs and is instead wired to a pair of transistors. An NPN transistor connects the microcontroller’s UART transmit pin to the light emitter, while the phototransistor is connected to the UART receive pin via a PNP transistor.

The PC interface is therefore just a simple optical UART port, albeit limited to half-duplex communication.

Power-up troubles #

To communicate with the EDPW board via the optical interface, the PCB has to be connected to an appropriate power supply. Luckily, the EDPW 206’s technical documentation includes the complete pinout of the board’s connectors:

EDPW 206 connector pinout
EDPW 206 connector pinout

However, simply supplying the board with 5 V from a lab power supply didn’t seem to have any effect. Taking a closer look at the pinout shows that the board also expects 20 V for the U C voltage, which is connected to the ULN2003A’s common-cathode node for the integrated flyback diodes. This voltage seems to be sensed by the microcontroller through a resistive divider. Unfortunately, even with those two voltages, the EDPW didn’t seem to turn on. Further investigation revealed that the board also requires an AC zero-crossing detection signal, referred to by the term Netznulldurchgang (NND) in German. This signal is generated by an optocoupler on the power electronics board, resulting in an alternating wave based on the line frequency. Supplying a 50 Hz square wave from a Pi Pico in place of this signal finally brought the EDPW to life:

Working EDPW 206 board (with blinking red LEDs)
Working EDPW 206 board (with blinking red LEDs)

While all basic functions of the EDPW seemed to work fine, including the program selection knob and the configuration buttons, I quickly noticed that the check intake and check drain indicators were flashing. Because the check intake LED also serves as the optical interface, this issue had to be resolved before any communication was possible. I initially assumed that the analog pressure sensor was giving some incorrect readings, but further investigations ruled out this theory. Instead, this issue was related to the missing relays that would normally be connected to the board. As it turns out, the microcontroller actually checks the presence of the prewash and main wash relays by sensing the voltage on the board’s relay outputs. When the relays are connected to U C on one side, the voltage at the transistor array’s collector pins is also equal to U C . Both the prewash and main wash outputs then go to a BAV70 common cathode double diode chip that is wired to the microcontroller:

BAV70 common cathode double diode (marked as A4W)
BAV70 common cathode double diode (marked as A4W)

Connecting a 10 kOhm resistor between the pin for the main wash relay and U C therefore stops the red LEDs from blinking. With this workaround in place, the EDPW board was now fully functional.

Dumping the EEPROM data #

Before reverse engineering the PC interface, it is worth taking a closer look at the EEPROM chip. Removing the chip from the PCB and soldering it to an SOIC adapter allows its contents to be read using a CH341A EEPROM programmer. It should be noted that the adapter can’t be plugged into the socket directly, as the pinout of the 93XXX chip differs from classic 25XXX EEPROMs that this programmer is designed for.

Adapter PCB for SOIC EEPROM chips
Adapter PCB for SOIC EEPROM chips
CH341A EEPROM and flash programmer
CH341A EEPROM and flash programmer

Reading the EEPROM contents with IMSProg revealed that only 42 bytes are actually used, with almost all remaining bytes set to ff , indicating erased or unused memory:

00000000  ff ff f0 ff ff ff cc fb  b0 55 fd ff ca ce f7 db
00000010  ff ff ff d3 fd ff ff ff  ff f7 fe f8 fe 11 f8 f8
00000020  f8 f8 f8 f8 f8 fa f8 f8  f8 63 ff ff ff ff ff ff
00000030  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff
*
000001f0  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff 00

To analyze how the stored data changes under different washing program settings and conditions, the EEPROM chip was soldered to the PCB again, while also attaching a logic analyzer to monitor the Microwire interface:

EDPW board with probes connected to EEPROM chip
EDPW board with probes connected to EEPROM chip
USB logic analyzer with 8 channels
USB logic analyzer with 8 channels

When the EDPW board is powered on, the microcontroller reads the first 42 bytes from the EEPROM. As soon as either the U C voltage or the zero-crossing signal are lost, the memory contents are written back to the EEPROM.

EEPROM Microwire signal capture in PulseView
EEPROM Microwire signal capture in PulseView

After trying out different washing programs and observing the changes to the individual bytes, the full EEPROM contents can be deciphered:

00000000  00 00 0f 00 00 00 33 04  4f aa 02 00 35 31 08 24
00000010  00 00 00 2c 02 00 00 00  00 08 01 07 01 ee 07 07
00000020  07 07 07 07 07 05 07 07  07 9c 00 00 00 00 00 00
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
*
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 ff

All bytes are inverted before being written to the EEPROM by the microcontroller. The first 12 bytes store general information about the washing machine. As an example, this includes the currently running program and program phase (first and second byte) and the operating hours of the appliance (third and fourth byte). This section ends with a simple one byte checksum that is computed by summing the preceding bytes (modulo ff ) and inverting the result.

The next group of bytes encodes the fixed washing machine configuration that can only be changed by entering the programming mode. Settings such as the water type and region code are stored in this section. Another checksum is appended to these bytes, again computed over all previous bytes.

Configuration options that are chosen during the normal operation of the machine (e.g. short, prewash, spin cycle RPM, etc.) are preserved in the subsequent bytes , stored separately for each individual washing program.

The last section is mostly empty, but most likely used to further customize the behavior of washing programs as part of the program correction mechanism. Selecting the Cottons 95 °C program for example causes the microcontroller to continuously read the byte at address 40 from the EEPROM, probably checking for some bit to be set. Despite some areas being unused or unclear, this EEPROM analysis provided some valuable insights into the internal operation of the EDPW.

Analyzing the firmware #

Returning to the PC interface, I wondered whether it might be possible to extract the microcontroller’s firmware to reverse engineer the diagnostic protocol. As previously noted , the firmware is stored in a mask ROM during manufacturing. Gaining access to the ROM’s contents would therefore require the chip to be decapped, which requires special equipment and practice that I don’t have. However, according to its manual, the Mitsubishi M37451 seems to feature multiple processor modes that might allow the ROM to be dumped:

Mitsubishi M37451 processor modes
Mitsubishi M37451 processor modes

The processor is running in single-chip mode by default, as the CNV SS pin is pulled to GND on the EDPW board. Connecting CNV SS to 5 V would cause the chip to enter microprocessor mode, loading the program code from external memory. This would in theory allow the embedded firmware to be dumped, but unfortunately, access to the internal ROM is blocked in this case, as specified in the manual. This restriction, likely implemented for security reasons, is not present for the memory expansion mode, but this processor mode can only be entered by writing to a register when running in single-chip mode.

Although techniques like fault injection or voltage glitching might bypass these limitations, I decided to continue the reverse engineering process without access to the firmware.

To communicate with the EDPW’s optical interface, I connected a USB-UART adapter directly to the microcontroller’s UART pins. Lacking a proper 5 V USB-UART adapter, I used an Arduino Uno clone just for its UART capabilities. As the optical interface seemed to be very sensitive to the infrared radiation of the sunlight, I decided to disconnect the phototransistor from the UART’s receive pin:

Reverse engineering hardware setup
Reverse engineering hardware setup

Now that the hardware was set up, it was time to focus on the actual serial communication. As might be expected, the interface doesn’t send any data on its own during normal operation, so its protocol must be based on a request-response scheme. However, figuring out the exact commands that need to be sent would be extremely difficult without any details about the protocol. It is often a good idea to look for similar protocols in this situation, which might provide some clues about the general command structure. After a more elaborate online search, I found a forum post (in German) describing the Miele@home interface, which is used to add remote control functionality to Miele appliances. It provides a detailed analysis of the Miele@home communication module and its serial protocol, including a full communication log between module and appliance.

The serial interface is initially configured for a data rate of 2400 baud with 8 data bits, 1 stop bit and no parity. After a short handshake sequence, the communication switches to a speed of 9600 baud. Considering that the Miele PC interface was introduced in 1996, it doesn’t seem unlikely that first implementations were limited to 2400 baud, which would explain why the communication begins at this baud rate. The messages sent by the module and the appliance are always either 5 bytes or 1 byte long, where a single 00 byte indicates the successful reception of a valid message by the receiving side. All 5-byte messages begin with a command byte, 2 unknown bytes and a single byte which seemingly indicates the expected length of the response payload. These messages end with a simple 8-bit checksum that is computed by summing the previous bytes, similar to the EEPROM checksum discussed in the last section.

With basic knowledge of the protocol, the first couple of messages can now be analyzed in further detail:

Module -> Appliance: 11 00 00 02 13
Appliance -> Module: 00
Appliance -> Module: fb 08 03
Module -> Appliance: 00
Module -> Appliance: 21 00 00 00 21

The handshake starts with command 11 , expecting a response with a length of 02 . The request is acknowledged by the appliance, which responds with fb 08 . This response is likewise acknowledged by the module, which proceeds by sending the command 21 . The rest of the handshake continues in a similar manner until the communication switches to 9600 baud. Could this be the same protocol that is used by the PC interface?

To confirm this assumption, I tried sending the initial 11 command via the USB-UART adapter:

Logic analyzer trace of UART signals
Logic analyzer trace of UART signals

Unfortunately, this didn’t lead to any response from the PC interface. At this point, I decided to take another look at the microcontroller’s datasheet, focusing on the UART section:

Mitsubishi M37451 UART operation
Mitsubishi M37451 UART operation

Assuming that the UART is indeed configured for 2400 baud, the only remaining options that can be configured are the number of data bits, stop bits and the parity mode. At this baud rate, a combination of 8 data bits and 1 stop bit would seem to be the most likely choice. However, since the communication is based on an optical interface, the parity bit might actually be used. And sure enough, configuring the USB-UART adapter for even parity and sending the same message again triggered a response from the EDPW:

UART trace with even parity
UART trace with even parity

In contrast to the Miele@home communication log, the payload of the response was a3 01 (419 in decimal) in this case. According to the technical documentation for the EDPW 206, this seems to be the so-called software ID of the board. Feeling relatively optimistic at this point, I tried transmitting the next handshake message, hoping to receive another response. However, upon sending the 21 command, the EDPW just answered with a single 02 byte. Trying other random commands also led to the same response, except for the 10 command which was acknowledged with 00 . Sending a valid message with an incorrect checksum caused the EDPW to reply with 01 .

Using a short Python script, I tested every single possible command, but none of the other commands received an acknowledgement from the EDPW. Nevertheless, even invalid commands had to be acknowledged by the PC before the EDPW would accept the next message:

PC -> EDPW: XX 00 00 02 CC
EDPW -> PC: 02
PC -> EDPW: 00

<continue with XX + 1>

At this point, the only two commands that triggered a positive response from the EDPW were 10 and 11 . While messing around with command 11 , I realized that the EDPW would not react to messages sent after this command unless the PC responded with its own 4-byte payload:

PC -> EDPW: 11 00 00 02 13
EDPW -> PC: 00 a3 01 a4
PC -> EDPW: 00
PC -> EDPW: 00 00 00 00

<next message can be sent>

However, changing the values of these 4 bytes didn’t seem to trigger another response from the EDPW. Wondering whether the handshake might require a certain sequence of commands, I modified the script to transmit command 11 before every iteration:

PC -> EDPW: 11 00 00 02 13
EDPW -> PC: 00 a3 01 a4
PC -> EDPW: 00
PC -> EDPW: 00 00 00 00
PC -> EDPW: XX 00 00 00 CC
EDPW -> PC: 02
PC -> EDPW: 00

<continue with XX + 1>

This revealed yet another valid command that was part of the handshake sequence: 20 . However, that was apparently still not enough to successfully complete the handshake process. None of the commands I tried after this point yielded any meaningful response from the EDPW. The response to command 20 was always 00 , no matter what parameter values I used for the message. After reading up on common diagnostic protocols from the 1990s, I came up with the following theory:

  • Command 20 is used to unlock the diagnostic interface, but requires a certain set of parameters (a secret key)
  • As part of the unlock sequence, command 11 always has to be sent before 20
  • Upon reception of command 10 , the diagnostic interface is locked again

But how can the secret key for the unlock command be determined? Assuming that the key is encoded in the two parameter bytes of the message, a simple brute-force approach would require up to 65536 tries to guess the key. However, without knowing whether the key was actually correctly guessed and which commands are unlocked if the correct key is provided, the total number of required attempts would increase significantly. Considering the interface’s low speed of 2400 baud, this strategy didn’t seem to be feasible at all.

Finding a different attack vector #

I decided to take a closer look at the microcontroller on the EDPW board in search of other attack vectors. As previously mentioned , the Mitsubishi M37451 is configured for single-chip mode, executing its firmware directly from the internal mask ROM. However, for the two other processor modes, the M37451 provides some additional output signals which can be used to control an external EEPROM. These signals are named WR, RD, R/W and SYNC, as can be seen in the bottom right corner of the microcontroller’s pinout:

Mitsubishi M37451 pinout
Mitsubishi M37451 pinout

According to the datasheet, the SYNC signal is high while the microcontroller is fetching an operation code from its memory. Reading from the data bus sets RD high, while writing to an external component sets WR high. The bus transfer direction is also indicated by a combined R/W signal, which is high during bus reads and low during bus writes.

One would expect these signals to be disabled when the microcontroller operates in single-chip mode, right? Well, to my surprise, attaching a logic analyzer to the SYNC pin actually showed significant activity:

Logic analyzer trace of the microcontroller’s SYNC pin
Logic analyzer trace of the microcontroller’s SYNC pin

It turned out that all of the data bus signals are enabled, even in single-chip mode when the internal mask ROM is used. Could this SYNC signal be used to observe the program execution while receiving a diagnostic message via the PC interface? Yes, in fact the whole message processing can be seen in the logic analyzer trace:

SYNC trace during UART reception
SYNC trace during UART reception

Zooming in slightly after the stop bit shows the actual UART interrupt service routine that is being executed:

SYNC trace during interrupt handling of a valid message
SYNC trace during interrupt handling of a valid message

When sending an invalid diagnostic message instead, the microcontroller seems to return from the interrupt routine much earlier:

SYNC pin behavior when receiving an invalid message
SYNC pin behavior when receiving an invalid message

The length of the interrupt routine can therefore be used to distinguish between valid and invalid messages.

Leaking the secret key #

With these observations in mind, it should be possible to figure out the secret parameters for the unlock command. The unlock command likely looks something like this in pseudocode:

fn handle_cmd_unlock(param1: u8, param2: u8) {
    if (param1 != UNLOCK_KEY1)
        return;

    if (param2 != UNLOCK_KEY2)
        return;

    unlock_interface();
}

Therefore, guessing the correct value for the first parameter should lead to a slightly longer execution time of the interrupt routine. This same procedure can then be repeated for the second parameter, while keeping the correct value for the first parameter to discover the correct unlock message. Unlike a full brute-force approach, this approach only takes 512 tries at maximum. Each unlock attempt then consists of the following steps:

  1. Send command 11 to the PC interface as the first part of the unlock sequence
  2. Start logic analyzer capture, triggering on the falling edge of the UART receive signal
  3. Transmit command 20 with the chosen parameter values
  4. Decode the recorded trace and convert the SYNC sequence into a string of 1’s and 0’s

As before, this process is automated using a Python script and the sigrok-cli tool. As part of the decoding process, the script samples the SYNC signal on every falling edge of the microcontroller’s clock output signal:

SYNC trace with clock signal
SYNC trace with clock signal

The recorded SYNC trace is thereby turned into a long bit string:

10100101001010100010101000001000101000101000010000100101000100100100001000010010100010001010001001010100001010000100001010101000101010001010001010100010100010101000101000101010100101001010001000100000100101010101010010101010100010000101010001010101000100000100010001000100010100001000100101010000010010100010100010101000001001000100010100001000100000100010000010000010001000101000101000

As the microcontroller is constantly executing instructions, the actual start and end of the interrupt routine are always at a different position in the bit string. It is hard to distinguish between this routine and instructions that run before or after the actual interrupt. To find the boundaries of the routine, I defined two bit patterns that are common for every bit string:

Start pattern: 10100101001010100010101000
End   pattern: 10000010001000101000101000

Using these patterns, the actual payload of the SYNC trace can be determined:

10100101001010100010101000001000101000101000010000100101000100100100001000010010100010001010001001010100001010000100001010101000101010001010001010100010100010101000101000101010100101001010001000100000100101010101010010101010100010000101010001010101000100000100010001000100010100001000100101010000010010100010100010101000001001000100010100001000100000100010000010000010001000101000101000

This result then allows measuring the interrupt routine’s execution time. A change in bit stream length therefore clearly indicates a correct unlock parameter. The Python script can now be used to capture the bit strings for all possible values of the first parameter :

[20, 00, 00, 00]: 00100...10001000101000010001000001000100000
...
[20, eb, 00, 00]: 00100...10001000101000010001000001000100000
[20, ec, 00, 00]: 00100...10001000101000010001000001000100000
[20, ed, 00, 00]: 00100...10001000101000010001000001000100000
[20, ee, 00, 00]: 00100...100010100100010001010000100010001000001000100000
[20, ef, 00, 00]: 00100...10001000101000010001000001000100000
[20, f0, 00, 00]: 00100...10001000101000010001000001000100000
[20, f1, 00, 00]: 00100...10001000101000010001000001000100000
...
[20, ff, 00, 00]: 00100...10001000101000010001000001000100000

And there it is: The correct first parameter seems to be ee . This procedure is now repeated for the second parameter :

[20, ee, 00, 00]: 00100...00010001010000100010001000001000100000
...
[20, ee, b1, 00]: 00100...00010001010000100010001000001000100000
[20, ee, b2, 00]: 00100...00010001010000100010001000001000100000
[20, ee, b3, 00]: 00100...00010001010000100010001000001000100000
[20, ee, b4, 00]: 00100...000101000101010100001000100010001000001000100000
[20, ee, b5, 00]: 00100...00010001010000100010001000001000100000
[20, ee, b6, 00]: 00100...00010001010000100010001000001000100000
[20, ee, b7, 00]: 00100...00010001010000100010001000001000100000
...
[20, ee, ff, 00]: 00100...00010001010000100010001000001000100000

A few seconds later, the correct parameter combination for the unlock message is identified: ee , b4 .

Unlocking the PC interface #

The full functionality of the diagnostic interface can now be enabled by completing the unlock sequence:

PC -> EDPW: 11 00 00 02 13
EDPW -> PC: 00 a3 01 a4
PC -> EDPW: 00
PC -> EDPW: 00 00 00 00
PC -> EDPW: 20 ee b4 00 c2
EDPW -> PC: 00
PC -> EDPW: 00

But what did this sequence actually do? Re-testing the command set reveals the presence of commands 30 , 31 and 32 , which are now successfully acknowledged by the EDPW.

While messing around with these new commands, I discovered that sending the command 31 causes the EDPW to respond with a maximum of 4 bytes, depending on the requested response length of the message. However, all returned bytes were zero, regardless of the parameter values:

PC -> EDPW: 31 00 00 04 35
EDPW -> PC: 00
EDPW -> PC: 00 00 00 00 00

Upon further inspection of the EDPW board, I noticed that I forgot to supply power to the EEPROM chip. Sending the command again now resulted in the following response:

PC -> EDPW: 31 00 00 04 35
EDPW -> PC: 00
EDPW -> PC: 00 00 0f 00 0f

Trying different parameter values resulted in varying responses from the EDPW, revealing that command 31 reads the (inverted) EEPROM data starting at the specified offset.

Moving on to command 30 , I quickly noticed that its behavior closely followed the EEPROM read command. However, its response didn’t appear to depend on the presence of the EEPROM chip. Reading the first 256 bytes using this command resulted in the following data:

00000000  00 00 00 00 00 00 33 00  4f 8a 02 00 35 33 02 01
00000010  00 05 00 89 ff ff ff ff  00 08 01 07 01 ee 05 07
00000020  07 07 07 07 07 05 07 07  04 9c fe 03 02 00 00 00
00000030  44 00 00 00 00 00 0a 00  83 00 00 00 00 00 00 00
00000040  00 00 ff ff fd 00 00 f8  44 00 00 00 00 00 26 00
00000050  e3 db 4a 00 00 00 04 20  80 3f 00 00 00 30 00 00
00000060  00 00 00 00 00 00 00 00  40 00 02 00 00 00 00 00
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
00000080  00 00 00 00 20 00 00 00  54 01 00 00 00 00 00 00
00000090  00 ff ff 00 ff ff 00 5c  43 00 00 00 00 00 00 00
000000a0  05 20 18 00 01 00 18 00  00 ff ff 07 06 02 2c 06
000000b0  02 00 00 00 00 05 30 b4  00 10 f8 00 10 f8 00 08
000000c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
000000d0  3e ce 38 c4 00 fb b6 e9  9b 00 00 ff 02 fd 70 90
000000e0  00 00 6e 14 04 01 24 84  b8 f2 81 84 ff 00 00 03
000000f0  cc 32 c4 09 64 07 c4 09  9e 5c ff ff 00 21 e8 1c

Hmm, could this be the internal memory of the microcontroller? To verify this, I consulted the memory map of the Mitsubishi M37451:

Memory map of the Mitsubishi M37451
Memory map of the Mitsubishi M37451

The area marked as not used in the diagram ranges from address c0 to cf . Assuming that these bytes were zero, this would match with the response data from command 30 . Another memory area to check would be the SFR region . Knowing that the baud rate generator was configured for a baud rate of 2400, the BRG register at address ea would have to be set to 81 . This value could also be found in the previous response. The rest of the memory contents were part of the RAM region . This confirmed that command 30 reads memory contents based on the provided address parameters.

Dumping the full memory contents #

After some quick modifications to the Python script, I managed to dump the complete memory contents from the microcontroller:

Reading the Mitsubishi M37451’s memory contents

This whole process took around half an hour due to the low baud rate of the PC interface. Taking a closer look at the memory dump reveals that it actually contains the full internal mask ROM contents of the microcontroller:

0000a000  1f d4 2a 90 04 2f d4 80  02 3f d4 0f d4 88 d0 f0
0000a010  60 2f d5 0f d5 ef d3 ff  d2 3f d4 1f d4 0f d4 1f
0000a020  d4 ef d2 0f d4 1f d4 2f  d4 0f d4 a0 02 20 00 a0
0000a030  a0 08 a5 4b 20 00 a0 1f  d4 60 85 4b a9 80 20 11
<snip>
0000ffd0  ff ff ff ff ff ff ff ff  ff ff 30 11 00 a3 01 03
0000ffe0  e9 b1 e9 b1 62 d7 43 d7  4a cd e9 b1 e9 b1 13 cd
0000fff0  d9 e2 71 b1 e9 b1 f2 b0  e9 b1 e9 b1 e9 b1 28 b4

And there you have it! A full firmware dump of the Mitsubishi M37451 on the Miele EDPW 206 board.

While this blog post about the Miele PC interface protocol is far from complete, I think it’s time to wrap things up for now. The full reverse engineering of the protocol will have to wait until next time, where I’ll dive into a detailed analysis of the firmware.

Thanks for reading!

Apple's Problem with Bodies

Hacker News
drobinin.com
2025-11-21 23:16:25
Comments...
Original Article

Building for iOS sometimes feels like archaeology: brush away enough guidelines and you hit something older and stranger. A system that can classify violence with forensic precision still can't decide if the human body is health, lifestyle, or sin.

One day I tried to ship a private intimacy tracker–nothing scandalous, just a journal for wellbeing–and App Store Connect assigned it the 16+ rating it uses for gambling apps and "unrestricted web access". The rating itself is fine: the target audience is well past that age anyway. What baffles me is the logic .

Silk –the app I’m talking about, almost reluctantly–is a wellbeing journal in the most boring sense possible. You choose a few words about your day, moods, closeness, symptoms, or whatever else matters to you and your partner(s). It lives entirely on-device, syncs with nothing and phones no one. The whole point is that nothing interesting happens to your data after you close the app.

Silk has 16+ rating (17+ in the previous age matrix). Fortnite has 13+ rating (12+ in the previous age matrix).

And yet, from the App Store’s point of view, you can build a game with guns and cartoon violence and happily ship it to kids, while tracking your own body needs a 16+ “mature themes” label.

Welcome to the grey zone.

A Category With No Name

If you were around for the early App Store, you’ll remember its optimism: accelerometer-driven beer glasses, wobbling jelly icons, flashlight apps that set brightness to 100% because no one had ever considered the idea before. The ecosystem assumed “content” meant pictures, sound, or the occasional cow-milking simulator–not a user quietly describing part of their life to themselves.

App Store in 2008
Source: techncrunch.com

The App Store still carries the outline of that first life. Its vocabulary came from iTunes, which came from film ratings, built for a world where "content" meant something you could point a camera at. When the App Store arrived in 2008, it reused that system because it was available–and because no one expected apps to do much beyond wobbling or making noise.

Those assumptions didn’t last. By 2009 the Store had hosted an infamous $999 app that did nothing but display a red gem, a game where you shook a crying baby until it died, and enough fart apps that one reportedly earned five figures daily [1] . The review process was learning in public.

Soundboards app Bacon Farts app Pocket Farts app
Source: thefanzine.com

Against that backdrop, Apple introduced age ratings in mid-2009 with iOS 3. The strictest category, 17+, wasn't really created for gore or gambling–it was a pressure valve for novelty apps where shaking your phone made cartoon clothes fall off. Anything that might show “objectionable content”, from bikini galleries to embedded browsers, went into the same bucket [2] .

Change of Direction

By 2010, Apple reversed course. After Steve Jobs declared "folks who want porn can buy an Android phone," thousands of sexy-but-not-explicit apps vanished overnight in what became known as the Great App Purge [3] . The platform moved from reactive cleanup to something more systematic.

The Age Ratings matrix Apple operates now in iOS 26 is far more precise. It defines violence with forensic granularity, subdivides gambling, categorises medical risk. It does all the things a global marketplace must do once everyone realises software can cause harm.

App Store Age Ratings Matrix

But the matrix still retains its original silhouette: everything is defined by "content," not context. App Review's logic is keyed to artifacts inside the bundle–screenshots, metadata, stored assets–not to what the software actually does. That works beautifully for games and media. It falls apart when the "content" is whatever the user decides to write that day.

Silk has no images, no user-generated photos, no feed, no external links. The matrix rated it 16+ anyway–the same tier as gambling apps and unrestricted browsers. The rating isn't describing what Silk does. It's describing the absence of a category that should exist.

What about Apple Health?

When HealthKit launched in 2014, Apple consciously avoided anything resembling "behavioural interpretation." Heart rate or steps were fine, relationships were not. A decade later, the API surface has expanded in every direction–sleep depth, sound exposure, handwashing, environmental allergens, even a "Sexual Activity" field added quietly in iOS 9 [4] . But relational wellbeing remains conspicuously absent.

HKCategoryTypeIdentifier/SexualActivity record

HealthKit tracks heart rate, inhaler usage, mindfulness minutes, and the more delicate end of gastrointestinal bookkeeping. Nowhere does it model intimacy, affection, or closeness–the things couples might actually want to track privately. If the platform doesn't have words for what you're building, the classification system can't label it correctly. The vocabulary doesn't exist.

Apple is not avoiding the topic here, they’re being literal. And when a system is literal in a domain that is inherently contextual, things start to get interesting.

Blind Spots Mean Constraints

The first place that breaks is search.

Apple's search infrastructure is fast and strict. Search for "budget app" and you get budget apps. Search for "meditation" and you get meditation apps plus a few over-confident habit trackers. Search for the phrases people actually use when they want what Silk does–"relationship journal", "couples diary", "private moments"–and you get wedding planners, travel blogs, generic note-taking apps, and the occasional CBT worksheet. The algorithm can't read between lines it doesn't know exist.

App Store search results for "relationship journal"

The Metadata Minefield

On the developer side, metadata stops being about discoverability and becomes a small diplomatic exercise. A few terms trigger moderation, a few trigger follow-up questions, and the keyword field turns into a minefield where every word is inspected for what it might imply rather than what it means. Too specific reads like medicine, too gentle reads like romance, and anything metaphorical gets outright rejected.

This isn't new though. In 2009, Ninjawords–a perfectly useful English dictionary–was delayed and forced into 17+ because it could return definitions for swear words [5] . Phil Schiller personally explained that since parental controls promised to filter profanity, any app displaying unfiltered words needed age-gating. Never mind that Safari could look up far worse. The rule was simple: uncurated content equals adult content, context be damned.

There's also the "massage" rule, mostly folklorfe but widely believed: any app with that word in its metadata triggers extended review, whether it’s physiotherapy or post-marathon recovery. The system was burned once by apps using "massage" as euphemism and never forgot. Most of the odd heuristics you encounter today are scars from 2009.

Ambiguity in meaning becomes ambiguity in engineering. Without shared vocabulary, misalignment cascades: classification shapes search, search shapes metadata, metadata triggers review flags. The policies update faster than the taxonomy beneath them evolves.

Once you see that, the problem becomes solvable–not culturally, but technically.

Time to Poke the System

At this point I did the only sensible thing: treated App Store Connect like a black box and started running experiments.

First test: keywords using only soft language–"relationship journal", "partner log", "connection tracker". Search rankings tanked. Silk dropped to page 8 for "relationship journal," outdone by printable worksheets for couples therapy. Good news: the algorithm was confident I wasn't selling anything objectionable. Bad news: it was equally confident I wasn't selling anything at all.

Replacing those with direct terms–"intimacy tracker", "sexual wellness", "couples health"–brought visibility back to page 2–3. It also triggered longer App Review cycles and required increasingly elaborate "Review Notes" explaining why the app shouldn't be rated 18+. Same binary, screenshots, and code, but different words in a metadata field neither the users nor I can even see in the UI.

Screenshots followed the same logic. Completely sterile set–empty fields, no microcopy, generic UI–sailed through but made Silk look like Notes with a different background. A more honest set showing what the app actually does triggered the 18+ question again. The framing changed the classification. The classification changed nothing about what the software does, but everything about where it appears and who finds it.

None of this is surprising if you assume the system is a classifier trained on categories from 2009. From the outside it feels arbitrary. From the inside it's doing exactly what it was built to do: match patterns it understands and escalate the ones it doesn’t. It just doesn't have a pattern for "private health journal that mentions bodies", even though there are lots of private health journals in the App Store these days. You can almost hear it thinking: This smells like health but reads like dating and contains the word 'intimacy.' Escalate!

Designing Around the Gap

Silk’s architecture was shaped by this lag in the same way Fermento’s safety checks were shaped by gaps in food-safety guidance , or Residency’s "compiler warnings" for travel emerged from inconsistent definitions of “presence” by different countries . It’s not a case study in “growth”; it’s just another example of what happens when you have to reverse-engineer the missing assumptions. When a domain refuses to state its rules, you provide the scaffolding yourself.

Most of the engineering time went into figuring out what not to build–not from fear of rejection, but from understanding how classifiers behave when they encounter undefined cases. Treat it like a compiler that hasn't learned your edge-case syntax yet and stay inside the subset of language it already understands. The discipline felt familiar–the same kind you develop when building in domains where the platform's rules aren't fully specified and you have to infer the boundaries from failed experiments.

Silk home screen and timeline

The more carefully you specify what the app does, the less the platform has to guess on your behalf. In 2010, you could ship "Mood Scanner" apps that claimed to read emotional states from fingerprints. They still exist–the App Store didn't purge them–but try submitting one in a category App Review associates with actual health data and you'll trigger very different questions. The scrutiny isn't random; it’s contextual. It depends on how your metadata accidentally pattern-matches against old problems.

Life in the Grey Zone

The closer Silk came to shipping, the more I understood the App Store's behaviour as conservatism–not ideological, but technical. The kind that keeps a global marketplace from accidentally approving malware. Some of this conservatism is regional: China's App Store has additional filters for "relationship content," South Korea requires separate disclosures for wellbeing data. Apple unifies this under one policy umbrella, which produces a system that's cautiously consistent across borders but not particularly imaginative about edge cases.

The post-Epic world made Apple more explicit about where liability lives. Ambiguity became expensive, underspecification became expensive, classifiers that behave "roughly right" became expensive. The safest rule became simple: if the system can't clearly state what something is, err on caution until the taxonomy expands.

The cost is that new categories appear slowly. Sleep apps lived on the periphery for years. Meditation apps bounced between "Health" and "Lifestyle" depending on screenshot aesthetics. Third-party cycle trackers existed for nearly a decade before Apple added native reproductive health tracking in 2015 and a dedicated Cycle Tracking experience in 2019 [6] . Digital wellbeing apps faced suspicion until Screen Time shipped in iOS 12. Each category began as an edge case, proved itself through user adoption, and eventually got formalized–usually announced in a single sentence at WWDC as if it had always existed.

Silk is at the beginning of that cycle. Eventually Apple will introduce a more nuanced descriptor, or HealthKit will model relational wellbeing, or the age matrix will gain more precision. The entire ecosystem will re-index overnight and everyone will move on.

It turns out the best way to handle a category that doesn’t exist is to build as if it does, then wait for the taxonomy to catch up. Until then, the grey zone is honestly not a bad neighbourhood. The users already know what the app is for. The platform will figure it out eventually.


Working in the quiet gaps of the platform? I build iOS software for the problems people don’t talk about. work@drobinin.com


  1. I Am Rich sold for $999.99 to eight people before Apple quietly yanked it. Baby Shaker made it through review until the backlash arrived. And iFart really did clear ~$10k/day at its peak. ↩︎

  2. The 17+ category arrived with iOS 3 in 2009. Officially it covered “mature themes” and gambling. Unofficially it was the overflow bin for anything vaguely sexy–from Wobble iBoobs to apps with embedded browsers, RSS readers, and Twitter clients, all lumped together because they might show something naughty. ↩︎

  3. The 2010 “Great App Purge” : 5–6k sexy-but-not-explicit apps disappeared overnight after Steve Jobs dropped the immortal “Folks who want porn can buy an Android phone.” A rare moment where Apple said the quiet part out loud. ↩︎

  4. HealthKit launched in iOS 8 with vitals and fitness but no reproductive health. Apple added “Sexual Activity” in iOS 9–a year after everyone noticed the omission. ↩︎

  5. Ninjawords, a dictionary, was rated 17+ because it could return swear words . Behold: a dictionary rated like a strip-poker app, because it contained words the app reviewers searched for on purpose. ↩︎

  6. Third-party cycle trackers existed for years, but Apple didn’t add reproductive health metrics until iOS 9 (2015), and didn’t ship dedicated Cycle Tracking until iOS 13 (2019). A category legitimised years after users created it. ↩︎

3D printing with unconventional vase mode

Hacker News
vorpal.se
2025-11-21 23:10:17
Comments...
Original Article

This article targets an advanced audience who are already familiar with the 3D printing. In this article I will try to collect some information I haven’t found written down in a single place yet. In particular, a lot of the information is seemingly only available in the form of YouTube videos that take a long time to get to the point.

If you are new to 3D printing and/or CAD for 3D printing, this is not the right article for you. Come back when you have done a bit of printing/design and want to learn advanced tricks to save on print time and material usage.

Basics of vase mode

With that out of the way what is this about? Vase mode is a printing mode where the printer prints a spiral path, with no seams. This is fast, avoids the visual blemishes of the seam but also has some downsides:

  • Only a single perimeter. This potentially means weaker parts.
  • No disconnected areas (per layer), you have to print with a single path.
  • No internal geometry. No infill. No top layers.
  • No supports.

Typically, it gets used for vases and pots. Thus, the name. Here is a crude example (I’m not an aesthetics focused designer, so imagine something prettier than this. If it fits and functions, it ships in my book):

Vase mode example

Of note here is that the model itself isn’t hollow, but the slicer will make it hollow for you (since it only prints a single perimeter). In PrusaSlicer this setting is found at “Print Settings” → “Layers and perimeters” → “Vertical shells” → “Spiral vase”. OrcaSlicer etc should have the same or similar setting as well somewhere else. I have no idea about Cura.

But there are some ways to stretch this mode to the limits, and that is what this article is about. This will make vase mode useful for more than just simple vases. And that can often be the fastest and lightest way to print a part, if you can pull it off.

To understand the tricks you do need to understand how vase mode works though. It takes solid geometry, and takes the outline of it. What is inside doesn’t matter. It will be ignored:

Broken vase mode

As can be seen, while the hole exists in the bottom solid layers, the slicer ignores it above that point.

So what can we do above that?

Internal geometry via slits

The idea comes from the RC plane 3D printing community, where they want to print lightweight but strong parts. In particular wings with internal supporting geometry. 2

There are two main tricks for unconventional vase mode prints. Let’s start with slits, as the next trick builds upon this first trick. As I’m no aircraft wing designer I will use other geometry for illustration purposes. The idea is useful in other contexts than RC wings, that is the whole point of this article.

Make a slit into the part. The left is for demonstration only, you need the slit to be really thin, 0.0001 1 mm or so, as shown on the right:

Slit example

If we extrude this into a block and slice it, PrusaSlicer will see this slit and print an outer perimeter going into the part, making a sort of internal support. You are basically modelling the infill yourself now:

Slit example extruded

If you try this, it will not work for you. This is because you are missing a crucial setting in PrusaSlicer. By default, PrusaSlicer will merge together close parts of the model. You need to change “Printer Settings” → “Advanced” → “Slicing” → “Slice gap closing radius”. Set it to 0.0. 3 Otherwise, none of this will work.

Slit example extruded

For our example with a hole in the middle from the introduction we could get the following result:

Slit example with hole

Note that the slit will be visible and you can feel it with your fingers, but it will be a fairly smooth indentation, not a sharp edge.

Double walls

Now, let’s expand on this technique to make it even more useful: Have you ever wanted to use vase mode but with two perimeters? We can build upon the previous trick to make a double wall:

Double wall example

This is done by making a slit through into the hollow inside and making sure the part itself is exactly wide enough for two perimeters that touch. You can find the width you should use by going into PrusaSlicer (with the same settings that you plan to use to print with) and looking at the info text in “Print Settings” → “Layers and perimeters” → “Vertical shells”:

Value in PrusaSlicer

That is the value you want to use for this to work correctly.

We can build upon this to make our internal geometry touch the opposite wall, like so:

Double wall CAD

We can also use this to anchor a slit to the outside wall. This allows us to anchor internal geometry to the outside wall without poking through. In fact, to ensure we have a single continuous outline, all but one slit must be done like this. The following picture shows what you need to do (note that the double wall thickness is 0.87 mm in this example, it will change depending on other settings):

Double wall with slit to outside

These two tricks presented so far form the basis of what I have seen called “unconventional vase mode”. 4

But there are some more tricks related to vase mode that are worth knowing about.

Extrusion width

To make a vase mode stronger, you can increase the extrusion width. The general recommendation is that you can go to about 2x the nozzle diameter and keep good quality. This works, since the nozzle has a bit of a flat spot around the orifice.

However, British Youtuber “Lost in Tech” did some tests showing that you can go way further than that, but I haven’t tested this myself, and quality does eventually start going down. It might be worth looking into if this is useful to you.

In PrusaSlicer you can change this in “Print Settings” → “Advanced” → “Extrusion width”. For vase mode “External perimeters” is what matters (above the solid base layers, that is):

Extrusion width in PrusaSlicer

Remember to rescale any double walls to fit the new extrusion width. It might be a good idea to use a variable in your CAD model to make it easier to update (at least if you use parametric CAD like OnShape, FreeCAD or Fusion 360 which support variables).

Fake vase mode

Finally, if you absolutely cannot print something in vase mode you can still get most of the benefits by what I have seen called “fake vase mode” 5 . To understand this, we should first consider exactly what settings vase mode changes. In PrusaSlicer vase mode changes the following settings:

  1. Single perimeter (except for the first few bottom layers).
  2. No top layers.
  3. No infill (except for the first few bottom layers).
  4. No supports
  5. Disables the setting “ensure vertical shell thickness”.
  6. Prints in a continuous spiral path.

You can do all of those except 6 by hand in the slicer. And you can mix and match those first five things as you see fit.

Let’s investigate this via a case study rather than simplified theoretical examples like we have done so far

Case study: spheres on sticks

I needed some spheres on the end of wooden sticks, to hold up a bird net over my strawberries on my balcony. I didn’t want the net to lie on the plants directly, and I needed something on the end of the sticks so that the net wouldn’t tear. Thus, spheres (or rather: truncated spheres for print bed adhesion and overhang reasons) on sticks.

Here is the basic design in a section view:

Sphere on sticks section view

This doesn’t quite work in vase mode, because the top of the sphere has very shallow overhangs. And the top needs to be smooth. (The “roof” of the internal hole is fine, thanks to the cone shape.) It is so close, we can almost use vase mode.

So first I designed this in CAD . We have a slit from the outside to the centre, as well as some slits from the centre that goes almost to the outside. In fact, they go to the “recommended object thin wall thickness” mentioned before. (Note that the slits do not go down into the solid bottom layers, for some additional strength.)

Sphere on stick CAD with slits

This results in the following in PrusaSlicer:

Sphere on stick in PrusaSlicer

Like true vase mode, I used zero infill. But I enabled “Ensure vertical shell thickness” and 1 top solid layer. This added a tiny bit of material just below the shallow top of the dome, making it printable, but still lighter than printing normally. Then I used a layer range modifier to disable “ensure vertical shell thickness” for the lower part of the print where it wasn’t needed, as PrusaSlicer wanted to add some material on the inside of the lower layers as well.

Sphere on stick in PrusaSlicer with additional extrusions

I also increased the extrusion width to 0.8 mm (with a 0.4 mm nozzle) to get additional strength, and I used scarf seams to make the outside seam almost invisible.

You can go further from true vase mode though: You could have an inner and outer perimeter like traditional non-vase slicing, but still model your own infill only where needed. You will get seams obviously, but you might still be able to print faster and save weight. We are moving further from true vase mode here, but only you can decide what exactly is best for your print:

Sphere on stick with no slit from outside to inside

In fact, when I printed some of these spheres, the version without a slit to the outside ended up the best looking 6 :

Sphere on stick printed

The slit is visible, but on the part printed without a slit extending to the outside there are no visible seams at all. The unevenness at the top is due to me filing away a small blob that the nozzle left behind as it pulled away at the end. It is smooth to the touch but reflects the light differently.

Conclusions

Vase mode and “fake vase mode” is an often underused printing mode for functional parts, and it can be used to save weight and print time. The difference will be most noticeable on larger parts, on smaller parts 10 vs 15 minutes might not be worth the extra design effort (unless you are planning to print many copies of the same part).

I’m a bit disappointed that the slit was as visible from the outside as it was. From the videos about RC aircraft wings that I saw I expected this to be less noticeable. But “fake vase mode” still comes to the rescue here, offering most of the benefits. And when combined with scarf joint seams (which I found truly impressive, first time I tried it), I don’t really see the need for true vase mode any more. You might as well get the best of both worlds.

I did not find any written resource online summarizing these techniques, so I hope this post is useful not just to remind myself in the future, but also to others looking for this information. With that in mind, below is a cheat sheet of the important points and settings to remember.

These techniques require tuning settings in your slicer. This may not be possible if you are printing with at a commercial print farm, or targeting people slicing with a dumbed down web based slicer (as has recently been launched by both Printables and Makerworld). But it would be a shame if such dumbed down slicers restricted what we could design and publish. I will always try to make the most what both CAD and the slicer exposes to me. 7

Do you have some other tips or tricks for vase mode? Did I get something wrong? Comment on Reddit or on Lemmy and I will likely see it (eventually).

Cheat sheet

Want to quickly remind yourself of the core ideas of this article when you are designing your next part? Here is a quick cheat sheet:

  • Slits: Use slits to add internal geometry.
    • 0.0001 mm wide (or 0.001 if your CAD software doesn’t like you that day).
    • PrusaSlicer: Set “Print Settings” → “Advanced” → “Slicing” → “Slice gap closing radius” to 0.
  • Double walls: Use double walls for more strength and to connect slits to the opposite wall.
    • PrusaSlicer: “Print Settings” → “Layers and perimeters” → “Vertical shells” (Look at info text to find width you need to use for your current print settings.)
  • Extrusion width: You can increase the extrusion width to 2x the nozzle diameter for additional strength with no quality downsides. You might be able to go even further, but eventually quality will start going down.
    • PrusaSlicer: “Print Settings” → “Advanced” → “Extrusion width” → “External perimeters”
  • Fake vase mode: You don’t need to use vase mode to get most of the benefits. You can mix and match all parts of normal vase mode except for the continuous spiral path. But consider scarf joints to hide seams.

California DMV approves map increase in Waymo driverless operations

Hacker News
www.dmv.ca.gov
2025-11-21 22:52:13
Comments...
Original Article
City Livermore County Alameda County City Alameda County Alameda County City Albany County Alameda County City Berkeley County Alameda County City Dublin County Alameda County City Emeryville County Alameda County City Fremont County Alameda County City Hayward County Alameda County City Newark County Alameda County City Oakland County Alameda County City Piedmont County Alameda County City Pleasanton County Alameda County City San Leandro County Alameda County City Union City County Alameda County City Antioch County Contra Costa County City Brentwood County Contra Costa County City Clayton County Contra Costa County City Concord County Contra Costa County City Danville County Contra Costa County City El Cerrito County Contra Costa County City Hercules County Contra Costa County City Lafayette County Contra Costa County City Martinez County Contra Costa County City Moraga County Contra Costa County City Oakley County Contra Costa County City Orinda County Contra Costa County City Pinole County Contra Costa County City Pittsburg County Contra Costa County City Pleasant Hill County Contra Costa County City Richmond County Contra Costa County City San Pablo County Contra Costa County City San Ramon County Contra Costa County City Walnut Creek County Contra Costa County City Agoura Hills County Los Angeles County City Arcadia County Los Angeles County City Artesia County Los Angeles County City Azusa County Los Angeles County City Baldwin Park County Los Angeles County City Bellflower County Los Angeles County City Bradbury County Los Angeles County City Burbank County Los Angeles County City Calabasas County Los Angeles County City Cerritos County Los Angeles County City Claremont County Los Angeles County City Covina County Los Angeles County City Diamond Bar County Los Angeles County City Duarte County Los Angeles County City El Monte County Los Angeles County City Glendora County Los Angeles County City Hawaiian Gardens County Los Angeles County City Hidden Hills County Los Angeles County City Industry County Los Angeles County City Irwindale County Los Angeles County City La Cañada Flintridge County Los Angeles County City La Habra Heights County Los Angeles County City La Mirada County Los Angeles County City La Puente County Los Angeles County City La Verne County Los Angeles County City Lakewood County Los Angeles County City Lomita County Los Angeles County City Malibu County Los Angeles County City Monrovia County Los Angeles County City Montebello County Los Angeles County City Norwalk County Los Angeles County City Palos Verdes Estates County Los Angeles County City Pasadena County Los Angeles County City Pico Rivera County Los Angeles County City Pomona County Los Angeles County City Rancho Palos Verdes County Los Angeles County City Rolling Hills County Los Angeles County City Rolling Hills Estates County Los Angeles County City Rosemead County Los Angeles County City San Dimas County Los Angeles County City San Fernando County Los Angeles County City San Gabriel County Los Angeles County City San Marino County Los Angeles County City Santa Clarita County Los Angeles County City Santa Fe Springs County Los Angeles County City Sierra Madre County Los Angeles County City Signal Hill County Los Angeles County City South El Monte County Los Angeles County City South Pasadena County Los Angeles County City Temple City County Los Angeles County City Walnut County Los Angeles County City West Covina County Los Angeles County City Westlake Village County Los Angeles County City Whittier County Los Angeles County City Alhambra County Los Angeles County City Bell County Los Angeles County City Bell Gardens County Los Angeles County City Carson County Los Angeles County City Commerce County Los Angeles County City Compton County Los Angeles County City Downey County Los Angeles County City El Segundo County Los Angeles County City Gardena County Los Angeles County City Glendale County Los Angeles County City Hermosa Beach County Los Angeles County City Long Beach County Los Angeles County City Los Angeles County Los Angeles County City Manhattan Beach County Los Angeles County City Monterey Park County Los Angeles County City Paramount County Los Angeles County City Redondo Beach County Los Angeles County City Santa Monica County Los Angeles County City South Gate County Los Angeles County City Torrance County Los Angeles County City Belvedere County Marin County City Corte Madera County Marin County City Fairfax County Marin County City Larkspur County Marin County City Mill Valley County Marin County City Novato County Marin County City Ross County Marin County City San Anselmo County Marin County City San Rafael County Marin County City Sausalito County Marin County City Tiburon County Marin County City American Canyon County Napa County City Calistoga County Napa County City Napa County Napa County City St. Helena County Napa County City Yountville County Napa County City Aliso Viejo County Orange County City Anaheim County Orange County City Brea County Orange County City Buena Park County Orange County City Costa Mesa County Orange County City Cypress County Orange County City Dana Point County Orange County City Fountain Valley County Orange County City Fullerton County Orange County City Garden Grove County Orange County City Huntington Beach County Orange County City Irvine Orange County Orange County City La Habra Orange County Orange County City La Palma Orange County Orange County City Laguna Beach County Orange County City Laguna Hills County Orange County City Laguna Niguel County Orange County City Laguna Woods County Orange County City Lake Forest County Orange County City Los Alamitos County Orange County City Mission Viejo County Orange County City Newport Beach County Orange County City Orange County Orange County City Placentia County Orange County City Rancho Santa Margarita County Orange County City San Clemente County Orange County City San Juan Capistrano County Orange County City Santa Ana County Orange County City Seal Beach County Orange County City Stanton County Orange County City Tustin County Orange County City Villa Park County Orange County City Westminster County Orange County City Yorba Linda County Orange County City Corona County Riverside County City Eastvale County Riverside County City Jurupa Valley County Riverside County City Lake Elsinore County Riverside County City Murrieta County Riverside County City Norco County Riverside County City Pechanga Reservation County Riverside County City Riverside County Riverside County City Temecula County Riverside County City Wildomar County Riverside County City Citrus Heights County Sacramento County City Elk Grove County Sacramento County City Folsom County Sacramento County City Galt County Sacramento County City Isleton County Sacramento County City Rancho Cordova County Sacramento County City Sacramento County Sacramento County City Chino Hills County San Bernardino County City Fontana County San Bernardino County City Montclair County San Bernardino County City Ontario County San Bernardino County City Rancho Cucamonga County San Bernardino County City Rialto County San Bernardino County City Upland County San Bernardino County City Barona Reservation County San Diego County City Capitan Grande County San Diego County City Carlsbad County San Diego County City Chula Vista County San Diego County City Coronado County San Diego County City Del Mar County San Diego County City El Cajon County San Diego County City Encinitas County San Diego County City Escondido County San Diego County City Imperial Beach County San Diego County City La Mesa County San Diego County City Lemon Grove County San Diego County City National City County San Diego County City Oceanside County San Diego County City Poway County San Diego County City San Diego County San Diego County City San Marcos County San Diego County City Santee County San Diego County City Solana Beach County San Diego County City Sycuan Reservation County San Diego County City Vista County San Diego County City San Francisco County San Francisco County City Half Moon County San Mateo County City Brisbane County San Mateo County City Burlingame County San Mateo County City East Palo Alto County San Mateo County City Foster City County San Mateo County City Menlo Park County San Mateo County City Pacifica County San Mateo County City Redwood City County San Mateo County City San Mateo County San Mateo County City South San Francisco County San Mateo County City Milpitas County Santa Clara County City Mountain View County Santa Clara County City Palo Alto Santa County Santa Clara County City San Jose County Santa Clara County City Sunnyvale County Santa Clara County City Unincorporated Area (Lexington
Hills area, overlapping Santa
Clara and Santa Cruz Counties) County Santa Cruz County City Benicia County Solano County City Dixon County Solano County City Fairfield County Solano County City Rio Vista County Solano County City Suisun City County Solano County City Vacaville County Solano County City Vallejo County Solano County City Cloverdale County Sonoma County City Cotati County Sonoma County City Dry Creek Rancheria County Sonoma County City Healdsburg County Sonoma County City Lytton Rancheria County Sonoma County City Petaluma County Sonoma County City Rohnert Park County Sonoma County City Santa Rosa County Sonoma County City Sebastopol County Sonoma County City Sonoma County Sonoma County City Stewarts Point Rancheria County Sonoma County City Windsor County Sonoma County City Camarillo County Ventura County City Moorpark County Ventura County City Simi Valley County Ventura County City Thousand Oaks County Ventura County City Davis County Yolo County City Rumsey Indian Rancheria County Yolo County City West Sacramento Yolo County Yolo County City Winters County Yolo County City Woodland County Yolo County

Web playground for training language models with WebGPU

Lobsters
sequence.toys
2025-11-21 22:46:44
Comments...
Original Article

Sequence Toy requires WebGPU support, and your browser doesn't support it yet. You have a few options:

  • Use the latest version of Chrome , or a Chromium-based browser (Edge, Arc, Brave, etc.)
    • If that doesn't work, try Chrome Canary and ensure WebGPU is enabled.
  • Use Firefox Nightly
  • Use the latest Safari Technology Preview or enable WebGPU via Feature Flags.
    • iOS: System Settings > Apps > Safari > Advanced > Feature Flags > Enable "WebGPU"
    • MacOS: Develop menu Feature Flags > Enable "WebGPU"

Train a language model in your browser with WebGPU

You could try…

…training a Transformer ? to sort characters , reverse a sequence , or find the numbers that add up to a sum , then compare with an LSTM ? . Visualize the gradients in the attention layer , all parameters in the network , or try writing your own visualization queries . Learn to match parentheses in a Dyck language ? using an encoder-only masked language modeling (MLM) objective ? . Want a taste of natural language? Try training a GPT on TinyStories ? ? , a dataset of short stories generated by GPT-4—and try different tokenizer sizes. Play with attention gating , MLP variants , learning rate schedulers , initialization , dropout , or QK normalization . Want to train an RNN? Mess with layer normalization , initialization , bidirectional encoder layers , encoder-decoder attention variants , or gradient norm clipping . Have a lot of VRAM and want to try something untested * ? Try training a GPT-2-sized model on FineWeb ? (and DM me if you get it to work!).

Vin Howe built Piston , a WebGPU automatic differentiation engine with a JavaScript API modeled after PyTorch, for this project. It started life as a fork of Ratchet ? by Christopher Fleetwood . Read about it in the blog post!

Footnotes

Personal blogs are back, should niche blogs be next?

Hacker News
disassociated.com
2025-11-21 22:40:28
Comments...
Original Article

20 November 2025

When it comes to blogging there are few rules. Write content that is somehow meaningful might be one of them though. I think it’s down to the individual to determine what constitutes meaningful.

In the hey-day, the so-called golden age of blogging, there were plenty of people prepared to offer definitions of meaningful, and how to write accordingly. It was natural. The web was once awash with all sorts of blogs. Likewise people who wanted to show others how to blog “successfully”.

Again, the definition of successful resided with the individual, but it was obvious this involved monetary return for some people. And why not. If you’re going to invest time and energy in creating a resource that is useful to other people, why shouldn’t you earn money, make a living even, from it?

One of these people blogging about blogging was Melbourne based Australian writer and author Darren Rowse, who launched his blogging resource Problogger in 2004. Without going into detail, because you can look it up for yourself, Rowse, as one of the earlier bloggers about blogging, did, and still does presumably, rather well for himself.

Rowse’s writing, and that of his contributors, attracted numerous readers keen to learn what they could about blogging, and the potential to make money from it.

Problogger is what’s called a niche blog. As a blog about blogging, it has a reasonably singular focus. Some people considered this niche principle to be a core tenet of blogging. There was this idea, in the earlier days of blogging, which possibly still persists, that blogs would do better if they had a speciality. Not only were search engines said to be in favour the approach, but the author of a speciality, or niche blog, would generally be considered to be an expert, of some sort, in their field.

A master of one trade, rather than the proverbial jack of all trades.

Regardless, the world was once full of blogs on every topic imaginable. It was a great time to be alive. If you wanted to learn about something in particular, there was a blog for you. Some publications featured quality content, others required a little fact checking, while some were definitely to be taken with a pinch of salt.

But niche blogging was never a format that suited everyone. There are people who did, still do, well, writing about a range, sometimes a wide range, of topics. Kottke is one of the better known blogs that does not have a specific speciality. Here, the publication itself is the speciality. To repeat what I wrote in the first sentence of this article: the rules of blogging are few.

But the facets of blogging covered at Problogger, and numerous other similar websites, usually only applied to blogs of a commercial nature. That’s not to say one or two personal bloggers might have looked at the tips posted there for increasing their audience, or improving their writing though. But in my view, personal bloggers were not, are not, part of Problogger’s target audience.

It’s been a long time since I last wrote about Problogger, let alone visited the website, maybe fifteen plus years, but a recent mention of it by Kev Quick , via ldstephens , caught my eye. But I don’t believe Rowse is being critical, in any way, of personal bloggers because they do not adhere to a niche or speciality publishing format. That’s not what Problogger, or Rowse, is about.

But this started me thinking, and writing another of my long posts.

In an age where social media, and influencers, have usurped blogs and their A-List authors, in the jostle for supremacy, it has to be wondered what role websites like Problogger still have. Only a handful of blogs generate liveable incomes today. Despite the doom and gloom though, the form has not completely died off. A backlash against social media, and a growing IndieWeb/SmallWeb community, has precipitated a revival in personal websites.

This is a largely non-commercial movement. Of course, there’s nothing wrong with personal websites. Many of us started out with them in the early days of the web. But the web was not only intended for personal journals. It was a vehicle for sharing all manner of information. The web could also empower individuals, and partnerships, to not only set up shop online, be that blogs, or quite literally shops, but potentially make a living at the same time.

But with the revival of personal blogs well underway, I think it’s time to bring niche blogs back into the fold. I’m talking about well written, quality, topic focused resources. This is material fast vanishing from the web, leaving ever diminishing options to source useful and accurate information. What are the alternatives? The misinformation morass that is social media? Being served AI generated summaries in response to search engine queries? A web choke full of AI slop?

At the same time, I’m not advocating for a return of niche blogs plastered with adverts, and popup boxes urging visitors to subscribe to say a newsletter, before they’ve even had a chance to blink at what they came to read.

I’m talking about work produced by independent writers, with an interest in their subject matter, who are not backed by large media organisations, or private equity. This is bringing back reliable sources of information, that also recompenses the content writers in some way. Hopefully we’ve learned a few lessons about monetisation since the earlier wave of niche blogging. We know it is possible to generate revenue without compromising the reader experience.

A resurgence in personal blogging is the first step in rebuilding a vibrant, thriving, web, of if you like, blogosphere. Now the focus needs to be on restoring the flow of accessible and trusted information.

, , , , , ,

Jmail

Daring Fireball
jmail.world
2025-11-21 22:25:12
Luke Igel and Riley Walz made a phony Gmail interface that, rather than showing you your email, shows you Jeffrey Epstein’s emails: You’re logged in as Jeffrey Epstein. We compiled these Epstein estate emails from the House Oversight release by converting the PDFs to structured text with an LLM....
Original Article

You are logged in as Jeffrey Epstein, [email protected] . These are real emails released by Congress. Explore by name, contribute to the starred list, search, or visit a random page .

Is Denmark going bankrupt?

- TOP STORIES FOR YOU Jeffrey's Digest --- Is Denmark going bankrupt? <image> Erik Cal

Jul 14, 2019

Is Denmark going bankrupt?

TOP STORIES FOR YOU Jeffrey's Digest --- Is Denmark going bankrupt? <image> Erik Cal

11 questions for Mueller

- Flipboard Interesting stories worth your time. --- The Seltzer Bubble <image> nytimes

Jul 13, 2019

11 questions for Mueller

Flipboard Interesting stories worth your time. --- The Seltzer Bubble <image> nytimes

Alex Acosta resigns, Jeffrey Epstein arrested and Trump ends bid for citizenship question on census

- Flipboard Biggest news stories from the past week. <image> Alex Acosta resigns as labor se

Jul 13, 2019

Alex Acosta resigns, Jeffrey Epstein arrested and Trump ends bid for citizenship question on census

Flipboard Biggest news stories from the past week. <image> Alex Acosta resigns as labor se

Do French people generally identify more with Germans, Italians, or Spaniards?

- Jeffrey's Digest TOP STORIES FOR YOU Do French people generally identify more wit

Jul 04, 2019

Do French people generally identify more with Germans, Italians, or Spaniards?

Jeffrey's Digest TOP STORIES FOR YOU Do French people generally identify more wit

Capital Market Outlook

- Capital Market Outlook Chief Investment Office Capital Market Outlook from our Chief Investme

Jul 02, 2019

Capital Market Outlook

Capital Market Outlook Chief Investment Office Capital Market Outlook from our Chief Investme

Kim-Trump border meeting: history or photo-op?

- Flipboard > Interesting stories worth your time. --- <image> What Does Putin Really Want?

Jun 30, 2019

Kim-Trump border meeting: history or photo-op?

Flipboard > Interesting stories worth your time. --- <image> What Does Putin Really Want?

2020 Democratic debates and major Supreme Court rulings

- Flipboard Biggest news stories from the past week. <image> 5 things we learned from tw

Jun 29, 2019

2020 Democratic debates and major Supreme Court rulings

Flipboard Biggest news stories from the past week. <image> 5 things we learned from tw

The hired guns of Instagram

- Flipboard Interesting stories worth your time. The hired guns of Instagram <image> vox.com /

Jun 19, 2019

The hired guns of Instagram

Flipboard Interesting stories worth your time. The hired guns of Instagram <image> vox.com /

RE:

- More bad news for our friend

May 07, 2019

RE:

More bad news for our friend

(no subject)

- https://www.nytimes.com/2019/05/06/us/politics/trump-inauguration-stephanie-winston-wolkoff.html?act

May 07, 2019

(no subject)

https://www.nytimes.com/2019/05/06/us/politics/trump-inauguration-stephanie-winston-wolkoff.html?act

Re: $25K from ASU

- What is the size of the gift? Sent from my iPhone

May 06, 2019

Re: $25K from ASU

What is the size of the gift? Sent from my iPhone

Re: Announcing the 2019 Hillman Prize winners

- Fix always in

Apr 23, 2019

Re: Announcing the 2019 Hillman Prize winners

The Sidney Hillman Foundation

3

Announcing the 2019 Hillman Prize winners

- Congratulations to the 2019 Hillman Prize recipients! Sidney Hillman Foundation Names Winners o

Apr 23, 2019

Announcing the 2019 Hillman Prize winners

Congratulations to the 2019 Hillman Prize recipients! Sidney Hillman Foundation Names Winners o

The Sidney Hillman Foundation

2

Announcing the 2019 Hillman Prize winners

- Congratulations to the 2019 Hillman Prize recipients! --- Sidney Hillman Foundation Names Winn

Apr 23, 2019

Announcing the 2019 Hillman Prize winners

Congratulations to the 2019 Hillman Prize recipients! --- Sidney Hillman Foundation Names Winn

Re: Fw: Netflix/Jeffrey Epstein

- and Patterson's not!!

Apr 23, 2019

Re: Fw: Netflix/Jeffrey Epstein

Fwd: Epstein prosecutor was rebuked for prior child sex case | Miami Herald

- Privileged - Redacted ---------- Forwarded message --------- -- please note The informatio

Mar 14, 2019

attach_file

Fwd: Epstein prosecutor was rebuked for prior child sex case | Miami Herald

Privileged - Redacted ---------- Forwarded message --------- -- please note The informatio

Opportunity Zones

- Hope all well, are you doing anything around these or clients? In a former warehouse on a dimly lit

Jan 15, 2019

attach_file

Opportunity Zones

Hope all well, are you doing anything around these or clients? In a former warehouse on a dimly lit

Update

- miss chatting with you am back at work; will give keynote to the Louisiana State Psychological Asso

Oct 11, 2018

attach_file

Update

miss chatting with you am back at work; will give keynote to the Louisiana State Psychological Asso

Fw: The 'Deplorables' Called Into Battle Again - WSJ

- Sent via BlackBerry by AT&T

Aug 27, 2018

Fw: The 'Deplorables' Called Into Battle Again - WSJ

Sent via BlackBerry by AT&T

Trump @ War

- https://app.frame.io/presentations/a87092af-cb6a-473b-9241-582c03dbb906 Sent from ProtonMail Mobile

Aug 23, 2018

Trump @ War

https://app.frame.io/presentations/a87092af-cb6a-473b-9241-582c03dbb906 Sent from ProtonMail Mobile

Re: NYTimes: Donald Trump's High Crimes and Misdemeanors

- ill call you from new york --- please note The information contained in this communication is confi

Aug 23, 2018

Re: NYTimes: Donald Trump's High Crimes and Misdemeanors

ill call you from new york --- please note The information contained in this communication is confi

Re: Bannon on How 2008 Planted the Seed for the Trump Presidency

- And Cohn corrupt

Aug 11, 2018

Re: Bannon on How 2008 Planted the Seed for the Trump Presidency

Bannon on How 2008 Planted the Seed for the Trump Presidency

- http://nymag.com/daily/intelligencer/2018/08/steve-bannon-on-how-2008-planted-the-seed-for-the-trump

Aug 11, 2018

Bannon on How 2008 Planted the Seed for the Trump Presidency

http://nymag.com/daily/intelligencer/2018/08/steve-bannon-on-how-2008-planted-the-seed-for-the-trump

Re: Exclusive: Bannon blasts 'con artist' Kochs, 'lame duck' Ryan, 'diminished' Kelly | TheHill

- Where is this from???

Aug 03, 2018

Re: Exclusive: Bannon blasts 'con artist' Kochs, 'lame duck' Ryan, 'diminished' Kelly | TheHill

Is Matrix Multiplication Ugly?

Hacker News
mathenchant.wordpress.com
2025-11-21 22:17:11
Comments...
Original Article

A few weeks ago I was minding my own business, peacefully reading a well-written and informative article about artificial intelligence , when I was ambushed by a passage in the article that aroused my pique. That’s one of the pitfalls of knowing too much about a topic a journalist is discussing; journalists often make mistakes that most readers wouldn’t notice but that raise the hackles or at least the blood pressure of those in the know.

The article in question appeared in The New Yorker . The author, Stephen Witt, was writing about the way that your typical Large Language Model, starting from a blank slate, or rather a slate full of random scribbles, is able to learn about the world, or rather the virtual world called the internet. Throughout the training process, billions of numbers called weights get repeatedly updated so as to steadily improve the model’s performance. Picture a tiny chip with electrons racing around in etched channels, and slowly zoom out: there are many such chips in each server node and many such nodes in each rack, with racks organized in rows, many rows per hall, many halls per building, many buildings per campus. It’s a sort of computer-age version of Borges’ Library of Babel. And the weight-update process that all these countless circuits are carrying out depends heavily on an operation known as matrix multiplication.

Witt explained this clearly and accurately, right up to the point where his essay took a very odd turn.

HAMMERING NAILS

Here’s what Witt went on to say about matrix multiplication:

“‘Beauty is the first test: there is no permanent place in the world for ugly mathematics,’ the mathematician G. H. Hardy wrote, in 1940. But matrix multiplication, to which our civilization is now devoting so many of its marginal resources, has all the elegance of a man hammering a nail into a board. It is possessed of neither beauty nor symmetry: in fact, in matrix multiplication, a times b is not the same as b times a .”

The last sentence struck me as a bizarre non sequitur, somewhat akin to saying “Number addition has neither beauty nor symmetry, because when you write two numbers backwards, their new sum isn’t just their original sum written backwards; for instance, 17 plus 34 is 51, but 71 plus 43 isn’t 15.”

The next day I sent the following letter to the magazine:

“I appreciate Stephen Witt shining a spotlight on matrices, which deserve more attention today than ever before: they play important roles in ecology, economics, physics, and now artificial intelligence (“ Information Overload ”, November 3). But Witt errs in bringing Hardy’s famous quote (“there is no permanent place in the world for ugly mathematics”) into his story. Matrix algebra is the language of symmetry and transformation, and the fact that a followed by b differs from b followed by a is no surprise; to expect the two transformations to coincide is to seek symmetry in the wrong place — like judging a dog’s beauty by whether its tail resembles its head. With its two-thousand-year-old roots in China, matrix algebra has secured a permanent place in mathematics, and it passes the beauty test with flying colors. In fact, matrices are commonplace in number theory, the branch of pure mathematics Hardy loved most.”

Confining my reply to 150 words required some finesse. Notice for instance that the opening sentence does double duty: it leavens my many words of negative criticism with a few words of praise, and it stresses the importance of the topic, preëmptively 1 rebutting editors who might be inclined to dismiss my correction as too arcane to merit publication.

I haven’t heard back from the editors, and I don’t expect to. Regardless, Witt’s misunderstanding deserves a more thorough response than 150 words can provide. Let’s see what I can do with 1500 words and a few pictures.

THE GEOMETRY OF TRANSFORMATIONS

As a static object, matrices are “just” rectangular arrays of numbers, but that doesn’t capture what they’re really about. If I had to express the essence of matrices in a single word, that word would be “transformation”.

One example of a transformation is the operation f that takes an image in the plane and flips it from left to right, as if in a vertical mirror.


Another example is the operation g that that takes an image in the plane and reflects it across a diagonal line that goes from lower left to upper right.


The key thing to notice here is that the effect of f followed by g is different from the effect of g followed by f . To see why, write a capital R on one side of a square piece of paper–preferably using a dark marker and/or translucent paper, so that you can still see the R even when the paper has been flipped over–and apply f followed by g ; you’ll get the original R rotated by 90 degrees clockwise. But if instead, starting from that original R, you were to apply g followed by f , you’d get the original R rotated by 90 degrees counterclockwise .

Same two operations, different outcomes! Symbolically we write g f f g , where g f means “First do f , then do g ” and f g means “First do g , then f ”. 2 The symbol ◦ denotes the meta-operation (operation-on-operations) called composition .

The fact that the order in which transformations are applied can affect the outcome shouldn’t surprise you. After all, when you’re composing a salad, if you forget to pour on salad dressing until after you’ve topped the base salad with grated cheese, your guests will have a different dining experience than if you’d remembered to pour on the dressing first. Likewise, when you’re composing a melody, a C-sharp followed by a D is different from a D followed by a C-sharp. And as long as mathematicians used the word “composition” rather than “multiplication”, nobody found it paradoxical that in many contexts, order matters.

THE ALGEBRA OF MATRICES

If we use the usual x , y coordinates in the plane, the geometric operation f can be understood as the numerical operation that sends the pair ( x , y ) to the pair (− x , y ), which we can represented via the 2-by-2 array

where more generally the array

stands for the transformation that sends the pair ( x , y ) to the pair ( ax + by , cx + dy ). This kind of array is called a matrix , and when we want to compose two operations like f and g together, all we have to do is combine the associated matrices under the rule that says that the matrix

composed with the matrix

equals the matrix

For more about where this formula comes from, see my Mathematical Enchantments essay “ What Is A Matrix ?”.

There’s nothing special about 2-by-2 matrices; you could compose two 3-by-3 matrices, or even two 1000-by-1000 matrices. Going in the other direction (smaller instead of bigger), if you look at 1-by-1 matrices, the composition of

and

is just

so ordinary number-multiplication arises as a special case of matrix composition; turning this around, we can see matrix-composition as a sort of generalized multiplication. So it was natural for mid-19th-century mathematicians to start using words like “multiply” and “product” instead of words like “compose” and “composition”, at roughly the same time they stopped talking about “substitutions” and “tableaux” and started to use the word “matrices”.

In importing the centuries-old symbolism for number multiplication into the new science of linear algebra, the 19th century algebraists were saying “Matrices behave kind of like numbers,” with the proviso “except when they don’t”. Witt is right when he says that when A and B are matrices, A times B is not always equal to B times A . Where he’s wrong is in asserting that is a blemish on linear algebra. Many mathematicians regard linear algebra as one of the most elegant sub-disciplines of mathematics ever devised, and it often serves as a role model for the kind of sleekness that a new mathematical discipline should strive to achieve. If you dislike matrix multiplication because AB isn’t always equal to BA , it’s because you haven’t yet learned what matrix multiplication is good for in math, physics, and many other subjects. It’s ironic that Witt invokes the notion of symmetry to disparage matrix multiplication, since matrix theory and an allied discipline called group theory are the tools mathematicians use in fleshing out our intuitive ideas about symmetry that arise in art and science.

So how did an intelligent person like Witt go so far astray?

PROOFS VS CALCULATIONS

I’m guessing that part of Witt’s confusion arises from the fact that actually multiplying matrices of numbers to get a matrix of bigger numbers can be very tedious, and tedium is psychologically adjacent to distaste and a perception of ugliness. But the tedium of matrix multiplication is tied up with its symmetry (whose existence Witt mistakenly denies). When you multiply two n -by- n matrices A and B in the straightforward way, you have to compute n 2 numbers in the same unvarying fashion, and each of those n 2 numbers is the sum of n terms, and each of those n terms is the product of an element of A and an element of B in a simple way. It’s only human to get bored and inattentive and then make mistakes because the process is so repetitive. We tend to think of symmetry and beauty as synonyms, but sometimes excessive symmetry breeds ennui; repetition in excess can be repellent. Picture the Library of Babel and the existential dread the image summons.

G. H. Hardy, whose famous remark Witt quotes, was in the business of proving theorems, and he favored conceptual proofs over calculational ones. If you showed him a proof of a theorem in which the linchpin of your argument was a 5-page verification that a certain matrix product had a particular value, he’d say you didn’t really understand your own theorem; he’d assert that you should find a more conceptual argument and then consign your brute-force proof to the trash. But Hardy’s aversion to brute force was specific to the domain of mathematical proof, which is far removed from math that calculates optimal pricing for annuities or computes the wind-shear on an airplane wing or fine-tunes the weights used by an AI. Furthermore, Hardy’s objection to your proof would focus on the length of the calculation, and not on whether the calculation involved matrices. If you showed him a proof that used 5 turgid pages of pre-19th-century calculation that never mentioned matrices once, he’d still say “Your proof is a piece of temporary mathematics; it convinces the reader that your theorem is true without truly explaining why the theorem is true.”

If you forced me at gunpoint to multiply two 5-by-5 matrices together, I’d be extremely unhappy, and not just because you were threatening my life; the task would be inherently unpleasant. But the same would be true if you asked me to add together a hundred random two-digit numbers. It’s not that matrix-multiplication or number-addition is ugly; it’s that such repetitive tasks are the diametrical opposite of the kind of conceptual thinking that Hardy loved and I love too. Any kind of mathematical content can be made stultifying when it’s stripped of its meaning and reduced to mindless toil. But that casts no shade on the underlying concepts. When we outsource number-addition or matrix-multiplication to a computer, we rightfully delegate the soul-crushing part of our labor to circuitry that has no soul. If we could peer into the innards of the circuits doing all those matrix multiplications, we would indeed see a nightmarish, Borgesian landscape, with billions of nails being hammered into billions of boards, over and over again. But please don’t confuse that labor with mathematics.

Join the discussion of this essay over at Hacker News!

This essay is related to chapter 10 (“Out of the Womb”) of a book I’m writing, tentatively called “What Can Numbers Be?: The Further, Stranger Adventures of Plus and Times”. If you think this sounds interesting and want to help me make the book better, check out http://jamespropp.org/readers.pdf . And as always, feel free to submit comments on this essay at the Mathematical Enchantments WordPress site!

ENDNOTES

#1. Note the New Yorker -ish diaresis in “preëmptively”: as long as I’m being critical, I might as well be diacritical.

#2. I know this convention may seem backwards on first acquaintance, but this is how ◦ is defined. Blame the people who first started writing things like “log x ” and “cos x “, with the x coming after the name of the operation. This led to the notation f ( x ) for the result of applying the function f to the number x . Then the symbol for the result of applying g to the result of applying f to x is g ( f ( x )); even though f is performed first, “ f ” appears to the right of “ g “. From there, it became natural to write the function that sends x to g ( f ( x )) as “ g f “.

LAPD Helicopter Tracker with Real-Time Operating Costs

Hacker News
lapdhelicoptertracker.com
2025-11-21 22:11:07
Comments...

Friday Squid Blogging: New “Squid” Sneaker

Schneier
www.schneier.com
2025-11-21 22:08:09
I did not know Adidas sold a sneaker called “Squid.” As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered. Blog moderation policy....

OSS Friday Update

Lobsters
noteflakes.com
2025-11-21 21:46:32
Comments...
Original Article

OSS Friday Update

21·11·2025

Note: while my schedule is quite hectic these last few weeks, I’ve taken the decision to dedicate at least one day per week for developing open-source tools, and henceforth I plan to post an update on my progress in this regard every Friday evening. Here’s the first update:

UringMachine Grant Work

As I wrote here previously, a few weeks ago I learned I’ve been selected as one of the recipients of a grant from the Ruby Association in Japan, for working on UringMachine, a new gem that brings low-level io_uring I/O to Ruby. For this project, I’ve been paired with a terrific mentor - Samuel Williams - who is the authority on all things related to Ruby fibers. We’ve had a talk about the project and discussed the different things that I’ll be able to work on. I’m really glad to be doing this project under his guidance.

UringMachine implements a quite low-level API for working with I/O. You basically work with raw file descriptors, you can spin up fibers for doing multiple things concurrently, and there are low-level classes for mutexes and queues (based on the io_uring implementation of the futex API). Incidentally, I find it really cool that futexes can be used with io_uring to synchronize fibers, with very low overhead.

The problem with this, of course, is that this API is useless when you want to use the standard Ruby I/O classes, or any third-party library that relies on those standard classes.

This is where the Ruby fiber scheduler comes into the picture. Early on in my work on UringMachine, it occurred to me that the Fiber::Scheduler added to Ruby by Samuel is a perfect way to integrate such a low-level API with the Ruby I/O layer and the entire Ruby ecosystem. An implementation of Fiber::Scheduler for UringMachine would use the different scheduler hooks to punt work to the low-level UringMachine API.

So this week I finally got around to making some progress on the UringMachine fiber scheduler, and there’s finally a basic working version that can do basic I/O, as well as some other stuff like sleeping, waiting on blocking operations (such as locking a mutex or waiting on a queue), and otherwise managing the life cycle of a scheduler.

This is also a learning process. The Ruby IO class implementation is really complex: the io.c file itself is about 10K LOCs! I’m still figuring out the mechanics of the fiber scheduler as I go, and lots of things are still unclear, but I’m taking it one step at a time, and when I hit a snag I just try to take the problem apart and try to understand what’s going on. But now that I have moved from a rough sketch to something that works and has some tests, I intend to continue working on it by adding more and more tests and TDD’ing my way to an implementation that is both complete (feature-wise) and robust.

Here are some of the things I’ve learned while working on the fiber scheduler:

  • When you call Kernel.puts , the trailing newline character is actually written separately (i.e. with a separate write operation), which can lead to unexpected output if for example you have multiple fibers writing to STDOUT at the same time. To prevent this, Ruby seems to use a mutex (per IO instance) to synchronize writes to the same IO.

  • There are inconsistencies in how different kinds of IO objects are handled, with regards to blocking/non-blocking operation ( O_NONBLOCK ):

    • Files and standard I/O are blocking.
    • Pipes are non-blocking.
    • Sockets are non-blocking.
    • OpenSSL sockets are non-blocking.

    The problem is that for io_uring to function properly, the fds passed to it should always be in blocking mode. To rectify this, I’ve added code to the fiber scheduler implementation that makes sure the IO instance is blocking:

    def io_write(io, buffer, length, offset)
      reset_nonblock(io)
      @machine.write(io.fileno, buffer.get_string)
    rescue Errno::EINTR
      retry
    end
    
    def reset_nonblock(io)
      return if @ios.key?(io)
            
      @ios[io] = true
      UM.io_set_nonblock(io, false)  
    end
    
  • A phenomenon I’ve observed is that in some situations of multiple fibers doing I/O, some of those I/O operations would raise an EINTR , which should mean the I/O operation was interrupted because of a signal sent to the process. This is weird! I’m still not sure where this is coming from, certainly something I’ll ask Samuel about.

  • There’s some interesting stuff going on when calling IO#close . Apparently there’s a mutex involved, and I noticed two scheduler hooks are being called: #blocking_operation_wait which means a blocking operation that should be ran on a separate thread, and #block , which means a mutex is being locked. I still need to figure out what is going on there and why it is so complex. FWIW, UringMachine has a #close_async method which, as its name suggests, submits a close operation, but does not wait for it to complete.

Improving and extending the fiber scheduler interface

One of the things I’ve discussed with Samuel is the possibility of extending the fiber scheduler interface by adding more hooks, for example a hook for closing an IO (from what I saw there’s already some preparation for that in the Ruby runtime), or a hook for doing a splice . We’ve also discussed working with pidfd_open to prevent race conditions when waiting on child processes. I think there’s still a lot of cool stuff that can be done by bringing low-level I/O functionality to Ruby.

I’ve also suggested to Samuel to use the relatively recent io_uring_prep_waitid API to wait for child processes, and more specifically to do this in Samuel’s own io-event gem, which provides a low-level cross-platform API For building async programs in Ruby. With the io_uring version of waitid , there’s no need to use pidfd_open (in order to poll for readiness when the relevant process terminates). Instead, we use the io_uring interface to directly wait for the process to terminate. Upon termination, the operation completes and we get back the pid and status of the terminated process. This is also has the added advantage that you can wait for any child process, or any child process in the process group, which means better compatibility with the Process.wait and associated methods.

One problem is that the fiber scheduler process_wait hook is supposed to return an instance of Process::Status . This is a core Ruby class, but you cannot create instances of it. So, if we use io_uring to directly wait for a child process to terminate, we also need a way to instantiate a Process::Status object with the information we get back from io_uring. I’ve submitted a PR that hopefully will be merged before the release of Ruby 4.0. I’ve also submitted a PR to io-event with the relevant changes.

Going forward

So here’s where the UringMachine project is currently at:

If you appreciate my OSS work, please consider sponsoring me .

My Consulting Work

Apart from my open-source work, I’m also doing consulting work for. Here’s some of the things I’m currently working on for my clients:

  • Transitioning a substantial PostgreSQL database (~4.5TB of data) from RDS to EC2. This is done strictly for the sake of reducing costs. My client should see a reduction of about 1000USD/month.
  • Provisioning of machines for the RealiteQ web platform to be used for industrial facilities in India.
  • Exploring the integration of AI tools for analyzing the performance of equipment such as water pumps for water treatment facilities. I’m still quite sceptical about LLM’s being the right approach for this. ML algorithms might be a better fit. Maybe, we’ll see…

Wow, President Trump Fucking LOVES This Mamdani Guy

hellgate
hellgatenyc.com
2025-11-21 21:45:10
Asdfjasdfjlsakdfsdhfsodfh;asdfasdf...
Original Article
Wow, President Trump Fucking LOVES This Mamdani Guy
President Donald Trump talks after meeting with New York City Mayor-elect Zohran Mamdani in the Oval Office of the White House (AP Photo / Evan Vucci)

Fresh Hell

Asdfjasdfjlsakdfsdhfsodfh;asdfasdf

In a surreal, hilarious, and brain-exploding press conference after their meeting at the White House, President Donald Trump repeatedly stressed his adulation for Mayor-elect Zohran Mamdani and promised to do whatever was necessary to help him carry out his affordability agenda in New York City.

Where to begin? The part where Trump said that he was going to tell Con Edison to lower their rates? Or when Mamdani said that Israel is committing a genocide in Gaza and Trump didn't disagree with him? OR, Trump saying, over and over again, how "people would be shocked" about what Mamdani believes in. "We agree on a lot more than I would have thought," Trump said.

Would President Trump, a billionaire, feel comfortable living in Mamdani's New York City?

"I would feel very, very comfortable being in New York," Trump cooed.

Q: Are you affirming that you think President Trump is a fascist?

MAMDANI: I've spoken about--

TRUMP: That's okay. You can just say yes. I don't mind. pic.twitter.com/uWZFRcmGxB

— Aaron Rupar (@atrupar) November 21, 2025

Nothing in a blog post can possibly do justice to the incredible body language on display here—Mamdani's rigid spine, his hands clasped tightly to prevent them from punching a hole in the Oval Office window and jumping out of it, Trump touching Mamdani's arm, smiling at him like the son he's never had.

"I expect to be helping him, not hurting him. A big help," Trump said, when asked about previous threats the administration has made to New York City's funding and the prospect of sending in the National Guard.

Trump even found time to compliment Mamdani's pick for police commissioner, Jessica Tisch. "He retained someone who is a great friend of some people in my family—Ivanka. And they say she's really good, really competent," Trump said.

Even if you remember that this is all theater, that tomorrow morning Trump could wake up and "Truth" something idiotic and racist that erases everything that just transpired, it is remarkable how consistent Trump is in this one aspect of his brain: He loves winners, he loves ratings, and he won't bother with anyone who can't command them.

"I'll tell ya, the press has eaten this up," Trump said, noting that there were way more reporters jonesing for a chance to see him with Mamdani than any other foreign leader he's met with. "You people have gone crazy."

"I think he's different," Trump added. "He has a chance to do something great for New York. And he does need the help of the federal government to succeed...And we'll help him."

The Untold History of Arduino

Hacker News
arduinohistory.github.io
2025-11-21 21:29:28
Comments...
Original Article

日本語 · italiano · Deutsch · Français

Index

Why Are You Writing This?

Hello. My name is Hernando Barragán.

Through the years, and more recently due to the affairs between Arduino LLC and Arduino S.R.L. , I have received a lot of questions from people about the history of Wiring and, of course, Arduino.

I was also shown this US Federal Courts website , which presents documents citing my work to support the plaintiff’s claims which, in my opinion, contribute to the distortion of information surrounding my work.

The history of Arduino has been told by many people, and no two stories match. I want to clarify some facts around the history of Arduino, with proper supported references and documents, to better communicate to people who are interested, about Arduino’s origin.

As well, I will attempt to correct some things that have distorted my role or work by pointing out common mistakes, misleading information, and poor journalism.

I will go through a summary of the history first, then I will answer a series of questions that I have been often asked over the years.


Why Did You Create Wiring?

I started Wiring in 2003 as my Master’s thesis project at the Interaction Design Institute Ivrea (IDII) in Italy.

The objective of the thesis was to make it easy for artists and designers to work with electronics, by abstracting away the often complicated details of electronics so they can focus on their own objectives.

The full thesis document can be downloaded here: http://people.interactionivrea.org/h.barragan/thesis/thesis_low_res.pdf

Massimo Banzi and Casey Reas (known for his work on Processing ) were supervisors for my thesis.

The project received plenty of attention at IDII, and was used for several other projects from 2004, up until the closure of the school in 2005.

Because of my thesis, I was proud to graduate with distinction; the only individual at IDII in 2004 to receive the distinction. I continued the development of Wiring while working at the Universidad de Los Andes in Colombia, where I began teaching as an instructor in Interaction Design.

What Wiring is, and why it was created can be extracted from the abstract section of my thesis document. Please keep in mind that it was 2003, and these premises are not to be taken lightly. You may have heard them before recited as proclamations:

“… Current prototyping tools for electronics and programming are mostly targeted to engineering, robotics and technical audiences. They are hard to learn, and the programming languages are far from useful in contexts outside a specific technology …”

“… It can also be used to teach and learn computer programming and prototyping with electronics…”

“Wiring builds on Processing…”

These were the key resulting elements of Wiring:

  1. Simple integrated development environment (IDE), based on the Processing.org IDE running on Microsoft Windows, Mac OS X, and Linux to create software programs or “sketches” 1 , with a simple editor
  2. Simple “language” or programming “framework” for microcontrollers
  3. Complete toolchain integration (transparent to user)
  4. Bootloader for easy uploading of programs
  5. Serial monitor to inspect and send data from/to the microcontroller
  6. Open source software
  7. Open source hardware designs based on an Atmel microcontroller
  8. Comprehensive online reference for the commands and libraries, examples, tutorials, forum and a showcase of projects done using Wiring

How Was Wiring Created?

Through the thesis document, it is possible to understand the design process I followed. Considerable research and references to prior work has served as a basis for my work. To quickly illustrate the process, a few key points are provided below.

The Language

Have you ever wondered where those commands come from?

Probably one of the most distinctive things, that is widely known and used today by Arduino users in their sketches, is the set of commands I created as the language definition for Wiring.

  • pinMode()
  • digitalRead()
  • digitalWrite()
  • analogRead()
  • analogWrite()
  • delay()
  • millis()
  • etc…

Abstracting the microcontroller pins as numbers was, without a doubt, a major decision, possible because the syntax was defined prior to implementation in any hardware platform. All the language command naming and syntax were the result of an exhaustive design process I conducted, which included user testing with students, observation, analysis, adjustment and iteration.

As I developed the hardware prototypes, the language also naturally developed. It wasn’t until after the final prototype had been made that the language became solid and refined.

If you are still curious about the design process, it is detailed in the thesis document, including earlier stages of command naming and syntax that were later discarded.

The Hardware

From a designer’s point of view, this was probably the most difficult part to address. I asked for or bought evaluation boards from different microcontroller manufacturers.

Here are some key moments in the hardware design for Wiring.

Prototype 1

The first prototype for Wiring used the Parallax Javelin Stamp microcontroller. It was a natural option since it was programmed in a subset of the Java language, which was already being used by Processing.

Problem: as described in the thesis document on page 40, compiling, linking and uploading of user’s programs relied on Parallax’s proprietary tools. Since Wiring was planned as open source software, the Javelin Stamp was simply not a viable option.

Wiring Prototype 1 - Javelin Stamp Photo of Javelin Stamp used for first prototype for Wiring hardware.

For the next prototypes, microcontrollers were chosen on a basis of availability of open source tools for compiling, linking and uploading the user’s code. This led to discarding the very popular Microchip PIC family of microcontrollers very early, because, at the time (circa 2003), Microchip did not have an open source toolchain.

Prototype 2

For the second Wiring hardware prototype, the Atmel ARM-based AT91R40008 microcontroller was selected, which lead to excellent results. The first sketch examples were developed and command naming testing began. For example, pinWrite() used to be the name of the now ubiquitous digitalWrite() .

The Atmel R40008 served as a test bed for the digital input/output API and the serial communications API, during my evaluation of its capabilities. The Atmel R40008 was a very powerful microcontroller, but was far too complex for a hands-on approach because it was almost impossible to solder by hand onto a printed circuit board.

For more information on this prototype, see page 42 in the thesis document.

Wiring Prototype 2 - Atmel AT91R40008 Photo of Atmel AT91R40008 used for second Wiring hardware prototype.

Prototype 3

The previous prototype experiments led to the third prototype, where the microcontroller was downscaled to one still powerful, yet with the possibility of tinkering with it without the requirements of specialized equipment or on-board extra peripherals.

I selected the Atmel ATmega128 microcontroller and bought an Atmel STK500 evaluation board with a special socket for the ATmega128.

Wiring Prototype 3 - Atmel STK500 with ATmega128 Photo of Atmel STK500 with ATmega128 expansion.

Tests with the STK500 were immediately successful, so I bought a MAVRIC board from BDMICRO with the ATmega128 soldered. Brian Dean’s work on his MAVRIC boards were unparalleled at that time, and his work drove him to build a software tool to easily upload new programs to his board. It is still used today in the Arduino software, and is called “avrdude”.

As traditional COM ports were disappearing from computers, I selected FTDI hardware for communication through a USB port on the host computer. FTDI provided drivers for Windows, Mac OS X and Linux which was required for the Wiring environment to work on all platforms.

BDMICRO MAVRIC-II Photo of BDMICRO MAVRIC-II used for the third Wiring hardware prototype.

FTDI FT232BM Evaluation Board Photo of an FTDI FT232BM evaluation board used in the third Wiring hardware prototype.

The FTDI evaluation board was interfaced with the MAVRIC board and tested with the third Wiring prototype.

Wiring Prototype 3 - BDMICRO and FTDI - 1 Wiring Prototype 3 - BDMICRO and FTDI - 2

Testing with the BDMICRO MAVRIC-II board and FTDI-FT232BM.

In early 2004, based on the prototype using the MAVRIC board (Prototype 3), I used Brian Dean’s and Pascal Stang’s schematic designs as a reference to create the first Wiring board design. It had the following features:

  • ATmega128
  • FTDI232BM for serial to USB conversion
  • An on-board LED connected to a pin
  • A power LED and serial RX/TX LEDs

I used Eagle PCB from Cadsoft to design the schematic and printed circuit board.

Wiring board schematic Wiring board schematic.

Wiring board PCB Wiring board printed circuit board layout.

Along with the third prototype, the final version of the API was tested and refined. More examples were added and I wrote the first LED blink example that is still used today as the first sketch that a user runs on an Arduino board to learn the environment. Even more examples were developed to support liquid crystal displays (LCDs), serial port communication, servo motors, etc. and even to interface Wiring with Processing via serial communication. Details can be found on page 50 in the thesis document.

In March 2004, 25 Wiring printed circuit boards were ordered and manufactured at SERP , and paid for by IDII.

I hand-soldered these 25 boards and started to conduct usability tests with some of my classmates at IDII. It was an exciting time!

Wiring PCB first article Showing Off Wiring Board

Working with the first Wiring Boards Working with the first Wiring Boards

Photos of the first Wiring board

Continuing the Development

After graduating from IDII in 2004, I moved back to Colombia, and began teaching as an instructor in Interaction Design at the Universidad de Los Andes. As I continued to develop Wiring, IDII decided to print and assemble a batch of 100 Wiring boards to teach physical computing at IDII in late 2004. Bill Verplank (a former IDII faculty member) asked Massimo Banzi to send 10 of the boards to me for use in my classes in Colombia.

In 2004, Faculty member Yaniv Steiner , former student Giorgio Olivero, and information designer consultant Paolo Sancis started the Instant Soup Project , based on Wiring at IDII.

First Major Success - Strangely Familiar

In the autumn of 2004, Wiring was used to teach physical computing at IDII through a project called Strangely Familiar, consisting of 22 students, and 11 successful projects. Four faculty members ran the 4-week project:

  • Massimo Banzi
  • Heather Martin
  • Yaniv Steiner
  • Reto Wettach

It turned out to be a resounding success for both the students as well as the professors and teachers. Strangely Familiar demonstrated the potential of Wiring as an innovation platform for interaction design.

On December 16th, 2004, Bill Verplank sent an email to me saying:

[The projects] were wonderful. Everyone had things working. Five of the projects had motors in them! The most advanced (from two MIT grads - architect and mathematician) allowed drawing a profile in Proce55ing and feeling it with a wheel/motor run by Wiring…

It is clear that one of the elements of success was [the] use of the Wiring board.

Here is the brief for the course:

Here is a booklet with the resulting projects:

Working on Tug Tug (Haiyan Zhang) Tug Tug

Tug Tug phones by Haiyan Zhang (with Aram Armstrong)

Working on Commitment Radio Commitment Radio

Commitment Radio by David Chiu and Alexandra Deschamps-Sonsino

Working on Speak Out Speak Out

Speak Out by Tristam Sparks and Andreea Cherlaru (with Ana Camila Amorim)

Working on Feel the Music I Feel the Music I

Feel the Music I by James Tichenor and David A. Mellis

Working on The Amazing All Band Radio The Amazing All Band Radio

The Amazing All Band Radio by Oren Horev & Myriel Milicevic (with Marcos Weskamp)

The Rest of the World

In May 2005, I contracted Advanced Circuits in the USA to print the first 200 printed circuit boards outside of IDII, and assembled them in Colombia. I began selling and shipping boards to various schools and universities, and by the end of 2005, Wiring was being used around the world.

Wiring's Reach by 2005 “Wiring’s Reach by 2005” graphic, provided by Collin Reisdorf


When Did Arduino Begin and Why Weren’t You a Member of the Arduino Team?

The Formation of Arduino

When IDII manufactured the first set of Wiring boards, the cost was probably around USD$50 each. (I don’t know what the actual cost was, as I wasn’t involved in the process. However, I was selling the boards from Colombia for about USD$60.) This was a considerable drop in price from the boards that were currently available, but it was still a significant cost for most people.

In 2005, Massimo Banzi, along with David Mellis (an IDII student at the time) and David Cuartielles, added support for the cheaper ATmega8 microcontroller to Wiring. Then they forked (or copied) the Wiring source code and started running it as a separate project, called Arduino.

There was no need to create a separate project, as I would have gladly helped them and developed support for the ATmega8 and any other microcontrollers. I had planned to do this all along.

Future Plans for Wiring I had inadvertantly taken a photo of some notes about my plans for Wiring, in the photo of Karmen Franinovic (former IDII student from 2002 to 2004) testing a stretch sensor for a lamp in March 2004.

Wiring and Arduino shared many of the early development done by Nicholas Zambetti , a former IDII student in the same class as David Mellis. For a brief time, Nicholas had been considered a member of the Arduino Team.

Around the same time, Gianluca Martino (he was a consultant at SERP, the printed circuit board factory at Ivrea where the first Wiring boards were made), joined the Arduino Team to help with manufacturing and hardware development. So, to reduce the cost of their boards, Gianluca, with some help from David Cuartielles, developed cheaper hardware by using the ATmega8.

Arduino's First Prototype: Wiring Lite Apparently this is the first “Arduino” prototype - dubbed Wiring Lite. I think Massimo Banzi designed this one, but I’m unsure.

Arduino Extreme v2 Arduino Extreme v2 - “Second production version of the Arduino USB boards. This has been properly engineered by Gianluca Martino.”

Tom Igoe (a faculty member at the ITP at NYU 2 ) was invited by Massimo Banzi to IDII for a workshop and became part of the Arduino Team.

To this day, I do not know exactly why the Arduino Team forked the code from Wiring. It was also puzzling why we didn’t work together. So, to answer the question, I was never asked to become a member of the Arduino Team.

Even though I was perplexed by the Arduino Team forking the code, I continued development on Wiring, and almost all of the improvements that had been made to Wiring, by me and plenty of contributors, were merged into the Arduino source code. I tried to ignore the fact that they were still taking my work and also wondered about the redundancy and waste of resources in duplicating efforts.

By the end of 2005, I started to work with Casey Reas on a chapter for the book “ Processing: A Programming Handbook for Visual Artists and Designers .” The chapter presents a short history of electronics in the Arts. It includes examples for interfacing Processing with Wiring and Arduino. I presented those examples in both platforms and made sure the examples included worked for both Wiring and Arduino.

The book got a second edition in 2013 and the chapter was revised again by Casey and me, and the extension has been made available online since 2014.


Did The Arduino Team Work with Wiring Before Arduino?

Yes, each of them had experience with Wiring before creating Arduino.

Massimo Banzi taught with Wiring at IDII from 2004.

Massimo Banzi Teaching with Wiring Massimo Banzi teaching interaction design at IDII with Wiring boards in 2004.

David Mellis was a student at IDII from 2004 to 2005.

David Mellis at IDII A blurry version of David Mellis learning physical computing with Wiring in 2004.

In January 2005, IDII hired David Cuartielles to develop a couple of plug-in boards for the Wiring board, for motor control and bluetooth connectivity.

Wiring Bluetooth Plugin Wiring Motor Controller Plugin

Two plug-in boards developed at IDII by David Cuartielles and his brother. Bluetooth shield on the left, and a motor controller shield on the right.

I showed early versions of Wiring to Tom Igoe during a visit to ITP in New York in 2003. At the time, he had no experience with Atmel hardware, as Tom was using PIC microcontrollers at ITP as an alternative to the costly platforms like Parallax Basic Stamp or Basic X. One of Tom’s recommendations at this visit was: “well, do it for PIC, because this is what we use here.”

Years later, in 2007, Tom Igoe released the first edition of the “Making Things Talk” book published by O’Reilly 3 , which presents the use of both Wiring and Arduino.

Gianluca Martino originally worked for SERP (the factory that made the first 25 Wiring circuit boards) and later he founded Smart Projects SRL (April 1st, 2004). Smart Projects made the first batch of 100 Wiring boards for IDII to teach physical computing in 2004.


Programma2003 was a Microchip PIC microcontroller board developed by Massimo Banzi in 2003. After using BasicX to teach Physical computing in the winter of 2002, Massimo decided to do a board using the PIC chip in 2003. The problem with the PIC microcontrollers was that there wasn’t an open source toolchain available at the time, to use a language like C to program them.

Programma2003 Programma2003 board designed by Massimo Banzi in 2003

Because of the lack of an open source toolchain, Massimo decided to use an environment called JAL (Just Another Language) to program the PIC microcontroller. JAL was created by Wouter van Ooijen.

It consisted of the JAL compiler, linker, uploader, bootloader and examples for the PIC. However, the software would only run on Windows.

To make JAL easier to use, Massimo used the base examples from JAL and simplified some of them for the distribution package for IDII.

However, in 2003, most students at IDII used Mac computers. So I volunteered to help Massimo by making a small and simple environment for Mac OS X so students with a Mac could use it as well.

In my thesis document, I characterized Programma2003 as a non-viable model to follow, since other more comprehensive tools were already available in the market. The main problems were:

  • the language is far from useful in any other context (e.g. you can’t program your computer using JAL)
  • it’s arcane syntax and the hardware design made it highly unlikely to go somewhere in the future for teaching and learning
  • the board didn’t have a power LED (a design flaw)

It was impossible to know if it was powered or not (frustrating/dangerous in a learning environment) and an additional RS232 to USB expensive converter was required to connect it to a computer.

As a gesture to help Massimo’s Programma2003 project, I also wrote something I called Programma2003 Interface, which basically interfaced any serial communication between a microcontroller and a computer with the network. This expanded the prototyping toolbox at IDII. It allowed students to use software like Adobe Flash (formerly Macromedia) to communicate with a microcontroller.

Programma2003 Interface Code Programma2003 Interface Code


Why Hasn’t Arduino Acknowledged Wiring Better?

I don’t know.

The reference to Wiring on the Arduino.cc website, although it has improved slightly over time, is misleading as it tries to attribute Wiring to Programma2003.

Arduino.cc Credits Page Excerpt - 2016-02-23 Arduino.cc website version of Arduino’s History from https://www.arduino.cc/en/Main/Credits

Adding to the confusion is this Flickr photo album by Massimo Banzi:

https://www.flickr.com/photos/mbanzi/albums/72157633136997919/with/8610131426/

It is called “Teaching: IDII 2004 Strangely Familiar”. Strangely Familiar was taught with Wiring (see above). This photo album seems to associate the Programma2003 with the class, but it was, in fact, never used. It is odd that the Wiring boards are absent from the album, however one Wiring board picture does appear.

It is no secret that the acknowledgement of Wiring has been very limited in the past. Back in 2013, at Open Hardware Summit at MIT, during the panel “Implications of Open Source Business: Forking and Attribution”, David Mellis acknowledges, for the first time, that the Arduino Team hadn’t done a very good job acknowledging Wiring. Unfortunately, he didn’t go into details why they hadn’t.


The Plaintiff vs. The Defendant

I’ve been quiet about everything that has happened with Arduino for a long time. But now that people are fraudulently saying that my work is their’s, I feel like I need to speak up about the past.

For example, in the ongoing case between Arduino LLC and Arduino S.R.L., there is a claim , by the Plaintiff, such that:

34. Banzi is the creator of the Programma2003 Development Platform, a precursor of the many ARDUINO-branded products. See: http://sourceforge.net/projects/programma2003/ . Banzi was also the Master’s Thesis advisor of Hernando Barragan whose work would result in the Wiring Development Platform which inspired Arduino.

Here is what, in my opinion, is wrong with that claim:

  1. The Programma2003 was not a Development Platform, it was simply a board. There was no software developed by the Plaintiff to accompany that board.
  2. The link is empty, there are no files in that Sourceforge repository, so why present an empty repository as evidence?
  3. The idea that the mere fact that Banzi was my thesis advisor gives him some sort of higher claim to the work done on Wiring, is, to say the least, frustrating to read.

Further on:

39. The Founders, assisted by Nicholas Zambetti, another student at IDII, undertook and developed a project in which they designed a platform and environment for microcontroller boards (“Boards”) to replace the Wiring Development Project. Banzi gave the project its name, the ARDUINO project.

Here are the questions I’d ask “The Founders:”

  • Why did the “Wiring Development Project” need to be replaced?
  • Did you ask the developer if he would work with you?
  • Did you not like the original name? (Banzi gave the project its name, after all)

I know it might be done now and again, but, in my opinion, it is unethical and a bad example for academics to do something like this with the work of a student. Educators, more than anybody else, should avoid taking advantage of their student’s work. In a way, I still feel violated by “The Founders” for calling my work their’s.

It may be legal to take an open source software and hardware project’s model, philosophy, discourse, and the thousands of hours of work by its author, exert a branding exercise on it, and release it to the world as something “new” or “inspired”, but… is it right?


Continuous Misleading Information

Someone once said:

“If we don’t make things ultra clear, people draw their own conclusions and they become facts even if we never said anything like that.” 4

It seems to me that this is universally true, and especially if you mislead people with only slight alterations of the truth, you can have control over their conclusions.

Here are a couple of mainstream examples of misleading information.

The Infamous Diagram

Interaction Ivrea (Weird) Diagram http://blog.experientia.com/uploads/2013/10/Interaction_Ivrea_arduino.pdf

This diagram was produced to tell the story of the prototyping tools developed at IDII. It was beautifully done by Giorgio Olivero, using the content provided by the school in 2005, and released in 2006.

The projects presented in the red blobs, although they were made with Wiring, appear to be associated with Arduino at a time when Arduino didn’t even exist , nor was even close to being ready to do them.

Some of the authors of the projects inquired about the mistake, and why their projects were shifted to Arduino, but received no response.

Despite the fact that nothing was changed in this highly public document, I have to thank the support of the students who pointed it out and inquired about it.

The Arduino Documentary

Another very public piece of media from 2010 was The Arduino Documentary (written and directed by Raúl Alaejos, Rodrigo Calvo).

This one is very interesting, especially seeing it today in 2016. I think the idea of doing a documentary is very good, especially for a project with such a rich history.

Here are some parts that present some interesting contradictions:

1:45 - “We wanted it to be open source so that everybody could come and help, and contribute.” It is suggested here that Wiring was closed source. Because part of Wiring was based on Processing, and Processing was GPL open source, as well as all the libraries, Wiring, and hence Arduino, had to be open source. It was not an option to have it be closed source. Also, the insinuation that they made the software easier is misleading, since nothing changed in the language, which is the essence of the project’s simplicity.

3:20 - David Cuartielles already knew about Wiring, as he was hired to design two plug-in boards for it by IDII in 2005 as pointed out earlier in this document. David Mellis learned physical computing using Wiring as a student at IDII in 2004. Interestingly, Gianluca came in as the person who was able to design the board itself (he wasn’t just a contractor for manufacturing); he was part of the “Arduino Team”.

8:53 - David Cuartielles is presenting at the Media Lab in Madrid, in July 2005: “Arduino is the last project, I finished it last week. I talked to Ivrea’s technical director and told him: Wouldn’t it be great if we can do something we offer for free? he says - For free? - Yeah!” David comes across here as the author of a project that he completed “last week”, and convincing the “technical director” at IDII to offer it for free.

18:56 - Massimo Banzi:

For us at the beginning it was a specific need: we knew the school was closing and we were afraid that lawyers would show up one day and say - Everything here goes into a box and gets forgotten about. - So we thought - OK, if we open everything about this, then we can survive the closing of the school - So that was the first step.

This one is very special. It misleadingly presents the fact of making Arduino open source as the consequence of the school closing. This poses a question: why would a bunch of lawyers “put in a box” a project based on other open source projects? It is almost puerile. The problem is, common people might think this is true, forming altruistic reasons for the team to make Arduino open source.


Absence of Recognition Beyond Wiring

There seems to be a trend in how the Arduino Team fails to recognize significant parties that contributed to their success.

In October 2013, Jan-Christoph Zoels (a former IDII faculty member) wrote to the IDII community mail list, a message presenting the article released at Core77 about the Intel-Arduino news on Wired UK :

A proud moment to see Intel referring to an Interaction Ivrea initiative.

And a good investment too:

Arduino development was started and developed at Interaction Design Institute Ivrea with an original funding of circa 250.000€. Another good decision was to keep Arduino as open source at the end of Interaction Ivrea in 2005 before merging with Domus.

To which Massimo Banzi responded:

I would like to point out that we never got any funding from Ivrea for Arduino (apart from buying 50 of them in the last year of the institute)

250.000 EUR is ridiculous…

This article must be retracted now

Sorry JC but you had nothing to do.with this…. You can’t possibly try to get credit for.something you hadn’t been involved with

Celebration Email Thread Posting Celebration Email Thread Response

It was nice, however, to get this a few days later in the same email thread:

Celebration Email Thread Follow-up


Distorted Public Information

In this section, I just wanted to show a fraction of the many different articles (and other press) that have been written about Arduino, which include its history that is rarely told the same way twice.

So, please, read them at your leisure, and form your own opinions, and, definitely, ask questions!

Poor Journalism

It is rare to see well researched journalism these days. The articles below are excellent examples of that postulate.

Wired

In a 2008 Wired interview , Banzi explains how he did Arduino in a weekend:

The two decided to design their own board and enlisted one of Banzi’s students—David Mellis—to write the programming language for it. In two days, Mellis banged out the code ; three days more and the board was complete. They called it the Arduino, after a nearby pub, and it was an instant hit with the students.

This article has been written without any fact checking. It certainly doesn’t help that the interviewee isn’t telling them the right information.

IEEE Spectrum

Here is a 2011 IEEE Spectrum article , titled “The Making of Arduino”.

Again, the history is taken verbatim from the interviewee. I was not contacted before the article was published, even though I was mentioned. And I doubt that anyone from IDII was contacted.

Just one of the many confusing parts of Arduino’s history is in this quote:

Since the purpose was to create a quick and easily accessible platform, they felt they’d be better off opening up the project to as many people as possible rather than keeping it closed.

It was never closed.

Circuits Today

A 2014 article from Circuits Today has a very confusing opening:

It was in the Interactive Design Institute [sic] that a hardware thesis was contributed for a wiring design by a Colombian student named Hernando Barragan. The title of the thesis was “Arduino–La rivoluzione dell’open hardware” (“Arduino – The Revolution of Open Hardware”). Yes, it sounded a little different from the usual thesis but none would have imagined that it would carve a niche in the field of electronics.

A team of five developers worked on this thesis and when the new wiring platform was complete, they worked to make it much lighter, less expensive, and available to the open source community.

The title of my thesis is obviously wrong. There weren’t five “developers” working on the thesis. And the code was always open source.

Again, I wasn’t contacted for reference.

Makezine

In a 2013 interview by Dale Dougherty with Massimo Banzi , once again the story changes:

Wiring had an expensive board, about $100, because it used an expensive chip. I didn’t like that, and the student developer and I disagreed.

In this version of the story by Massimo Banzi, Arduino originated from Wiring, but it is implied that I was insistent on having an expensive board.

Regarding the “disagreement”: I never had a discussion with Massimo Banzi about the board being too expensive. I wish that he and I would have had more discussions on such matters, as I had with other advisors and colleagues, as I find it very enriching. The closest thing to a disagreement took place after a successful thesis presentation event, where Massimo showed some odd behaviour towards me. Because he was my advisor, I was at a disadvantage, but I asked Massimo why he was behaving badly towards me, to which I received no answer. I felt threatened, and it was very awkward.

His odd behaviour extended to those who collaborated with me on Wiring later on.

I decided that we could make an open source version of Wiring, starting from scratch. I asked Gianluca Martino [now one of the five Arduino partners] to help me manufacture the first prototypes, the first boards.

Here, Massimo is again implying that Wiring wasn’t open source, which it was. And also that they would build the software from “scratch”, which they didn’t.

Academic Mistakes

I understand how easy it is to engage people with good storytelling and compelling tales, but academics are expected to do their homework, and at least check the facts behind unsubstantiated statements.

In this book, Making Futures: Marginal Notes on Innovation, Design, and Democracy Hardcover – October 31, 2014 by Pelle Ehn (Editor), Elisabet M. Nilsson (Editor), Richard Topgaard (Editor):

Chapter 8: How Deep is Your Love? On Open-Source Hardware (David Cuartielles)

In 2005, at the Interaction Design Institute Ivrea, we had the vision that making a small prototyping platform aimed at designers would help them getting a better understanding of technology.

David Cuartielles’ version of Arduino’s history doesn’t even include Wiring.

This book has been released chapter by chapter under Creative Commons: http://dspace.mah.se/handle/2043/17985

Wiring as predecessor to Arduino:

Interview with Ben Fry and Casey Reas:

Safari Books Online, Casey Reas, Getting Started with Processing, Chapter One, Family Tree:

Nicholas Zambetti Arduino Project Page:

(Nicholas did a lot of work with both Wiring and Arduino)

Articles About Arduino vs. Arduino

Wired Italy - What’s happening in Arduino?

http://www.wired.it/gadget/computer/2015/02/12/arduino-nel-caos-situazione/

Repubblica Italy - Massimo Banzi: “The Reason of the War for Arduino”

http://playground.blogautore.repubblica.it/2015/02/11/la-guerra-per-arduino-la-perla-hi-tech-italiana-nel-caos/

Makezine - Massimo Banzi Fighting for Arduino

http://makezine.com/2015/03/19/massimo-banzi-fighting-for-arduino/

Hackaday - Federico Musto of Arduino SRL discusses Arduino legal situation

http://hackaday.com/2015/07/23/hackaday-interviews-federico-musto-of-arduino-srl/

Hackaday - Federico Musto of Arduino SRL shows us new products and new directions

http://hackaday.com/2016/01/04/new-products-and-new-directions-an-interview-with-federico-musto-of-arduino-srl/

Video

Massimo going to Ted Talk – candid (2012-08-06)

https://www.youtube.com/watch?v=tZxY8_CNiCw

This is a candid view of Massimo just before performing at a TED Talk. You can make your own mind up about the majority of the video, however, the most interesting comment, in my opinion, is at the end , where he says:

… Innovation without asking for permission. So, in a way, Open Source allows you to be innovative without asking for permission.


Thank You!

Thank you for taking time to read this. I think it is very important, not just in the academic world, to properly acknowledge the origin of things. As I learned from fantastic educators, doing this properly not only enriches your work, but also positions it better to allow others to investigate and see where your ideas come from. Maybe they will find other alternatives or improve what was done and better position their own ideas.

Personally, watching the outreach of what I created back in 2003 in so many different contexts, seeing those commands bringing to life people’s ideas and creations from all over the world, has brought me so many satisfactions, surprises, new questions, ideas, awareness and friendships. I am thankful for that.

I think it is important to know the past to avoid making the same mistakes in the future. Sometimes I wish I would have had a chance to talk about this differently, for a different motif. Instead, many times I have come across journalists and common people compromised in their independence. Either they had direct business with Arduino, or simply wanted to avoid upsetting Massimo Banzi. Or there are the close-minded individuals following a cause and refusing to see or hear anything different from what they believe. And then there are the individuals who are just part of the crowd that reproduce what they are told to reproduce. For those others, this document is an invitation to trust your curiosity, to question, to dig deeper in whatever interests you and is important to you as an individual or as a member of a community.

I’ll see you soon,

Hernando.


Index


Celebrating Books on Building a Better Future

Electronic Frontier Foundation
www.eff.org
2025-11-21 21:18:30
One of our favorite—and most important—things that we do at EFF is to work toward a better future. It can be easy to get caught up in all the crazy things that are happening in the moment, especially with the fires that need to be put out. But it’s just as important to keep our eyes on new technolog...
Original Article

One of our favorite—and most important—things that we do at EFF is to work toward a better future. It can be easy to get caught up in all the crazy things that are happening in the moment, especially with the fires that need to be put out. But it’s just as important to keep our eyes on new technologies, how they are impacting digital rights, and how we can ensure that our rights and freedoms expand over time.

That's why EFF is excited to spotlight two free book events this December that look ahead, providing insight on how to build this better future . Featuring EFF’s Executive Director Cindy Cohn, we’ll be exploring how stories, technology, and policy shape the world around us. Here’s how you can join us this year and learn more about next year’s events:

Exploring Progressive Social Change at The Booksmith - We Will Rise Again

December 2 | 7:00 PM Pacific Time | The Booksmith, San Francisco

We’re celebrating the release of We Will Rise Again, a new anthology of speculative stories from writers across the world, including Cindy Cohn, Annalee Newitz, Charlie Jane Anders, Reo Eveleth, Andrea Dehlendorf, and Vida Jame. This collection explores topics ranging from disability justice and environmental activism to community care and collective worldbuilding to offer tools for organizing, interrogating the status quo, and a blueprint for building a better world.

Join Cindy Cohn and her fellow panelists at this event to learn how speculative fiction helps us think critically about technology, civil liberties, and the kind of world we want to create. We hope to see some familiar faces there!

RSVP AND LEARN MORE

AI, Politics, and the Future of Democracy - Rewiring Democracy

December 3 | 6:00 PM Pacific Time | Virtual

We’re also geared up to join an online discussion with EFF Board Member Bruce Schneier and Nathan E. Sanders about their new book, Rewiring Democracy: How AI Will Transform Our Politics, Government, and Citizenship . In this time when AI is taking up every conversation—from generative AI tools to algorithmic decision-making in government—this book cuts through the hype to examine the ways that the technology is transforming every aspect of democracy, for good and bad.

Cindy Cohn will join Schneier and Sanders for a forward-looking conversation about what’s possible, and what’s at stake, as AI weaves itself into our governments and how to steer it in the right direction. We’ll see you online for this one!

RSVP AND LEARN MORE

Announcing Cindy Cohn's New Book, Privacy's Defender

In March we’ll be kicking off the celebration for Cindy Cohn’s new book, Privacy’s Defender , chronicling her thirty-year battle to protect everyone’s right to digital privacy and offering insights into the ongoing fight for our civil liberties online. Stay tuned for more information about our first event at City Lights on Tuesday, March 10!

The celebration doesn’t stop there. Look out for more celebrations for Privacy’s Defender throughout the year, and we hope we’ll see you at one of them. Plus, you can learn more about the book and even preorder it today !

PREORDER PRIVACY'S DEFENDER

You can keep up to date on these book events, and more EFF happenings when you sign up for our EFFector newsletter and check out our full event calendar .

Arduino Terms of Service and Privacy Policy update: setting the record straight

Hacker News
blog.arduino.cc
2025-11-21 21:13:11
Comments...
Original Article

The Arduino Terms of Service and Privacy Policy update: setting the record straight

November 21st, 2025

We’ve heard some questions and concerns following our recent Terms of Service and Privacy Policy updates. We are thankful our community cares enough to engage with us and we believe transparency and open dialogue are foundational to Arduino.

Let us be absolutely clear: we have been open-source long before it was fashionable. We’re not going to change now. The Qualcomm acquisition doesn’t modify how user data is handled or how we apply our open-source principles.

We periodically update our legal documents to reflect new features, evolving regulations, and best practices.

What remains the same

  • Open Source and reverse-engineering . Any hardware, software or services (e.g. Arduino IDE, hardware schematics, tooling and libraries) released with Open Source licenses remain available as before. Restrictions on reverse-engineering apply specifically to our Software-as-a-Service cloud applications. Anything that was open, stays open.
  • Ownership of your creations. The Terms of Service clarifies that the content you choose to publish on the Arduino platform remains yours, and can be used to enable features you’ve requested, such as cloud services and collaboration tools.
  • Minors’ data and privacy. Our privacy disclosures have been strengthened, including enhanced protections for minors’ data. We’ve updated our data retention policies and age limits to provide age-appropriate services. We limit data retention for inactive users by automatically deactivating their accounts after 24 months of inactivity, in which case usernames would still be preserved in the Arduino Forum to address an explicit request from the Forum community to maintain attribution for user-generated content; where user requests account deletion, the username would be promptly removed and related posts would become anonymous.

Why we updated our terms: clarity and compliance

These latest changes are about clarity, compliance, and supporting the innovative environment you expect.

Here’s what the updates actually cover:

  • Enhanced transparency around data practices: We’ve made our privacy disclosures more precise and more detailed, including what data we retain, to protect your privacy.
  • New product capabilities and AI: As we introduce optional AI-powered features, such as those in the Arduino UNO Q and Arduino App Lab, we needed to update our terms to reflect these new capabilities and encourage their safe, responsible, and ethical use.
  • More precise commercial terms: For users of our Premium Services, we’ve clarified billing mechanics, recurring payments, and refund rights to make purchasing and returns easier.
  • Legal compliance: We’ve updated language to address US-specific privacy laws, export controls, and other regulatory requirements, while ensuring compliance with global standards.

Our 20-year commitment to open-source is unwavering

We are very proud of the Arduino community, and we would like to reaffirm our fundamental, non-negotiable commitment to the principles that founded Arduino.

Please read the full Terms of Service and Privacy Policy , to appreciate how they support the innovative, collaborative environment you’ve come to expect.

If you have specific questions or concerns, please consult our detailed Q&A in our FAQ section or reach out to us at privacy@arduino.cc .

We are Arduino. We are open. We’re not going anywhere.

The Newly Reopened Studio Museum in Harlem Is Shockingly Huge and Shockingly Accessible

hellgate
hellgatenyc.com
2025-11-21 21:10:00
Co-curator Connie Choi discussed how the reopened institution will support Black Harlemites for generations to come....
Original Article

The new home of the Studio Museum in Harlem is well worth your time, and your money—$16 for an adult ticket, though the museum is free to visit on Sundays . After being closed for seven years to move into its new building on 125th Street, the cavernous new space is frankly magnificent. And the museum's institutional collection, on display throughout six floors, is a monumental survey of the Studio Museum's history of collecting Black and Afrodiasporic art, if a bit haphazard in its arrangement. On the other hand, there's just so much of it.

I met Connie Choi, a curator of the museum's permanent collection, in the entrance hall. Above us, we could see granite-colored staircases leading up to exhibition and residency spaces, and below us was "The Stoop," a kind of giant and elongated wooden staircase Choi said was meant to emulate the famed stoops of New York City brownstones. "This area is totally unticketed," Choi explained. "So it very much is intended for the public to just come in and hang out here. If you want to have a meeting here with someone, get a coffee. It is meant to be a communal space."

"From Now: A Collection in Context" (The Studio Museum in Harlem)

Give us your email to read the full story

Sign up now for our free newsletters.

Sign up

The senior population is booming. Caregiving is struggling to keep up

Hacker News
www.cnbc.com
2025-11-21 21:05:09
Comments...
Original Article

Maskot | Maskot | Getty Images

In November 2022, Beth Pinsker's 76-year-old mother began to get sick.

Ann Pinsker, an otherwise healthy woman, had elected to have a spinal surgery to preserve her ability to walk after having back issues. What Ann and Beth had thought would be a straightforward recovery process instead yielded complications and infections, landing Ann in one assisted living facility after another as her daughter navigated her care.

Eventually, by July of the following year, Ann died.

"We thought she'd be back up to speed a few weeks after hospital stay, rehab, home, but she had complications, and it was all a lot harder than she thought," Beth Pinsker, a certified financial planner and financial planning columnist at MarketWatch who has written a book on caregiving, told CNBC.

It wasn't Pinsker's first time navigating senior care. Five years before her mother's death, she took care of her father, and before that, her grandparents.

But throughout each of those processes, Pinsker said she noticed a significant shift in the senior caregiving sector.

"From the level of care that my grandparents received to the level of care that my mom received, prices skyrocketed and services decreased," she said.

It's evocative of a larger trend across the sector as the senior population in the U.S. booms and the labor force struggles to keep up.

Recent data from the U.S. Census Bureau found that the population of people ages 65 and older in the country grew from 12.4% in 2004 to 18% in 2024, and the number of older adults outnumbered children in 11 states — up from just three states in 2020.

Along with that population change came other shifts, including increased demand for care for older people.

According to the U.S. Bureau of Labor Statistics, the prices for senior care services are rising faster than the price of inflation. In September, the Consumer Price Index rose 3% annually, while prices for nursing homes and adult day services rose more than 4% over the same period.

But the labor force hasn't necessarily kept up with the surge.

The demand for home care workers is soaring as the gap widens, with a projected 4.6 million unfulfilled jobs by 2032, according to Harvard Public Health. And McKnight's Senior Living, a trade publication that caters to senior care businesses, found that the labor gap for long-term care is more severe than any other sector in health care, down more than 7% since 2020.

'A critical labor shortage'

That shortage is primarily driven by a combination of low wages, poor job quality and difficulty climbing the ranks, according to experts.

"This is coming for us, and we are going to have this create an enormous need for long-term care," Massachusetts Institute of Technology economist Jonathan Gruber told CNBC.

Gruber said the country is entering a period of "peak demand" for aging baby boomers, creating a situation where rising demand and pay do not sufficiently match up, leading to a "critical labor shortage."

On top of that, the jobs at nursing homes are often strenuous and vary in skills depending on the specific needs of each senior, he said, leading nursing assistants to be staffed in difficult jobs that often only pay slightly more than a retail job, despite requiring more training.

According to the BLS' most recent wage data from May 2024, the average base salary for home health and personal care aides was $16.82 per hour, compared with $15.07 per hour for fast food and counter workers.

"If we can create a better caring system with an entitlement to all care for those who need it, that will free millions of workers to make our economy grow, so this is a drag on economic growth," Gruber said.

Pinsker said she saw that shortage play out firsthand. At one of the assisted living facilities she toured for her mother, she noticed nurses wheeling residents into the dining hall for lunch at 10:30 a.m., an hour and a half before lunch would be served, because the home did not have enough caregivers to retrieve them at noon.

"They were bringing them in one at a time, whoever was available, seating them in rows at their tables, and just leaving them there to sit and wait," Pinsker said. "This was their morning activity for these people in this nursing home. … They just don't have enough people to push them around. That's what a staffing shortage looks like in real time."

Pinsker said her mother was placed in a nursing rehab facility, unable to walk or get out of bed, and that her facility had zero doctors on the premises. Most often, she said the facility was just staffed with business-level caretakers who change bedpans and clothing.

"They don't have enough doctors and registered nurses and physical therapists and occupational therapists and people to come and check blood pressure and take blood samples and that sort of stuff," she said. "They're short on all ends of the staffing spectrum."

Filling the gap

Gruber said there are three directions he thinks the country could go in to solve the labor gap: Pay more for these jobs, allow more immigration to fill the jobs or set up better career ladders within the sector.

"It's not rocket science — you've either got to pay more, or you've got to let in way more people. … There are wonderful, caring people all over the world who would like to come care for our seniors at the wages we're willing to pay, and we just have to let them in," Gruber said.

He's also part of an initiative in Massachusetts focused on making training more affordable for nurses to be able to climb the career ladder and pipelines to fill the shortages, which he said helps staff more people.

For Care.com CEO Brad Wilson, an overwhelming demand for senior care made it clear to the company that it needed to set up a separate category of job offerings. Care.com, which is most known for listing child care service jobs, met the demand and rolled out additional senior care options, as well as a tool for families trying to navigate what would work best for their situations and households.

Wilson said the company sees senior care as a $200 billion to $300 billion per year category. Now, it's the company's fastest-growing segment.

"We've heard from families that it's an enormous strain as they go through the senior care aspect of these things, because child care can be a little bit more planned, but sometimes your adult or senior care situation is sudden, and there's a lot to navigate," he said.

Care.com is also increasingly seeing demand rise for "house managers," Wilson said, who can help multiple people in a single household, as caregiving situations evolve.

"I can't underscore enough ... this is the most unforeseen part of the caregiving journey, and it's increasingly prevalent," he added.

And as the senior population booms, so too does the so-called sandwich generation, whose members are taking care of both their aging parents and their young children. Wilson said his family is in the thick of navigating caring for older family members while also raising three children.

"By 2034, there will actually be more seniors in this country than children," Wilson said, citing Census Bureau statistics. "Senior care is in a crisis. It's actually the very much unseen part of the caregiving crisis today, and we're really trying to bring some visibility to it and share that we have solutions that can help people."