Locally hosting an internet-connected server

Monday, 16 June 2025 09:20 pm
[personal profile] mjg59
I'm lucky enough to have a weird niche ISP available to me, so I'm paying $35 a month for around 600MBit symmetric data. Unfortunately they don't offer static IP addresses to residential customers, and nor do they allow multiple IP addresses per connection, and I'm the sort of person who'd like to run a bunch of stuff myself, so I've been looking for ways to manage this.

What I've ended up doing is renting a cheap VPS from a vendor that lets me add multiple IP addresses for minimal extra cost. The precise nature of the VPS isn't relevant - you just want a machine (it doesn't need much CPU, RAM, or storage) that has multiple world routeable IPv4 addresses associated with it and has no port blocks on incoming traffic. Ideally it's geographically local and peers with your ISP in order to reduce additional latency, but that's a nice to have rather than a requirement.

By setting that up you now have multiple real-world IP addresses that people can get to. How do we get them to the machine in your house you want to be accessible? First we need a connection between that machine and your VPS, and the easiest approach here is Wireguard. We only need a point-to-point link, nothing routable, and none of the IP addresses involved need to have anything to do with any of the rest of your network. So, on your local machine you want something like:

[Interface]
PrivateKey = privkeyhere
ListenPort = 51820
Address = localaddr/32

[Peer]
Endpoint = VPS:51820
PublicKey = pubkeyhere
AllowedIPs = VPS/0


And on your VPS, something like:

[Interface]
Address = vpswgaddr/32
SaveConfig = true
ListenPort = 51820
PrivateKey = privkeyhere

[Peer]
PublicKey = pubkeyhere
AllowedIPs = localaddr/32


The addresses here are (other than the VPS address) arbitrary - but they do need to be consistent, otherwise Wireguard is going to be unhappy and your packets will not have a fun time. Bring that interface up with wg-quick and make sure the devices can ping each other. Hurrah! That's the easy bit.

Now you want packets from the outside world to get to your internal machine. Let's say the external IP address you're going to use for that machine is 321.985.520.309 and the wireguard address of your local system is 867.420.696.005. On the VPS, you're going to want to do:

iptables -t nat -A PREROUTING -p tcp -d 321.985.520.309 -j DNAT --to-destination 867.420.696.005

Now, all incoming packets for 321.985.520.309 will be rewritten to head towards 867.420.696.005 instead (make sure you've set net.ipv4.ip_forward to 1 via sysctl!). Victory! Or is it? Well, no.

What we're doing here is rewriting the destination address of the packets so instead of heading to an address associated with the VPS, they're now going to head to your internal system over the Wireguard link. Which is then going to ignore them, because the AllowedIPs statement in the config only allows packets coming from your VPS, and these packets still have their original source IP. We could rewrite the source IP to match the VPS IP, but then you'd have no idea where any of these packets were coming from, and that sucks. Let's do something better. On the local machine, in the peer, let's update AllowedIps to 0.0.0.0/0 to permit packets form any source to appear over our Wireguard link. But if we bring the interface up now, it'll try to route all traffic over the Wireguard link, which isn't what we want. So we'll add table = off to the interface stanza of the config to disable that, and now we can bring the interface up without breaking everything but still allowing packets to reach us. However, we do still need to tell the kernel how to reach the remote VPN endpoint, which we can do with ip route add vpswgaddr dev wg0. Add this to the interface stanza as:

PostUp = ip route add vpswgaddr dev wg0
PreDown = ip route del vpswgaddr dev wg0


That's half the battle. The problem is that they're going to show up there with the source address still set to the original source IP, and your internal system is (because Linux) going to notice it has the ability to just send replies to the outside world via your ISP rather than via Wireguard and nothing is going to work. Thanks, Linux. Thinux.

But there's a way to solve this - policy routing. Linux allows you to have multiple separate routing tables, and define policy that controls which routing table will be used for a given packet. First, let's define a new table reference. On the local machine, edit /etc/iproute2/rt_tables and add a new entry that's something like:

1 wireguard


where "1" is just a standin for a number not otherwise used there. Now edit your wireguard config and replace table=off with table=wireguard - Wireguard will now update the wireguard routing table rather than the global one. Now all we need to do is to tell the kernel to push packets into the appropriate routing table - we can do that with ip rule add from localaddr lookup wireguard, which tells the kernel to take any packet coming from our Wireguard address and push it via the Wireguard routing table. Add that to your Wireguard interface config as:

PostUp = ip rule add from localaddr lookup wireguard
PreDown = ip rule del from localaddr lookup wireguard

and now your local system is effectively on the internet.

You can do this for multiple systems - just configure additional Wireguard interfaces on the VPS and make sure they're all listening on different ports. If your local IP changes then your local machines will end up reconnecting to the VPS, but to the outside world their accessible IP address will remain the same. It's like having a real IP without the pain of convincing your ISP to give it to you.

retrobootstrapping rust for some reason

Monday, 16 June 2025 10:07 am
graydon2: (Default)
[personal profile] graydon2
Elsewhere I've been asked about the task of replaying the bootstrap process for rust. I figured it would be fairly straightforward, if slow. But as we got into it, there were just enough tricky / non-obvious bits in the process that it's worth making some notes here for posterity.

UPDATE: someone has also scripted many of the subsequent snapshot builds covering many years of rust's post-bootstrap development. Consider the rest of this post just a verbose primer for interpreting their work.

context


Rust started its life as a compiler written in ocaml, called rustboot. This compiler did not use LLVM, it just emitted 32-bit i386 machine code in 3 object file formats (Linux PE, macOS Mach-O, and Windows PE).

We then wrote a second compiler in Rust called rustc that did use LLVM as its backend (and which, yes, is the genesis of today's rustc) and ran rustboot on rustc to produce a so-called "stage0 rustc". Then stage0 rustc was fed the sources of rustc again, producing a stage1 rustc. Successfully executing this stage0 -> stage1 step (rather than just crashing mid-compilation) is what we're going to call "bootstrapping". There's also a third step: running stage1 rustc on rustc's sources again to get a stage2 rustc and checking that it is bit-identical to the stage1 rustc. Successfully doing that we're going to call "fixpoint".

Shortly after we reached the fixpoint we discarded rustboot. We stored stage1 rustc binaries as snapshots on a shared download server and all subsequent rust builds were based on downloading and running that. Any time there was an incompatible language change made, we'd add support and re-snapshot the resulting stage1, gradually growing a long list of snapshots marking the progress of rust over time.

time travel and bit rot


Each snapshot can typically only compile rust code in the rust repository written between its birth and the next snapshot. This makes replaying the entire history awkward (see above). We're not going to do that here. This post is just about replaying the initial bootstrap and fixpoint, which happened back in April 2011, 14 years ago.

Unfortunately all the tools involved -- from the host OS and system libraries involved to compilers and compiler-components -- were and are moving targets. Everything bitrots. Some examples discovered along the way:

  • Modern clang and gcc won't compile the LLVM used back then (C++ has changed too much -- and I tried several CXXFLAGS=-std=c++NN variants!)
  • Modern gcc won't even compile the gcc used back then (apparently C as well!)
  • Modern ocaml won't compile rustboot (ditto)
  • 14-year-old git won't even connect to modern github (ssh and ssl have changed too much)


debian


We're in a certain amount of luck though:

  • Debian has maintained both EOL'ed docker images and still-functioning fetchable package archives at the same URLs as 14 years ago. So we can time-travel using that. A VM image would also do, and if you have old install media you could presumably build one up again if you are patient.
  • It is easier to use i386 since that's all rustboot emitted. There's some indication in the Makefile of support for multilib-based builds from x86-64 (I honestly don't remember if my desktop was 64 bit at the time) but 32bit is much more straightforward.
  • So: docker pull --platform linux/386 debian/eol:squeeze gets you an environment that works.
  • You'll need to install rust's prerequisites also: g++, make, ocaml, ocaml-native-compilers, python.


rust


The next problem is figuring out the code to build. Not totally trivial but not too hard. The best resource for tracking this period of time in rust's history is actually the rust-dev mailing list archive. There's a copy online at mail-archive.com (and Brian keeps a public backup of the mbox file in case that goes away). Here's the announcement that we hit a fixpoint in April 2011. You kinda have to just know that's what to look for. So that's the rust commit to use: 6daf440037cb10baab332fde2b471712a3a42c76. This commit still exists in the rust-lang/rust repo, no problem getting it (besides having to copy it into the container since the container can't contact github, haha).

LLVM


Unfortunately we only started pinning LLVM to specific versions, using submodules, after bootstrap, closer to the initial "0.1 release". So we have to guess at the LLVM version to use. To add some difficulty: LLVM at the time was developed on subversion, and we were developing rust against a fork of a git mirror of their SVN. Fishing around in that repo at least finds a version that builds -- 45e1a53efd40a594fa8bb59aee75bb0984770d29, which is "the commit that exposed LLVMAddEarlyCSEPass", a symbol used in the rustc LLVM interface. I bootstrapped with that (brson/llvm) commit but subversion also numbers all commits, and they were preserved in the conversion to the modern LLVM repo, so you can see the same svn id 129087 as e4e4e3758097d7967fa6edf4ff878ba430f84f6e over in the official LLVM git repo, in case brson/llvm goes away in the future.

Configuring LLVM for this build is also a little bit subtle. The best bet is to actually read the rust 0.1 configure script -- when it was managing the LLVM build itself -- and work out what it would have done. But I have done that and can now save you the effort: ./configure --enable-targets=x86 --build=i686-unknown-linux-gnu --host=i686-unknown-linux-gnu --target=i686-unknown-linux-gnu --disable-docs --disable-jit --enable-bindings=none --disable-threads --disable-pthreads --enable-optimized

So: configure and build that, stick the resulting bin dir in your path, and configure and make rust, and you're good to go!
root@65b73ba6edcc:/src/rust# sha1sum stage*/rustc
639f3ab8351d839ede644b090dae90ec2245dfff  stage0/rustc
81e8f14fcf155e1946f4b7bf88cefc20dba32bb9  stage1/rustc
81e8f14fcf155e1946f4b7bf88cefc20dba32bb9  stage2/rustc


Observations


On my machine I get: 1m50s to build stage0, 3m40s to build stage1, 2m2s to build stage2. Also stage0/rustc is a 4.4mb binary whereas stage1/rustc and stage2/rustc are (identical) 13mb binaries.

While this is somewhat congruent with my recollections -- rustboot produced code faster, but its code ran slower -- the effect size is actually much less than I remember. I'd convinced myself retroactively that rustboot was produced abysmally worse code than rustc-with-LLVM. But out-of-the-gate LLVM only boosted performance by 2x (and cost of 3x the code size)! Of course I also have a faster machine now. At the time bootstrap cycles took about a half hour each (according to this: 15 minutes for the 2nd stage).

Of course you can still see this as a condemnation of the entire "super slow dynamic polymorphism" model of rust-at-the-time, either way. It may seem funny that this version of rustc bootstraps faster than today's rustc, but this "can barely bootstrap" version was a mere 25kloc. Today's rustc is 600kloc. It's really comparing apples to oranges.

Photo cross-post

Monday, 16 June 2025 12:20 pm
andrewducker: (Default)
[personal profile] andrewducker


I done found me a tree goblin!
Original is here on Pixelfed.scot.

Photo cross-post

Sunday, 15 June 2025 05:15 am
andrewducker: (Default)
[personal profile] andrewducker


It was about ten seconds later that we realised how terrible Crocs are for climbing.
Original is here on Pixelfed.scot.

Confused by Disney ineptitude

Saturday, 14 June 2025 12:03 pm
andrewducker: (Default)
[personal profile] andrewducker
Two weeks after seeing the CGI Lilo and Stitch at the cinema I'm watching the original with the kids. And it's so much better. The direction, the writing, the acting are all just much higher quality.

The remake felt much clumsier. And I don't really understand why.

Edit: Just realised that they entirely cut the Ugly Duckling part from the remake. Why would you do that? It's key to Stitch's arc!
And all of the bits where Lilo how to be like Elvis.

In fact, nearly all of the bits where Lilo talked to Stitch and built a relationship with him.

In (near) Prague

Saturday, 14 June 2025 08:18 am
rmc28: Rachel in hockey gear on the frozen fen at Upware, near Cambridge (Default)
[personal profile] rmc28

On an ice hockey camp in Slaný, near Prague. I flew out on Thursday afternoon with two friends from Kodiaks. We arrived at the rink hotel in time to check in, have a little walk down to the nearby supermarket and get food, and settle in for the night. For reasons the three of us were all sharing a dormitory room the first night, and we decided the perfect film to watch over our picnic dinner was Inside Out 2 - also set at a 3-day hockey camp. I hadn't seen it before, though the other two had, and I enjoyed it very much.

Friday morning was pretty relaxed; a fourth Kodiak joined us after leaving home at awful-o-clock in the morning, and we were moved into the nicer ensuite twin rooms in pairs for the rest of the camp. We met in the dressing room at 1pm, were on ice at 2pm and again at 6pm, with a stickhandling session in between. Then dinner at 8 and falling into bed not long after.

It's excellent coaching, I'm being pushed well out of my comfort zone and the balance of drill and rest in each session and between sessions is just right. I hit my "cannot actually skate any more" limit about 3 minutes before the end of the last ice session.

Today will be two ice sessions at either end of the day, with video review (argh), optional swim+spa (yes!), and stickhandling again in between. My muscles this morning are making themselves known but I'm not exhausted. All is good. Time to go get changed.

andrewducker: (my brain)
[personal profile] andrewducker
Reading this article on advice to teachers in the UK about using AI, they suggest using it for things like "marking quizzes" and "generating routine letters".

And what really annoys me about this is that it's a perfect example of where simple automation could be used without the need for AI.

The precise example in the article is "Generate a letter to parents about a head lice outbreak." - which is a fairly common thing to happen in schools. So why on earth isn't there one standard letter per school, if not one standard letter for the whole country, that can be reused by absolutely everyone whenever this standard event happens? Why does this require AI to generate a new one every time, rather than just being a standard email that gets sent?

Same with marking quizzes. If children get multiple-choice quizzes regularly across all schools, and marking them uses precious teacher time, why is there not a standard piece of software, paid for once (or written once internally) which enables all children to do quizzes in a standard way, and get them marked automatically?

If we're investing a bunch of money into automating the various processes that teachers spend far too much time on, start with simple automation, which is cheap, easy, and reliable.

Also, wouldn't it be sensible to do some research into how accurately AI marks homework *before* you tell teachers to use it to do that? Here's some research from February which shows that its agreement with examiners was only 0.61 (where 1.00 would be perfect agreement). So I'm sceptical about the quality of the marking it's going to be doing...

performance of random floats (amended)

Monday, 9 June 2025 10:00 pm
fanf: (Default)
[personal profile] fanf

After I found some issues with my benchmark which invalidated my previous results, I have substantially revised my previous blog entry. There are two main differences:

  • A proper baseline revealed that my amd64 numbers were nonsense because I wasn’t fencing enough, and after tearing my hair out and eventually fixing that I found that the bithack conversion is one or two cycles faster.

  • A newer compiler can radically improve the multiply conversion on arm64 so it’s the same speed as the bithack conversion; I've added some source and assembly snippets to the blog post to highlight how nice arm64 is compared to amd64 for this task.

Page generated Thursday, 19 June 2025 02:24 pm
Powered by Dreamwidth Studios