David Chisnall (*Now with 50% more sarcasm!*)

@crankylinuxuser @stefano @wronglang

With small systems, you need complete duplicates for reliability because one failure can take out the entire system. But this isn’t true at larger scales.

If you run a personal mail server and want to handle failure of a single machine, you need two computers, which doubles the cost. If you are hosting enough that it’s worth having two computers and load balancing between them, if one can handle the load with slightly degraded performance, you can get away without redundancy, but if you want to preserve performance in the event of single-node failure, you need one spare computer, which adds only 50% to the cost.

At cloud scales, this should be much cheaper. Cloud storage is very reliable because they build rack-scale storage systems that expose virtual block devices to the rest of the system and do a load of error-correction coding across disks for redundancy. They assume some percentage of disks (based on models informed by prior failures) will fail and design the systems around this. For cool storage, MSR had a system (which, I think, was deployed around the time I left MS) that took advantage of really bad disks that were guaranteed for 50 rewrite cycles for cold data. Over 50% of the total used disk bandwidth was spent rewriting redundancy because those disks failed a lot, but that didn’t matter, the rack kept working.

And yet, in spite of that, everything else seems t9 have terrible uptime. When I was at MS, my personal email had higher uptime than my work email. My personal email was self hosted on a single machine. Work email was on M365 and was supposedly run on a cluster of mail servers spread across independent failure domains.

@Axel1973 @nixCraft

The last time I used dd on a Mac was a couple of months ago. I don’t know what your point is, it’s been there since macOS was called NeXTSTEP and it is still used for writing to raw devices, for example.

@nixCraft

It’s a good thing this is hard to do because the i and o keys are so far apart on the keyboard.

The kitten next door is seeking funding to evaluate an important hypothesis: fireworks make noise and move quickly and are therefore probably food, but are too far away to be eaten by conventional means.

@stefano

It really amazes me how much downtime big providers experience. I assume that economies of scale make it easy to build redundant hardware and transparent fail-over, but they often seem to have worse uptime that a computer run by a volunteer.