@jordyd In theory, yes, but a given masto instance only has so many workers it can deploy, and when they /all/ get tied up trying to connect to a single other instance...

@noelle @jordyd Fairness algorithms can be surprisingly hard to tune correctly.

@jordyd @noelle The issue is how HTML protocol requests work. In order to try and send something, it has to wait for the other server to respond to initiate the send.

The server needs to send 300,000 things to the other server. It's not smart enough to say "Hey, let's skip all the requests to this server because it's being too slow"

So it just keeps sending them, one at a time, and waiting. While it does have multiple threads to do this, it seems all are taken up with the huge load.

@viomi @noelle oh god that sounds terrible

@viomi @noelle glad we all decided to do everything over HTTP instead of TCP (not on you guys, on the industry as a whole)

@jordyd @noelle The solution is, we add a feature to mastodon where if after a certain amount of times trying to make a connection, it temporarily stops trying to send those requests to server 2 and somehow stores them to wait to send them until the requests to said server are going faster. Not sure if you'd throw them all in the postresql or like.. Take up a shitton of RAM temporarily

@viomi @noelle who needs a database when you have a swap file amirite

@catoutofbed explanation above

@jordyd
i saw
it's still just
huh.....

@catoutofbed yeah a beautiful tragedy

@minego when I, from m.s, see this and want to reassure you that you'll get a message when it's resolved. But... like..... Yeah.... You wont see this for like 30min or something @noelle

@maloki @noelle I just got this, but I don't know how long ago you wrote it.

@jeff @noelle My impression was most of those instances are just gone outright. I migrated my follows here, and while trying to track down the missing ones, it just seems like a lot of them don't exist anymore, but Mastodon views it as a TLS failure.