Federation of Imageboards

This topic started out in the webring thread and I'm transferring it here so it doesn't go off-topic. A federation is (to my understanding) a new term to describe a decentralized network websites where all hosts share the same implementation, meaning that each site, or rather instance, can be configured to be functionally identical as another instance. The only differences between instances are physical infrastructure and administration. As to how it's been implemented so far, consider Mastodon, which is the most prominent example of a federated network. While I have never used it myself, it is my understanding that you are able to view, reply to, reblog, and like any post that exists on one instance while posting from another instance. As to what it could mean for imageboards, I posted a few ideas in the other thread that I will reiterate here: >cross-site linking >hash-checking files before upload and automatically pulling, embedding, or censoring them from other instances if they already exist within the network >multiple front-ends like booru-styled or overboards without losing any functionality >non-administration-owned boards that do not have direct exposure to the Internet and inherit the administration policies they are connected to (i.e. a single board hosted by a lainon that was previously a server to lainchan could be reattached to, let's say, 420chan and automatically inherit their moderators and rules) The purpose of this thread isn't focused on how the hypothetical structure would function administratively and politically, though feel free to have that as a side discussion, but on the development of a set of standards and possibly a specification to make such a network possible. Even if nothing comes of it and if we're being honest here this is most likely the case, treat it as an exercise and a thought experiment. Two standards I consider key and need further development are: >clientside and serverside verification that backend software and security policies of an untrusted server are compliant >a singular protocol that handles the distribution of configuration files and variable data (e.g. lists of hosted files)

I think the first question that needs to be asked is this: What happened to NNTPchan? Why didn't it catch on? Is there something it did wrong that should be done differently?

>>66537 This seems vaguely familiar, like I'm sure I've heard about it at some point. How did it operate?

>>66538 NNTPChan (previously known as overchan) is a decentralized imageboard that uses the NNTP protocol (network-news transfer protocol) to synchronize content between many different servers. It utilizes cryptographically signed posts to perform optional/opt-in decentralized moderation. https://github.com/majestrate/nntpchan

>>66536 What's the point, though? Imageboards aren't like mastodon. You don't need an account and nothing is tied to you. There's no benefit for me, as an end user, if instead of opening lainchan's mega in one tab and whatever other imageboard in another versus going to some federated instance that has both, where I'll still open the boards in different tabs. All you end up with is a system where now mods not only have to moderated posts to their own instance, but also moderate content from federated instances and mitigate new attack vectors. >cross-site linking We can do this now by just copypasting the url. >hash-checking files before upload and automatically pulling, embedding, or censoring them from other instances if they already exist within the network What do you mean by pulling? My browser would pull from the instance with that file instead of the instance I'm browsing? Because that is a privacy issue. Or do you mean that if I upload an image that is already on a different instance, the instance I'm uploading would pull it from there? Because that's just a duplication of effort since the instance I uploaded too will need the image to compute the hash in the first place. Or do you mean I can tell the instance that I'm going to upload an image with x hash, and then it will check itself for it and say "don't waste the bandwidth we have that image already?" I can't imagine that image upload performance is bad enough that this makes a difference to hosts and it wouldn't stop people from spamming large files. >multiple front-ends like booru-styled or overboards without losing any functionality We have this on most chans already. It's been a long time since I checked, but I think most chans also implement the same API as 4chan if you want to have a consistent custom interface across different imageboard software. >non-administration-owned boards that do not have direct exposure to the Internet and inherit the administration policies they are connected to (i.e. a single board hosted by a lainon that was previously a server to lainchan could be reattached to, let's say, 420chan and automatically inherit their moderators and rules) Whats different about this than how it works with normal imageboards?

>>66540 >a system where now mods not only have to moderated posts to their own instance, but also moderate content from federated instances That's not a prerequisite for federation at all, and it's certainly not how most systems in the Fediverse operate.

>>66542 Maybe moderate is a poor choice of words there, but I intended to convey that instance operators still have to watch to ensure they're not federating with an instance that allows content they don't want to host and keep their blacklist updated accordingly. When such content is discovered on an instace they're federating with, they have to check to see if the instance operater is allowing it or if they've not had time to delete it so they can make an informed decision on whether or not to blacklist the instance. If, as in OPs design, you're able to post to chan 1 from chan 2, chan 1 also has to ensure that chan 2 is making an effort to prevent ban evasion. This is all just more work for an instance operator/moderator that does nothing to benefit the instance's imageboard itself.

>>66536 >hash-checking files before upload and automatically pulling, embedding, or censoring them from other instances if they already exist within the network This weakens the design. If the original instance of an image goes offline, the image won't be available. Something that works like retroshare would be good, over i2p. Posts get duplicated across people interested in a specific board, thus things become far more resilient, and everyone contributes, thus no centralization that'd give a target to state actors. NNTPchan also had a good structure, though it's federated rather than p2p, still giving a small group of targets. >>66537 I tried setting it up, but the setup guide was out of date and it didn't "just werk", so I gave up because I wasn't planning on running a main instance, just a backup for something, which could be achieved just as well with a simpler setup. If it was equally easy to set up to other imageboard software, it'd probably have caught on more.

>>66548 >nothing to benefit the instance's imageboard itself I'm not sure why you don't appear to see it as such, but the profound benefit is being able to participate in a broader community from the vantage of a smaller one.

>>66555 But I can already do that. Nothing stops me from posting on separate imageboards right now.

>>66540 >All you end up with is a system where now mods not only have to moderated posts to their own instance, but also moderate content from federated instances and mitigate new attack vectors. That's not what I mean, but it is very possible that I could be misunderstanding the definition of federation and an unclarified scope of what would be shared. It's my understanding that PeerTube instances, for example, doesn't distribute videos on one instance to another. I don't know if they can be configured to do this, or if it's just a simpler thing like being able to comment on other instances if your login is authenticated on the instance it is attached to, but the point is that the level of communication between instance is seemingly non-existent. This begs the question as to whether like PeerTube or invidious, as they seemingly do not communicate between instances, are even a federated network, but as I shared in the OP, the qualification is that they all share the same implementation. The kind of sharing I think you're talking about is rather excessive in my opinion, though it should be feasible under the standard. >All you end up with is a system where now mods not only have to moderated posts to their own instance, but also moderate content from federated instances and mitigate new attack vectors. Under your thinking, if posts are transferred to other instances, I don't see why moderation of these posts couldn't also be transferred >What do you mean by pulling? Might've been a bad use in retrospect. "Pulling" is duplication and "embedding" is having the browser retrieve the content from another instance on the network. All of these would be controllable by the end user, and there would be verification in place to ensure that all instances within a network would share the same security policies. For the rest of your post, I don't consider any of these ideas to be new. Even the last idea is just a VPN. My only objective is to formalize a structure using extant protocols that are at least capable of facilitating the degree of sharing described regardless of whether it is actually in use. >>66549 >Posts get duplicated across people interested in a specific board, thus things become far more resilient, and everyone contributes, thus no centralization that'd give a target to state actors. Resiliency is typically the goal when it comes to decentralization and something that should be accommodated, but imageboards lack the permanence you would find on forums, hence the existence of archives. Distributing resources to reduce the amount of administrative tasks and infrastructure needed for file hosting as well as reducing legal liability is something that I consider far more precedent than infrastructure resiliency. To make it a concise analogy, you're talking RAID 1 and I'm talking RAID 0.

>>66556 NTL, but there is an interface barrier; you need to be made aware of a particular imageboard, enter the url, and spend time engaging with it specifically. In practice the number of separate boards which you will consciously come back to, post on and read the content of is finite and restrained by this limitation.

>>66559 >That's not what I mean, but it is very possible that I could be misunderstanding the definition of federation and an unclarified scope of what would be shared. Generally federation means that there is sharing of data between instances. In your peertube example, each peertube shares what content it has and when a user of one instance requests content from another, the second instance streams it to the first instance which then delivers it to the user. It's similar to mastodon where when one users toots, the instance communicates that to instances it is federating with. Both are built on ActivityPub. >Under your thinking, if posts are transferred to other instances, I don't see why moderation of these posts couldn't also be transferred This doesn't address the root issue. Just like with mastodon, an instance operator can delete posts and tell other instances the posts are deleted. But the problem arises when an operator of an instance you're federating with does not moderate their content. This leads to instance operators having to set up blacklists for instances that don't moderate. Confirming that an instance should be blacklisted is a time consuming task - is the instance truly unmoderated and allowing spam, illegal content, or other content you dont want to host, or do they just get raided while their mods are asleep? This is time consuming as the number of instances, and often results in terrible "solutions" like reports on #fediblock that are easily trolled and lead to blacklisting of valid instances from false reports. It's an issue even on federated software where user authentication is required, to the point that many instances require applying to be accepted to prevent bad actors from mass registering and getting their instance blacklisted. This becomes even worse when you take into consideration the anonymous, no account nature of imageboards.

Bringing it up because it hasn't been mentioned: https://github.com/FChannel0/FChannel-Server I've done a bit of work on it and I'm doing an overhaul privately for now and correcting numerous mistakes I made in the past. When I'm done with the first major portion of my overhaul I'm going to start a Tor instance for lains, provided this thread doesn't give me a change of heart, in which case I would publish my changes and call it a day. I haven't actually administered an instance of it myself but I can see spam becoming an issue, which is apparently why the original instance (fchan.xyz) died killing the entire network save one instance (usagi.reisen) in the process. A point that was brought up in this thread and definitely contributed was the lack of moderation on some instances. Pizza deliveries were being left up too long. Although in my time, the spam didn't seem bad even in its heyday. I also don't recall it being easy to block instances, but what I do know is that you can ban media based on hash and the comment based on regexps. As for the protocol, I generally don't feel ActivityPub was even a great choice to begin with, and I considered designing my own protocol numerous times because while there was a goal to federate out of the echo chamber because I'm not sure that it would amount to anything of value. The only reason I can think of for why you would want to post to an imageboard from Mastodon would be "because I can."

>>66584 This is the same thing that happened to multichan and nntpchan.

This is an important idea that I don't think has gotten the discussion it deserved just yet.

>I generally don't feel ActivityPub was even a great choice to begin with Anyone have thoughts about Nostr for this? Was made in response to frustrations with ActivityPub and has been blowing up lately.