If I were to create my own instance federated with all the other instances, as of today, how much data would I be storing, since I would make a copy of all the content?

I know this will vary a lot, but I’m looking for a ballpark figure to have an idea. I don’t think it would be a lot, but I can’t find an estimate anywhere.

Reposted from https://lemmy.world/post/55030 as I think this community is probably a better fit

  • poVoq@slrpnk.net
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    1 year ago

    Our smaller instance that has been federating for a bit more than a year now (started in March 2022) is now at 2.4gb for the database and 7gb for the image storage (which probably needs some clean-up from previous image spam waves).

    • utzer [Friendica]@social.yl.ms
      link
      fedilink
      arrow-up
      0
      ·
      1 year ago

      @poVoq @ndr this will surely grow a lot, if you look at other activity pub compatible systems you’ll see a huge grow, it depends on the retention of “old” post and media, if you say just store all for a year you might keep it smaller, but if you want to replace #Reddit or so it would be better to keep stuff a bit longer, but then on the other hand the #Fediverse is probable not meant to store stuff for long term.

      On my Friendica node I have a rather short period to store foreign posts and media, and my storage is only about 47 GB, most of the media is stored in the database as well (easier and faster to backup, much slower to retrieve) and it is a single user instance with just a handful of bots besides the account I write from.

      • Petri@sh.itjust.works
        link
        fedilink
        arrow-up
        0
        ·
        1 year ago

        Keeping more of the history is probably a good thing if we want to replace Reddit. Think of all the homelab/server posts you’ve used that are over a year old. Good info can last a while.

        • utzer [Friendica]@social.yl.ms
          link
          fedilink
          arrow-up
          1
          ·
          1 year ago

          @sedawk probably… maybe threads can be closed and then some data can be removed.

          This is a culprit of the Fediverse players, they’re unlikely to keep all data forever (as some services do, at least until they’re gone for good like Google Waves). Storage costs money, the need will grow forever, maybe some more cost effective storage can be used for old data/posts/threads/media, just like internet archive does, they don’t used fast storage, so it takes seconds to load old website versions. But that also seems like a big leap for amateur technology enthusiasts (like most admins of Fediverse systems are).

  • Knighthawk 0811@lemmy.one
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 year ago

    it’s my understanding that you won’t make a full copy of everything. you’ll only copy the communities that are added and you won’t be copying the full history.

    I’m unsure how far back it goes but it might be just the one day. i saw talk of changing it to go back as far as someone actually scrolled and pressed next but that was just an idea at this point.

        • pimeys@lemmy.nauk.io
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 year ago

          I actually kind of like how Akkoma does this. It creates a constant size proxy with nginx, and all the images come from predetermined host instead of all over the net. It’s a good mixture of not using tens of gigabytes of space and still spreading the load a bit between the instances.

          • utzer [Friendica]@social.yl.ms
            link
            fedilink
            arrow-up
            1
            ·
            1 year ago

            @pimeys @knighthawk0811 yes, that sounds more privacy friendly, #Friendica also has a proxy function, it is also important that the proxy is just accessible for logged in users… so it get’s more complicated. Otherwise bad people will posts media some place and link it in scam pages/emails and such.

            • utzer [Friendica]@social.yl.ms
              link
              fedilink
              arrow-up
              1
              ·
              1 year ago

              @knighthawk0811 @pimeys I think the proxy does not really store the media files anymore, it just retrieves them and serves them… because of the potential danger by scammers. So it protects the users privacy but does not enable people to exploit the proxy as as host for scam.