Rocksolid Light

Welcome to RetroBBS

mail  files  register  newsreader  groups  login

Message-ID:  

Your best consolation is the hope that the things you failed to get weren't really worth having.


rocksolid / Rocksolid Nodes / Re: i2pn2 spool size

SubjectAuthor
* i2pn2 spool sizeRetro Guy
+- i2pn2 spool sizeanon
`* i2pn2 spool sizeNeodome Admin
 `* i2pn2 spool sizeRetro Guy
  `* i2pn2 spool sizeNeodome Admin
   `* i2pn2 spool sizeRetro Guy
    `- i2pn2 spool sizeNeodome Admin

1
Re: i2pn2 spool size

<20191125023701.456f463f@desktop14.dt>

 copy mid   Newsgroups: rocksolid.nodes
Path: i2pn2.org!rocksolid3!.POSTED.localhost!not-for-mail
From: retro.guy@retrobbs.rocksolidbbs.com.remove-lpj-this (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: i2pn2 spool size
Date: Mon, 25 Nov 2019 02:37:01 -0700
Organization: RetroBBS
Message-ID: <20191125023701.456f463f@desktop14.dt>
References: <2fa0abef5a22734274df8f094d97325f$1@news.novabbs.com><a6c706435262b2fefc8c9306d03314c4@def4>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Injection-Info: rocksolidbbs.com; posting-host="localhost:127.0.0.1";
logging-data="19265"; mail-complaints-to="usenet@rocksolidbbs.com"
To: All
X-Comment-To: All
In-Reply-To: <2fa0abef5a22734274df8f094d97325f$1@news.novabbs.com><a6c706435262b2fefc8c9306d03314c4@def4>
X-FTN-PID: Synchronet 3.17a-Linux Dec 29 2018 GCC 6.3.0
X-Newsreader: Claws Mail 3.11.1 (GTK+ 2.24.25; x86_64-pc-linux-gnu)
X-Gateway: retrobbs.rocksolidbbs.com [Synchronet 3.17a-Linux NewsLink 1.110]
 by: Retro Guy - Mon, 25 Nov 2019 09:37 UTC

On Sun, 24 Nov 2019 13:02:39 +0000
"anon" <anon@anon.com> wrote:

>
> this change is something you have to do when creating the partition,
> right ? so you need to create a new one, and then move the spool to
> it ?

Exactly.

> can you list what you have been doing, exactly ?

i2pn2 has been running out of inodes, while bytes availble is high
(many GBs) This is due to the small size of news articles just wasting
inodes.

I created a new partition not much bigger than the original, but with
more available inodes. The new partition 20% larger than the old, but
has 500% more inodes available.

If you format the partition as 'mkfs.ext4 -t news', you will end up
with an ext2 filesystem that is very appropriate for a tradspool inn2
setup.

I've done this two times, and both times I found it necessary to repair
the install with tdx-util to repair tradindexd inode mismatch errors.
This has happened each time. Once repaired 'tdx-util -F', it seems to
be working well.

I'm only working with i2pn2 right now, as it's not a peer to anyone by
my own systems. I don't want to break connectivity to anyone else until
I (think) I know what I'm doing. While the downtime kept articles from
retroBBS temporarily, they should be there now (i2pn2 is between
rocksolid2 and retroBBS).

I'll list any other issues here, or final results whether good or bad
when I'm done testing.

Retro Guy

Re: i2pn2 spool size

<992766df1f36ad8ce7f3923e4c04e049@def4>

 copy mid   Newsgroups: rocksolid.nodes
Path: i2pn2.org!rocksolid2!def5!POSTED.localhost!not-for-mail
From: anon@anon.com (anon)
Newsgroups: rocksolid.nodes
Message-ID: <992766df1f36ad8ce7f3923e4c04e049@def4>
Subject: Re: i2pn2 spool size
Date: Tue, 26 Nov 2019 21:25:08+0000
Organization: def5
In-Reply-To: <20191125023701.456f463f@desktop14.dt>
References: <20191125023701.456f463f@desktop14.dt>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
 by: anon - Tue, 26 Nov 2019 21:25 UTC

thanks for the walkthrough. that might come in handy one day.

Posted on def4

Re: i2pn2 spool size

<qsa03c$2p5j$1@neodome.net>

 copy mid   Newsgroups: rocksolid.nodes
Path: i2pn2.org!i2pn.org!news.neodome.net!.POSTED!not-for-mail
From: admin@neodome.net (Neodome Admin)
Newsgroups: rocksolid.nodes
Subject: Re: i2pn2 spool size
Date: Thu, 5 Dec 2019 04:12:28 -0000 (UTC)
Organization: Neodome
Message-ID: <qsa03c$2p5j$1@neodome.net>
References: <2fa0abef5a22734274df8f094d97325f$1@news.novabbs.com>
<a6c706435262b2fefc8c9306d03314c4@def4>
<20191125023701.456f463f@desktop14.dt>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 5 Dec 2019 04:12:28 -0000 (UTC)
Injection-Info: neodome.net; mail-complaints-to="abuse@neodome.net"
User-Agent: NewsTap/5.3.5 (iPhone/iPod Touch)
Cancel-Lock: sha1:+vENXHf9x0eTOh7UL6AIoV+PIlc=
 by: Neodome Admin - Thu, 5 Dec 2019 04:12 UTC

Retro Guy <retro.guy@retrobbs.rocksolidbbs.com.remove-lpj-this> wrote:
> On Sun, 24 Nov 2019 13:02:39 +0000
> "anon" <anon@anon.com> wrote:
>
>>
>> this change is something you have to do when creating the partition,
>> right ? so you need to create a new one, and then move the spool to
>> it ?
>
> Exactly.
>
>> can you list what you have been doing, exactly ?
>
> i2pn2 has been running out of inodes, while bytes availble is high
> (many GBs) This is due to the small size of news articles just wasting
> inodes.
>
> I created a new partition not much bigger than the original, but with
> more available inodes. The new partition 20% larger than the old, but
> has 500% more inodes available.

Perhaps ZFS might be better solution? No issues with inodes on ZFS.

--
Neodome

Re: i2pn2 spool size

<073897a0ef783b3d05db8690166cd1e4$1@www.rocksolidbbs.com>

 copy mid   Newsgroups: rocksolid.nodes
Path: i2pn2.org!rocksolid3!.POSTED.localhost!not-for-mail
From: retro.guy@retrobbs.rocksolidbbs.com.remove-zn5-this (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: i2pn2 spool size
Date: Thu, 5 Dec 2019 04:46:15 +0000
Organization: RetroBBS
Message-ID: <073897a0ef783b3d05db8690166cd1e4$1@www.rocksolidbbs.com>
References: <qsa03c$2p5j$1@neodome.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: rocksolidbbs.com; posting-host="localhost:127.0.0.1";
logging-data="3742"; mail-complaints-to="usenet@rocksolidbbs.com"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
To: Neodome Admin
X-Comment-To: Neodome Admin
In-Reply-To: <qsa03c$2p5j$1@neodome.net>
X-FTN-PID: Synchronet 3.17a-Linux Dec 29 2018 GCC 6.3.0
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on rocksolidbbs.com
X-Rslight-Site: $2y$10$AYdn3ZrCImeVZQDEz2OWiuqr1AvrYzTzr.q.KxErJewQm.RihAB1q
X-Gateway: retrobbs.rocksolidbbs.com [Synchronet 3.17a-Linux NewsLink 1.110]
 by: Retro Guy - Thu, 5 Dec 2019 04:46 UTC

To: Neodome Admin
Neodome Admin wrote:

> Retro Guy <retro.guy@retrobbs.rocksolidbbs.com.remove-lpj-this> wrote:
>> On Sun, 24 Nov 2019 13:02:39 +0000
>> "anon" <anon@anon.com> wrote:
>>
>>>
>>> this change is something you have to do when creating the partition,
>>> right ? so you need to create a new one, and then move the spool to
>>> it ?
>>
>> Exactly.
>>
>>> can you list what you have been doing, exactly ?
>>
>> i2pn2 has been running out of inodes, while bytes availble is high
>> (many GBs) This is due to the small size of news articles just wasting
>> inodes.
>>
>> I created a new partition not much bigger than the original, but with
>> more available inodes. The new partition 20% larger than the old, but
>> has 500% more inodes available.

> Perhaps ZFS might be better solution? No issues with inodes on ZFS.

I've used ZFS before and really like the ability to add partitions, etc. to the filesystem. That's a good idea. I'm planning to expand a backup machine local to me soon and may use ZFS.

If you don't mind me asking, what fs do you prefer for news spools yourself, are you using ZFS?

Retro Guy

--
Posted on RetroBBS

Re: i2pn2 spool size

<qsaekf$2dc9$1@neodome.net>

 copy mid   Newsgroups: rocksolid.nodes
Path: i2pn2.org!i2pn.org!news.neodome.net!.POSTED!not-for-mail
From: admin@neodome.net (Neodome Admin)
Newsgroups: rocksolid.nodes
Subject: Re: i2pn2 spool size
Date: Thu, 5 Dec 2019 08:20:31 -0000 (UTC)
Organization: Neodome
Message-ID: <qsaekf$2dc9$1@neodome.net>
References: <qsa03c$2p5j$1@neodome.net>
<073897a0ef783b3d05db8690166cd1e4$1@www.rocksolidbbs.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 5 Dec 2019 08:20:31 -0000 (UTC)
Injection-Info: neodome.net; mail-complaints-to="abuse@neodome.net"
User-Agent: NewsTap/5.3.5 (iPhone/iPod Touch)
Cancel-Lock: sha1:PevNO93m+AoZC4WiWiwFKMyoLzA=
 by: Neodome Admin - Thu, 5 Dec 2019 08:20 UTC

Retro Guy <retro.guy@retrobbs.rocksolidbbs.com.remove-zn5-this> wrote:
> To: Neodome Admin
> Neodome Admin wrote:
>
>> Retro Guy <retro.guy@retrobbs.rocksolidbbs.com.remove-lpj-this> wrote:
>>> On Sun, 24 Nov 2019 13:02:39 +0000
>>> "anon" <anon@anon.com> wrote:
>>>
>>>>
>>>> this change is something you have to do when creating the partition,
>>>> right ? so you need to create a new one, and then move the spool to
>>>> it ?
>>>
>>> Exactly.
>>>
>>>> can you list what you have been doing, exactly ?
>>>
>>> i2pn2 has been running out of inodes, while bytes availble is high
>>> (many GBs) This is due to the small size of news articles just wasting
>>> inodes.
>>>
>>> I created a new partition not much bigger than the original, but with
>>> more available inodes. The new partition 20% larger than the old, but
>>> has 500% more inodes available.
>
>> Perhaps ZFS might be better solution? No issues with inodes on ZFS.
>
> I've used ZFS before and really like the ability to add partitions, etc.
> to the filesystem. That's a good idea. I'm planning to expand a backup
> machine local to me soon and may use ZFS.
>
> If you don't mind me asking, what fs do you prefer for news spools
> yourself, are you using ZFS?
>
> Retro Guy
>

Yes, I’m running FreeBSD on ZFS. However, filesystem choice currently does
not affect my setup that much because I’m using CNFS buffers for spool
storage. However, I know some admins are using tradspool on ZFS to archive
text newsgroups and they have no issues even with very large quantities of
articles.

I always used CNFS buffers because previously I had limited space and
didn’t want to run out of it. When I moved server to bigger machine I kept
same setup because I was planning to play with some large databases and
wasn’t sure how much storage I would need for that. I think I have about 50
GBs for text groups, which gives me several months of retention.

--
Neodome

Re: i2pn2 spool size

<939e63a22f7a2a718926303360d0b9cc$1@news.novabbs.com>

 copy mid   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: Retro Guy@rslight.i2p (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: i2pn2 spool size
Date: Fri, 6 Dec 2019 01:58:21 -0000 (UTC)
Organization: Rocksolid Light
Message-ID: <939e63a22f7a2a718926303360d0b9cc$1@news.novabbs.com>
References: <qsa03c$2p5j$1@neodome.net> <073897a0ef783b3d05db8690166cd1e4$1@www.rocksolidbbs.com> <qsaekf$2dc9$1@neodome.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 6 Dec 2019 01:58:21 -0000 (UTC)
Injection-Info: i2pn2-novalink.localdomain; posting-account="retrobbs1";
logging-data="29558"; mail-complaints-to="usenet@i2pn2-novalink.localdomain"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on novabbs.com
X-Rslight-Site: $2y$10$xP5tPkuS9uFZn0OYuQ2w5euKcMCuItTHrZ6/g0D5OjEyoiByxI.x6
 by: Retro Guy - Fri, 6 Dec 2019 01:58 UTC

Neodome Admin wrote:

>>> Perhaps ZFS might be better solution? No issues with inodes on ZFS.
>>
>> I've used ZFS before and really like the ability to add partitions, etc.
>> to the filesystem. That's a good idea. I'm planning to expand a backup
>> machine local to me soon and may use ZFS.
>>
>> If you don't mind me asking, what fs do you prefer for news spools
>> yourself, are you using ZFS?
>>
>> Retro Guy
>>

> Yes, I’m running FreeBSD on ZFS. However, filesystem choice currently does
> not affect my setup that much because I’m using CNFS buffers for spool
> storage. However, I know some admins are using tradspool on ZFS to archive
> text newsgroups and they have no issues even with very large quantities of
> articles.

> I always used CNFS buffers because previously I had limited space and
> didn’t want to run out of it. When I moved server to bigger machine I kept
> same setup because I was planning to play with some large databases and
> wasn’t sure how much storage I would need for that. I think I have about 50
> GBs for text groups, which gives me several months of retention.

When I started using inn, I didn't realize there were options other than tradspool, so that's how it ran. I've been reading a bit about CNFS just to learn and it looks good.

Checking the change to increasing inodes, I see now that 5 months of text groups is using 23% of disk space, but only 12% of inodes. So it looks like space will now be the limitation, not inodes.

Still, ZFS makes it so easy to add space, I still need to consider it.

Retro Guy
--
Posted on Rocksolid Light

Re: i2pn2 spool size

<qsgfdo$1ch9$1@neodome.net>

 copy mid   Newsgroups: rocksolid.nodes
Path: i2pn2.org!i2pn.org!news.neodome.net!.POSTED!not-for-mail
From: admin@neodome.net (Neodome Admin)
Newsgroups: rocksolid.nodes
Subject: Re: i2pn2 spool size
Date: Sat, 7 Dec 2019 15:10:48 -0000 (UTC)
Organization: Neodome
Message-ID: <qsgfdo$1ch9$1@neodome.net>
References: <qsa03c$2p5j$1@neodome.net>
<073897a0ef783b3d05db8690166cd1e4$1@www.rocksolidbbs.com>
<qsaekf$2dc9$1@neodome.net>
<939e63a22f7a2a718926303360d0b9cc$1@news.novabbs.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 7 Dec 2019 15:10:48 -0000 (UTC)
Injection-Info: neodome.net; mail-complaints-to="abuse@neodome.net"
User-Agent: NewsTap/5.3.5 (iPhone/iPod Touch)
Cancel-Lock: sha1:Kq9CWrK1Wp9/BtgTNNnsN7GEFa4=
 by: Neodome Admin - Sat, 7 Dec 2019 15:10 UTC

Retro Guy <Retro Guy@rslight.i2p> wrote:
> Neodome Admin wrote:
>
>>>> Perhaps ZFS might be better solution? No issues with inodes on ZFS.
>>>
>>> I've used ZFS before and really like the ability to add partitions, etc.
>>> to the filesystem. That's a good idea. I'm planning to expand a backup
>>> machine local to me soon and may use ZFS.
>>>
>>> If you don't mind me asking, what fs do you prefer for news spools
>>> yourself, are you using ZFS?
>>>
>>> Retro Guy
>>>
>
>> Yes, I’m running FreeBSD on ZFS. However, filesystem choice currently does
>> not affect my setup that much because I’m using CNFS buffers for spool
>> storage. However, I know some admins are using tradspool on ZFS to archive
>> text newsgroups and they have no issues even with very large quantities of
>> articles.
>
>> I always used CNFS buffers because previously I had limited space and
>> didn’t want to run out of it. When I moved server to bigger machine I kept
>> same setup because I was planning to play with some large databases and
>> wasn’t sure how much storage I would need for that. I think I have about 50
>> GBs for text groups, which gives me several months of retention.
>
> When I started using inn, I didn't realize there were options other than
> tradspool, so that's how it ran. I've been reading a bit about CNFS just
> to learn and it looks good.

CNFS buffers are good if you don’t care about having less control over
expiration of articles. For example, if you have 50 GB buffer and all space
was used, new articles will just start overwriting oldest ones, thus making
sure that you won’t accidentally run out of space. It’s also harder to work
with individual articles in the CNFS buffer. If you have some kind of
script directly accessing articles in the spool you’ll have to use internal
INN program to print each article to stdout, or you’ll have to process them
as they come in.

> Checking the change to increasing inodes, I see now that 5 months of text
> groups is using 23% of disk space, but only 12% of inodes. So it looks
> like space will now be the limitation, not inodes.
>
> Still, ZFS makes it so easy to add space, I still need to consider it.

I would, too. Seems that ZFS was ready for production for a while.

--
Neodome

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor