On 09/07/16, Richard Menedetter pondered and said...

*** Answering a message in area fsx_bot (fsx_bot).

Hi

HPT pushed the 2/2 into my dupebase. (the new fsx areas are not yet changed to delete dupes).

So it seems that if a message is split up, it does not use a new MSGID.
Or something else is irritating HPT.

Just another area you might want to look into.

07 Sep 2016 21:53, from fsxNet HUB -> All:

area msg days date.writ date.proc msgid path

CU, Ricsi

Thanks for this Ricsi. It was a test post of a larger file and indeed it did seem to 1) split the post across two posts and 2) use the same message ID for both. So that's where I can see how your system may go 'huh?'

I'll aim to remember to ask g00r00 about this when he's back. I'm picking
each posted message should have a unique message ID even if the message is being split across multiple posts automatically by MUTIL? Not sure but will find out when I can.

Best, Paul

--- Mystic BBS v1.12 A31 (Windows)
* Origin: Agency BBS | New Zealand | agency.bbs.geek.nz (21:1/101)

Re: Re: Stats Snapshot Test #8795 (1/2)
By: Avon to Richard Menedetter on Thu Sep 08 2016 07:04 am

I'll aim to remember to ask g00r00 about this when he's back. I'm picking each posted message should have a unique message ID even if the message is being split across multiple posts automatically by MUTIL? Not sure but will find out when I can.

I see quite a few dupes in my dupes area that I haven't been seeing before. I think they are bot posts probably posted by mutil. They seem to have the same msgid posted across areas and even nets.

I haven't looked deeper into that to get better answers but that is what I am thinking is happening.

I have actually used that feature of mutil to post 5 or so bbs ads at a time but I never looked at the msgid of those posts. Mystic will pass those msgs OK but other software (like hpt) will not.

...A camel is a horse planned by committee
--- SBBSecho 3.00-Linux
* Origin: The Rusty MailBox - Penticton, BC Canada (21:1/110.1)

Hi Avon!

08 Sep 2016 07:04, from Avon -> Richard Menedetter:

Thanks for this Ricsi. It was a test post of a larger file and indeed
it did seem to 1) split the post across two posts and 2) use the same message ID for both. So that's where I can see how your system may go 'huh?'

I'll aim to remember to ask g00r00 about this when he's back. I'm
picking each posted message should have a unique message ID even if
the message is being split across multiple posts automatically by
MUTIL? Not sure but will find out when I can.

I am no fido standards guru ... but I assume that every message should have its
own unique msgid.

BTW also messages from NecroBOT are affected ... I guess he uses mutil or so. (maybe he posts multiple messages in one go, and they share MSGID?

CU, Ricsi

--- GoldED+/LNX
* Origin: Wisdom is knowing what to do with what you know. (21:1/104)

On 09/07/16, Al pondered and said...

I see quite a few dupes in my dupes area that I haven't been seeing before. I think they are bot posts probably posted by mutil. They seem
to have the same msgid posted across areas and even nets.

Good to know and yes I suspect that is what you and Rici are seeing. One for g00r00 to comment on when he returns.

Thanks guys :)

Best, Paul

--- Mystic BBS v1.12 A31 (Windows)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

On 09/08/16, Richard Menedetter pondered and said...

I am no fido standards guru ... but I assume that every message should have its own unique msgid.

I'm the same, my guess is that also.

BTW also messages from NecroBOT are affected ... I guess he uses mutil
or so. (maybe he posts multiple messages in one go, and they share MSGID?

From what I can see here the posts I did one at a time went out with unique
IDs but the post that was long and was auto split 1/2 and 2/2 went out with
the same ID. That's what I think is happening but I may well be wrong :)

Best, Paul

--- Mystic BBS v1.12 A31 (Windows)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

On 09/07/16, Al said the following...

I see quite a few dupes in my dupes area that I haven't been seeing before. I think they are bot posts probably posted by mutil. They seem
to have the same msgid posted across areas and even nets.

I just ran a test using a .txt file that was 2218 lines long. Posted from the HUB into the FSX_BOT area. It split it into 3 posts.

+ Sep 09 13:30:13 Process: Post Messages
+ Sep 09 13:30:13 Post: 9 Subj: 2218 Lines Long Test (1/3)
+ Sep 09 13:30:13 Post: 9 Subj: 2218 Lines Long Test (2/3)
+ Sep 09 13:30:13 Post: 9 Subj: 2218 Lines Long Test (3/3)
+ Sep 09 13:30:13 Results: Posted 3 Msgs in 0.03s
+ Sep 09 13:30:13 Shutdown Normal (0)

When I look back at the posts using the full screen message editor I see
three posts (the break point seems to be closest to line 1000 for each) each with the same message ID and the third post containing the other SEEN-BY and PATH kludge lines.

So my guess is this is what other tossers will be baulking at? It would be interesting to get g00r00's take on this upon his return. Certainly my Mystic system at Agency imported them just fine as its within the same echosystem
but it does view them as seperate messages on import unlike on export from
the HUB.

! Sep 09 13:32:05 Import from c:\bbs\mystic\echomail\in\
+ Sep 09 13:32:05 Extracting 0000ffff.frp
- Sep 09 13:32:05 unzip -oqqjC c:\bbs\mystic\echomail\in\0000ffff.frp
*.pkt -d c:\bbs\mystic\temputil\
+ Sep 09 13:32:05 Importing 05a78468.pkt (21:1/100 to 21:1/101)
+ Sep 09 13:32:05 Import #49 to FSX_BOT
+ Sep 09 13:32:05 Import #50 to FSX_BOT
+ Sep 09 13:32:05 Import #51 to FSX_BOT
! Sep 09 13:32:05 Import from c:\bbs\mystic\echomail\in\unsec\
+ Sep 09 13:32:05 Results: 3 echo, 0 net, 0 dupes, 0 tossed in 0.09s
+ Sep 09 13:32:05 Shutdown Normal (0)

Best, Paul

--- Mystic BBS v1.12 A31 (Windows)
* Origin: fsxNet HUB (21:1/100)

Friday September 09 2016 13:40, you wrote to me:

So my guess is this is what other tossers will be baulking at? It
would be interesting to get g00r00's take on this upon his return. Certainly my Mystic system at Agency imported them just fine as its
within the same echosystem but it does view them as seperate messages
on import unlike on export from the HUB.

Yep, currently I think mystic is being a little lenient as far as dupes are concerned and I think hpt is being a little hard as far as dupes concerned.

I did post some examples of what I have seen in the hpt area but I think it is being developed in russia now. I don't know if the developers read my post or not.

If I remember those posts correctly (but I might not now) hpt was detecting dupes purely on the msgid, it wasn't taking the To: or Subject: into account. I
think it should do that.

The MSGID should be unique or it shows what happens in all the different software in use today!

... Put on your seatbelt - I'm gonna try something new!
--- GoldED+/LNX 1.1.5-b20160322
* Origin: The Rusty MailBox - Penticton, BC Canada (21:1/110)

I think we may have found the common factor in the Dupes: Time.
When we use QWK/QWKE the rep packet is all together (sent at the same time), the same is true of the MUTIL [PostTextFiles] if they are done together.
I looked at the two [ANSI] Twinkle BBS posts and I see (MSGID 49278a2b):

From: Pequito
To: All
Subj: [ANSI] Twinkle BBS
Date: 09/07/16 17:17
Base: FSXNet BOT

@TID: Mystic BBS 1.12 A31
@MSGID: 21:1/126 49278a2b
@SEEN-BY: 1/1 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 @SEEN-BY: 116 117 118 119 120 121 122 123 124 125 126 127 128 130 131 133 134 @SEEN-BY: 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 @SEEN-BY: 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 @PATH: 1/126 100

From: Pequito
To: All
Subj: [ANSI] Twinkle BBS
Date: 09/07/16 17:17
Base: BBS Advertisements

@TID: Mystic BBS 1.12 A31
@MSGID: 80:774/15 49278a2b
@SEEN-BY: 210/0 49/0 603/0 770/0 774/0 1 10 11 12 13 14 15 16 17 2 3 4 5 6 7 8 @SEEN-BY: 860/0 937/0
@PATH: 774/15 0

Seems we need to do something to split them up in time to fix the Dupes right now.

--- Mystic BBS v1.12 A31 (Raspberry Pi)
* Origin: Mystic Pi BBS bcw142.zapto.org (21:1/145)

On 09/09/16, Avon said the following...

When I look back at the posts using the full screen message editor I see three posts (the break point seems to be closest to line 1000 for each) each with the same message ID and the third post containing the other SEEN-BY and PATH kludge lines.

So my guess is this is what other tossers will be baulking at? It would
be interesting to get g00r00's take on this upon his return. Certainly
my Mystic system at Agency imported them just fine as its within the
same echosystem but it does view them as seperate messages on import unlike on export from the HUB.

Yes it's what they are baulking at. You know strict Fidonet considers them dupes due to same MSGID. If you recall g00r00 setup MSGID from date/time, but hadn't taken automatic stuff like QWK & the mutil postings in to account at
the time. I also noticed that 1006 line limit on message base messages, If
you remember the 1.12 Alpha 1 he had increase it to 10,000 but it seems to
have revered somewhere along the line of Alphas. Interesting that it split it up nicely, he did a good job there. I suspect he'll fix it with dupe checking and a count up or down to make them unique when he gets back to it.
In the meantime I stopped using QWK and will use that mutil post only as separate .ini files per message with at least a second between to keep it unique on MSGID.

--- Mystic BBS v1.12 A31 (Raspberry Pi)
* Origin: Mystic Pi BBS bcw142.zapto.org (21:1/145)

Hi Bcw142!

07 Sep 2016 21:47, from bcw142 -> Avon:

I think we may have found the common factor in the Dupes: Time.
When we use QWK/QWKE the rep packet is all together (sent at the same time), the same is true of the MUTIL [PostTextFiles] if they are done together.

Looks like it.
And HPT seems also to have a problem, as mine is configured to:
DupeBaseType HashDupesWMsgId
So it should not only rely on MSGID.

But anyhow ... creating identical MSGIDs is not a good idea!

Seems we need to do something to split them up in time to fix the
Dupes right now.

Actually QWK is a much bigger problem.
Those messages are entered by persons, and some will be deleted as dupes ...
so they are lost.
For mutil if an automated post is lost, usually this is not such a problem.

CU, Ricsi

--- GoldED+/LNX
* Origin: BEEP - Keyboard not found. Press any key to continue (21:1/104)