• Mystic Dupe Detection

    From Black Panther@21:1/186 to All on Tuesday, November 21, 2017 16:22:25
    Hi All,

    I'm in the process of setting up a secondary feed for my Fidonet. Everything seems to working smoothly, until this afternoon...

    I noticed a dupe message get tossed into the echo, with the same @MSGID.

    Here are the kludge lines for the first message: (I'm removing the @ from the start of the lines, so it doesn't cause any problems)

    MSGID: 2:280/1027@fidonet 55f5fe17
    PID: MBSE-FIDO 1.0.6 (GNU/Linux-x86_64)
    CHRS: CP437 2
    TZUTC: 0100
    TID: MBSE-FIDO 1.0.6 (GNU/Linux-x86_64)
    PATH: 280/1027 261/38 15/0

    The second message, from the secondary feed:

    MSGID: 2:280/1027@fidonet 55f5fe17
    PID: MBSE-FIDO 1.0.6 (GNU/Linux-x86_64)
    CHRS: CP437 2
    TZUTC: 0100
    TID: MBSE-FIDO 1.0.6 (GNU/Linux-x86_64)
    PATH: 280/1027 261/38 393/68 770/1

    The information in my 'mailin.ini' file for Mutil, is set as this:

    [ImportEchoMail]

    dupe_msg_index = 4

    dupe_db_size = 250000

    strip_seenby = false


    ---

    Black Panther
    a.k.a. Dan Richter
    Sysop - Castle Rock BBS (RCS)
    The sparrows are flying again....

    --- Mystic BBS v1.12 A36 2017/11/20 (Linux/64)
    * Origin: Castle Rock BBS - castlerockbbs.com (21:1/186)
  • From g00r00@21:1/108 to Black Panther on Tuesday, November 21, 2017 19:20:24
    I noticed a dupe message get tossed into the echo, with the same @MSGID.

    MSGID isn't used for dupe detection with Mystic.

    Mystic evaluates actual message content to determine duplicate content. The factors used are:

    Message date, message time, echotag, from, to, subject, and actual message text (without kludge lines/seenbys).

    Unfortunately, in order for me to analyze what the situation is, I'd need the PKTs that had both messages in.

    --- Mystic BBS v1.12 A36 2017/11/21 (Windows/64)
    * Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)
  • From Black Panther@21:1/186 to g00r00 on Tuesday, November 21, 2017 17:43:33
    On 11/21/17, g00r00 said the following...

    MSGID isn't used for dupe detection with Mystic.

    Mystic evaluates actual message content to determine duplicate content. The factors used are:
    Message date, message time, echotag, from, to, subject, and actual
    message text (without kludge lines/seenbys).

    Ah, I didn't know that. Thank you.

    Unfortunately, in order for me to analyze what the situation is, I'd
    need the PKTs that had both messages in.

    What I'll do, is make copies of all of the incoming packets for a day or so.
    If I notice another one come through in that time, I'll have the packets.
    (That will ensure that no dupes get through to my system during that time) :)


    ---

    Black Panther
    a.k.a. Dan Richter
    Sysop - Castle Rock BBS (RCS)
    The sparrows are flying again....

    --- Mystic BBS v1.12 A36 2017/11/20 (Linux/64)
    * Origin: Castle Rock BBS - castlerockbbs.com (21:1/186)
  • From g00r00@21:1/108 to Black Panther on Tuesday, November 21, 2017 21:11:08
    What I'll do, is make copies of all of the incoming packets for a day or so. If I notice another one come through in that time, I'll have the packets. (That will ensure that no dupes get through to my system during that time) :)

    Sounds good. If you catch any fish let me know!

    --- Mystic BBS v1.12 A36 2017/11/21 (Windows/64)
    * Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)
  • From Black Panther@21:1/186 to g00r00 on Wednesday, November 22, 2017 18:53:58
    On 11/21/17, g00r00 said the following...

    What I'll do, is make copies of all of the incoming packets for a day so. If I notice another one come through in that time, I'll have the packets. (That will ensure that no dupes get through to my system dur that time) :)

    Sounds good. If you catch any fish let me know!

    I noticed a couple of dupes come in today. I was able to find which packets these messages arrived in. (I didn't really think about how many packets I would have in that directory in a day's time...)

    I can zip them up and email them to you.


    ---

    Black Panther
    a.k.a. Dan Richter
    Sysop - Castle Rock BBS (RCS)
    The sparrows are flying again....

    --- Mystic BBS v1.12 A36 2017/11/20 (Linux/64)
    * Origin: Castle Rock BBS - castlerockbbs.com (21:1/186)
  • From g00r00@21:1/108 to Black Panther on Thursday, November 23, 2017 12:59:33
    I noticed a couple of dupes come in today. I was able to find which packets these messages arrived in. (I didn't really think about how many packets I would have in that directory in a day's time...)

    I can zip them up and email them to you.

    Sure that would be great, thank you!

    --- Mystic BBS v1.12 A36 2017/11/22 (Windows/64)
    * Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)