The issue hit again at 1/100, I have sent you some log files in the form of 2
x .zip files :)

Best, Paul

--- Mystic BBS v1.12 A36 2017/11/27 (Windows/32)
* Origin: fsxNet HUB (21:1/100)

The issue hit again at 1/100, I have sent you some log files in the form of 2 x .zip files :)

Well at least we (should) have some logging now.

--- Mystic BBS v1.12 A36 2017/11/29 (Windows/64)
* Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)

On 11/29/17, g00r00 pondered and said...

of 2 x .zip files :)

Well at least we (should) have some logging now.

Yes yay for that... I was getting worried I was chasing ghosts..

--- Mystic BBS v1.12 A36 2017/11/26 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

The issue hit again at 1/100, I have sent you some log files in the form of 2 x .zip files :)

This was perfect, thank you. Unfortunately I missed a few places where I needed to add to the busy log so I can't see exactly where it happens like I thought I would be able to.

I am uploading a new build to prealpha in a few minutes that should have all
of the proper logging. I do plan to set up some tests though soon (ish) because I can see what happens I just don't have enough details.

--- Mystic BBS v1.12 A36 2017/11/29 (Windows/64)
* Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)

On 11/29/17, g00r00 pondered and said...

This was perfect, thank you. Unfortunately I missed a few places where I needed to add to the busy log so I can't see exactly where it happens
like I thought I would be able to.

I am uploading a new build to prealpha in a few minutes that should have all of the proper logging. I do plan to set up some tests though soon (ish) because I can see what happens I just don't have enough details.

OK thanks :) I will update either tonight / tomorrow night... got a bit on
this evening being a taxi service for my teenagers..

I was wondering if it was related to fidopoll running, a packet comes in via mis2, event runs to call mutil. mutil can't run as fidopoll is running
polling 99 nodes, so it errors with a fatal error after timing out/waiting as long as it can... in doing this fidopoll then skips the rest of the polling nodes with the bsy flags.. something like this is what I have observed I
think :)

--- Mystic BBS v1.12 A36 2017/11/26 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

On 11/30/17, Avon said the following...

I was wondering if it was related to fidopoll running, a packet comes in via mis2, event runs to call mutil. mutil can't run as fidopoll is
running polling 99 nodes, so it errors with a fatal error after timing out/waiting as long as it can... in doing this fidopoll then skips the rest of the polling nodes with the bsy flags.. something like this is what I have observed I think :)

Is everything being called from Mystic events? From what I've seen here, if there is an event running, such as fidopoll, Mystic would wait until that is completed before attempting to call the 'mailin' process.

There is one of my events, polling for newsgroups, that takes quite awhile to run. There are times, when that is running, that the events tab in MIS2 will show 2 or 3 other events waiting to run next.


---

Black Panther
a.k.a. Dan Richter
Sysop - Castle Rock BBS (RCS)
The sparrows are flying again....

--- Mystic BBS v1.12 A36 2017/11/27 (Linux/64)
* Origin: Castle Rock BBS - castlerockbbs.com (21:1/186)

I was wondering if it was related to fidopoll running, a packet comes in via mis2, event runs to call mutil. mutil can't run as fidopoll is
running polling 99 nodes, so it errors with a fatal error after timing out/waiting as long as it can... in doing this fidopoll then skips the rest of the polling nodes with the bsy flags.. something like this is what I have observed I think :)

The short answer is that FIDOPOLL was doing stuff and MUTIL was trying to get busy rights to every node. That is normal and the reason BUSY rights exist, but its not clear where it all went wrong.

Long answer:

The way it works (or is supposed to work) is that while MUTIL is waiting, it creates BUSY files for every node it can as they become available so that FIDOPOLL or the BINKP server cannot interfere with the import/export.

MUTIL needs full control because it needs to access packets for import and export and it cannot do that at the same time they are being transferred. Everytihng would be a big clump of corrupted files if it did. As a result it has to refuse to run unless it can get "busy rights" to every single node in your system.

MUTIL checks every 1 second for up to a minute and ends either when it gets the "busy rights" for every node or 1 minute passes. If one minute passes, it removes all of the busy files it created while waiting, and exits without exporting or importing anything. It tracks the BUSY files it was able to create while waiting and rolls it back. If it gets BUSY files for every node, then it does its magic, and removes the BUSY rights afterwards.

FIDOPOLL and MIS BINKP are less aggressive because they known what their workload will be. Since FIDOPOLL is contacting one node at a time, it will only BUSY the current node it is polling. In the case of the BINKP server it only locks the node that authenticates. Still, if even one node is locked by FIDOPOLL or MIS then MUTIL has to refuse to do anything because it cannot risk trying to process or modify packets that are in mid-transfer.

This is the exact thing that happened most recently to you. FIDOPOLL was running and MUTIL was trying to create busy rights to every node it could. What exactly went wrong and where is unknown, because I missed a few places
of logging where BUSY files were created or removed, so I can't see exactly where it failed.

Hopefully this most recent update will have enough logging that I can clearly pinpoint the issue.

--- Mystic BBS v1.12 A36 2017/11/29 (Windows/64)
* Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)

Is everything being called from Mystic events? From what I've seen here, if there is an event running, such as fidopoll, Mystic would wait until that is completed before attempting to call the 'mailin' process.

I am glad to hear that it is working properly! I'm pretty happy with the
MIS2 event system so far!

Yep, that is the idea behind the way that MIS2 queues events. If you are running off semaphores, there really shouldn't be a time when you have MUTIL and FIDOPOLL running together.

You cannot prevent the BINKP server though from doing things, because your downlinks can connect to you whenever they feel like it, so at a minimum in that case the BUSY rights system has to come into play to make sure MUTIL isn't packing mail for a node that is polling for mail at the same time.

--- Mystic BBS v1.12 A36 2017/11/29 (Windows/64)
* Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)

On 11/29/17, g00r00 said the following...

I am glad to hear that it is working properly! I'm pretty happy with the MIS2 event system so far!

I am too! :) The only complaint I had with the events in MIS, is if something was happening at the time another event was to start, it would skip the timed event. Now, it's just queued up and runs when everything has completed!

I also love that Mystic will detect the semaphores so quickly now. I run
Mystic in server mode, and like to watch the connections... It's only taking Mystic about 3 seconds or so, to detect the echomail.in semaphore, and have
the incoming mail tossed.

As I've stated before, you are doing a fantastic job with Mystic. I'm also excited to see some of ideas you have come to life in the future. Keep up the good work. (but feel free to take some time off for yourself as well. You deserve it!)


---

Black Panther
a.k.a. Dan Richter
Sysop - Castle Rock BBS (RCS)
The sparrows are flying again....

--- Mystic BBS v1.12 A36 2017/11/27 (Linux/64)
* Origin: Castle Rock BBS - castlerockbbs.com (21:1/186)

On 11/29/17, g00r00 pondered and said...

The short answer is that FIDOPOLL was doing stuff and MUTIL was trying
to get busy rights to every node. That is normal and the reason BUSY rights exist, but its not clear where it all went wrong.

Great reply thanks :) I will digest the longer version tomorrow when more awake. Sorry have run out of time to update 1/100 tonight but will do so tomorrow night after work. I'm sure we'll be able to crack this bug in time. Good to know the data gathered thus far helps :)

Best, Paul

--- Mystic BBS v1.12 A36 2017/11/26 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

I am too! :) The only complaint I had with the events in MIS, is if something was happening at the time another event was to start, it would skip the timed event. Now, it's just queued up and runs when everything has completed!

Yep that is another really good thing. It recalculates the event state for all events every 5 seconds when in the past it was about once per minute so you'd have to wait up to a minute for a semaphore event to kick off after it was found to exist.

As I've stated before, you are doing a fantastic job with Mystic. I'm
also excited to see some of ideas you have come to life in the future. Keep up the good work. (but feel free to take some time off for yourself as well. You deserve it!)

Thanks! I do have so many ideas I never seem to run out!

--- Mystic BBS v1.12 A36 2017/11/29 (Windows/64)
* Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)

On 11/30/17, Avon pondered and said...

awake. Sorry have run out of time to update 1/100 tonight but will do so tomorrow night after work. I'm sure we'll be able to crack this bug in time. Good to know the data gathered thus far helps :)

I woke up early and downloaded and applied the latest pre-alpha available. I found there was an issue with MUTIL in that despite being called with a different .ini name other than mutil.ini ... MUTIL refused to run and just errored with this

[snip]

----------------- MUTIL v1.12 A36 2017/11/29 Fri, Dec 01 2017 (loglevel 2)
+ Dec 01 06:46:11 Startup using mutil.ini
+ Dec 01 06:46:11 Results: No processes configured in 4118624.67s
! Dec 01 06:46:11 Status: FATAL
+ Dec 01 06:46:11 Shutdown Runtime error (1)

----------------- MUTIL v1.12 A36 2017/11/29 Fri, Dec 01 2017 (loglevel 2)
+ Dec 01 06:46:30 Startup using mutil.ini
+ Dec 01 06:46:30 Results: No processes configured in 4118643.84s
! Dec 01 06:46:30 Status: FATAL
+ Dec 01 06:46:30 Shutdown Runtime error (1)

[snip]

So my fix was to keep the 'latest' files in place but restore to the previous version of MUTIL to get it working again

[snip]

----------------- MUTIL v1.12 A36 2017/11/27 Fri, Dec 01 2017 (loglevel 3)
+ Dec 01 06:48:06 Startup using mailin.ini
- Dec 01 06:48:06 EXEC ImportEchoMail
- Dec 01 06:48:06 EXEC FileToss
+ Dec 01 06:48:06 Process: Toss FDN/TIC Files
+ Dec 01 06:48:06 Scanning Hatches
+ Dec 01 06:48:06 Results: 0 import, 0 toss, 0 hatch, 0 bad in 0.00s
+ Dec 01 06:48:06 Process: Importing EchoMail
+ Dec 01 06:48:06 Waiting for BUSY nodes
- Dec 01 06:48:06 LINKS:
- Dec 01 06:48:06 Node 14 21:1/112@fsxnet
- Dec 01 06:48:06 Node 9 21:1/108@fsxnet

[snip]

+ Dec 01 06:49:53 Results: 1 echo, 0 net, 0 dupes, 98 tossed in 2.23s
+ Dec 01 06:49:53 Shutdown Normal (0)

[snip]

So as of my Friday morning there's something amiss with the MUTIL in the
Win/32 version :)

I'll aim to update again after work tonight as no doubt you might find something and have an updated pre-alpha to install by then :)

I guess the only downside with what I did is the logging for *.BSY issue may not yet be complete as I am using mixed build binaries.

Best, Paul

--- Mystic BBS v1.12 A36 2017/11/26 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

called with a different .ini name other than mutil.ini ... MUTIL refused to run and just errored with this

I have a few minutes to look at it right now let me see what I can figure out.

--- Mystic BBS v1.12 A36 2017/11/29 (Windows/64)
* Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)

On 11/30/17, g00r00 pondered and said...

called with a different .ini name other than mutil.ini ... MUTIL refu to run and just errored with this

I have a few minutes to look at it right now let me see what I can
figure out.

Thanks :) 7am Fri here... I'm about to wake teenagers and remind them they have to go to school... if I should not return remember me with fondness - ha!

Best, Paul

--- Mystic BBS v1.12 A36 2017/11/26 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

called with a different .ini name other than mutil.ini ... MUTIL refused to run and just errored with this

This should be fixed now. I am uploading the new one as we speak. Sorry abouty that.

--- Mystic BBS v1.12 A36 2017/11/29 (Windows/64)
* Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)

Thanks :) 7am Fri here... I'm about to wake teenagers and remind them
they have to go to school... if I should not return remember me with fondness - ha!

Hah good luck! That reminds me of me when I was young...

When I was a teenager I lived with my grandmother for a while. I was a pain in the ass to get up for school (probably because I was up late every night on BBSes or teaching myself programming)...

I actually gave her a water gun at one point and told her to just walk in
the room and shoot me with it in the mornings. She never did use it though because she thought I might get super angry if she did.

Anyway, just an idea that might help get them up. And then you can use said water gun to defend yourself from the attacking teenagers! ;)

--- Mystic BBS v1.12 A36 2017/11/29 (Windows/64)
* Origin: Sector 7 [Mystic BBS WHQ] (21:1/108)

On 11/30/17, g00r00 pondered and said...

This should be fixed now. I am uploading the new one as we speak. Sorry abouty that.

8.53am checking messages from work... thanks for the fix, will apply it
tonight :)

--- Mystic BBS v1.12 A36 2017/11/26 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

On 11/30/17, g00r00 pondered and said...

Anyway, just an idea that might help get them up. And then you can use said water gun to defend yourself from the attacking teenagers! ;)

Ha! Love it... I just hover at the door, poke the bear and then run!

--- Mystic BBS v1.12 A36 2017/11/26 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (21:1/101)

pain in the ass to get up for school (probably because I was up late
every night on BBSes or teaching myself programming)...

I actually gave her a water gun at one point and told her to just walk in the room and shoot me with it in the mornings. She never did use it though because she thought I might get super angry if she did.

Yep I different methods on my boys, grabbing the mattress and lifting it usually did the trick, sometimes not, they just laid on the floor sometimes.

Ruben Figueroa aka Zazz
Mystic Prison Board Sysop
telnet://pb.darktech.org:24
Web: www.rdfig.net

--- Mystic BBS v1.12 A36 2017/11/28 (Windows/32)
* Origin: Mystic Prison Board BBS*Mesquite Tx (21:1/134)