Re: Posting a message with messagebase open
By: Nightfox to Digital Man on Sat Jul 11 2015 02:17 pm

While using my message reader (written in JavaScript), I've noticed that if it has a sub-board open with a MessageBase object and lets the user post a message in that sub-board, in Linux, the MessageBase object doesn't seem to get updated with the number of messages in the sub-board. In

Yeah.. I've noticed this as well..
Along with, and it's not at all consistent, sometimes when posting a message it just crashes and hangs up, without ever posting the message, and I don't think this is a SlyEdit issue either as it happends just after the save occurs and only then. Also, Synchronet 3.16 on Linux from CVS builds.

)))[Psi-Jack -//- Decker]

... Mistrust first impulses, they are always good.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Nightfox to Psi-Jack on Sat Jul 11 2015 08:37 pm

Re: Posting a message with messagebase open
By: Psi-Jack to Nightfox on Sat Jul 11 2015 21:19:17

While using my message reader (written in JavaScript), I've noticed
that if it has a sub-board open with a MessageBase object and lets
the user post a message in that sub-board, in Linux, the
MessageBase object doesn't seem to get updated with the number of
messages in the sub-board. In

Yeah.. I've noticed this as well..
Along with, and it's not at all consistent, sometimes when posting a
message it just crashes and hangs up, without ever posting the
message, and I don't think this is a SlyEdit issue either as it
happends just after the save occurs and only then.

I've noticed that sometimes too, in Windows. It seems to happen randomly, so I don't know how to reproduce it. My guess would be that if you happen to be posting a message at the same time that the sub-board messagebase is being updated (via a QWK update), maybe there's some resource conflict that causes a crash. I haven't verified that theory though. I made a local update to my message reader to close its MessageBase object while posting a message and re-open the MessageBase after the posting is done - That seems to allow the total number of messages to be updated (in Linux), and hopefully it will make it more stable too. I'll release an update to my reader after more testing.

Cool.. I will try it out when you do then. :)

I'm currently testing an idea of actually running Synchronet on CentOS 7, and so far, after grinding thoroughly through building custom RPMs for DOSEMU, BinkD, and Husky Tools, the main software needed for my full setup, I am actually coming up with astonishing results.

I even went above and beyond and made a DOS door maintenance script that would parallel run through each door that was configured in the script to allow running 2+ dosemu sessions at the same time one for each door, and tracking of each one.

Unfortunately the one limitation was: I ended up using 4DOS 8.00 (Freeware), to get proper logging done. I could never get FreeDOS's command.com to allow me to capture the date and time into a variable to output to a logfile, but this allows me now to track that each door actually ran to completion, and not aborted out in a DOSEMU crash. :D

)))[Psi-Jack -//- Decker]

... I reserve my abuse for lower life forms, like Civil Servants.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Nightfox to Psi-Jack on Sun Jul 12 2015 01:50 pm

I'm currently testing an idea of actually running Synchronet on
CentOS 7, and so far, after grinding thoroughly through building
custom RPMs for DOSEMU, BinkD, and Husky Tools, the main software
needed for my full setup, I am actually coming up with astonishing
results.

That's cool. CentOS is definitely a fairly good & stable server OS (since it's based on RedHat). I'm currently running Mint Linux on my BBS machine (although my BBS is running in a Windows XP VM), but I've considered possibly switching my BBS over to Linux at some point. Mint Linux seems to work well for Synchronet so far, but I had considered CentOS too.

Heh yeah. I've used Debian & Ubuntu, but DOSEMU on both of those would go
wonky after a few days. Still don't know actual root cause, but just oddities that were major abnormalities. My CentOS 7 system I'm testing out now, and looking to migrate my BBS to, since it's so easy to move Synchronet around when needed. Mostly I'm just trying to get away from Arch/Manjaro fully because I don't want to maintain my in-house repo for them anymore, and that becomes a sticky situation when it continues to be updated all the time.

I even went above and beyond and made a DOS door maintenance script
that would parallel run through each door that was configured in the
script to allow running 2+ dosemu sessions at the same time one for
each door, and tracking of each one.

Unfortunately the one limitation was: I ended up using 4DOS 8.00
(Freeware), to get proper logging done. I could never get FreeDOS's
command.com to allow me to capture the date and time into a variable
to output to a logfile, but this allows me now to track that each
door actually ran to completion, and not aborted out in a DOSEMU
crash. :D

I'm not sure I'd consider 4DOS a limitation, since (I believe) it's fully compatible with command.com. It's cool to hear that 4DOS is now freeware. I used to use 4DOS in the early 90s, when DOS was my primary OS, and I thought it provided some useful enhancements over the command.com included with MS-DOS.

I agree, but also think a limitation only because you /HAVE/ to go download 4DOS, install it, and move it to the DOSEMU Z drive afterwards, in order to use my system. But yeah, otherwise, 4DOS is awesome. Now freeware, makes it so much more awesome. The simple fact that %_DATE and %_TIME existed in 4DOS but not FreeDOS's command.com just makes me go hmmmm... FOR in FreeDOS doesn't even have /F, or any of the switches MS-DOS's FOR had. :/ The only way, with pure MS-DOS, to extract the date and time into usable variables was to get a third party DOS program, from GNU, sed, and awk, just to export the date to a file and read it back in then delete the file. Horrible, horrible, HORRIBLE. ;)

I tried CentOS before, since I'm a big user of it, and prefer RPM over any other package format. But, with no tools and little current knowledge at the time of what I'd need in order to run my BBS sanely and fully.. I opted to use something that already had it all. Now that I'm well enough beyond that, I was able to build all the RPMs myself, so now, CentOS 7 definitely has potential.

And as you say, it's rock solid as a server. I wanted to get it on CentOS because most of my other VM's are already CentOS, and I wanted to be able to manage it with Chef like I do everything else. And because Manjaro and Arch both, no matter than acpid was running, they would not shutdown when kvm sent the ACPI signal to shut down, so many times they got forced offline uncleanly.

I think by the end of this coming week, I'll have my BBS moved over to CentOS 7 fully, and nearly transparently, and sadly even better setup than before, since I'm taking the time to make things work simply better.

)))[Psi-Jack -//- Decker]

... I like long walks, especially when they are taken by people who annoy me.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Digital Man to Nightfox on Mon Jul 13 2015 03:35 pm

Is the file system in use on Linux a shared file system (SMB/CIFS or NFS) or just a local file system?

I can answer that one. Local. :)

)))[Psi-Jack -//- Decker]

... Superior ability breeds superior ambition. Spock, stardate 3141.9.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Digital Man to Psi-Jack on Mon Jul 13 2015 04:55 pm

Is the file system in use on Linux a shared file system (SMB/CIFS
or NFS) or just a local file system?

I can answer that one. Local. :)

I suspect there may need to be a buffer flush (or something comparable) added somewhere. A small test case (e.g. that could just be run with jsexec) would be helpful in isolating and resolving the problem.

I'd be happy to provide anything you need. I know I see the problem myself, and in my personal /specific/ case, if I post on the last message base of the forum I'm looking at, it doesn't refresh until I move to a previous message then back again.

My bigger problem is random BBS drops when navigating through messages, or finishing a post. That's my bigger problem. Not exactly the same issue Nightfox is trying to track down, but a more serious issue that I think is more Synchronet-specific, but I cannot verify, I just know Synchronet sees a disconnect, and nothing else, no error, nothing.

)))[Psi-Jack -//- Decker]

... Martyrdom is the only way a person can become famous without ability.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Digital Man to Nightfox on Mon Jul 13 2015 08:07 pm

Is the file system in use on Linux a shared file system (SMB/CIFS
or NFS) or just a local file system?

It's a local file system.

I just committed an additional file stream flush to the SMB "get status" function (in src/smblib). Please try this and if that doesn't fix it, provide more details or better yet, a small script to reproduce the problem (e.g. using jsexec).

I'll try out this, after I check a few things, then try again if I find any results.

What /I/ personally notice is that I randomly get dropped when posting, usually during prime time hours, when mail is being tossed the most. I wish I had a stats generator that could produce the flow of mail, where, and everything, but all I can do currently is with binkdstats.pl parsing binkd logs. I will fix this, and make more log parsers for synchronet specifically, heh.

One thing I'm testing now is whether I start a reply, and DOVE net processes, if I get dropped when I save, after it's done, and what happened during that process. New in, new out.. So far, new out alone doesn't /appear/ to be causing issue, but inbound..

So if you see this reply, that means I didn't crash after posting after a QWKnet packet.
What did happen during that however was:
evnt Finished Importing QWK Network Packet from VERT: (2 msgs) in 1 seconds (2 msgs/sec)

So, here goes.

)))[Psi-Jack -//- Decker]

... Virtue is a relative term. Spock, Friday's Child, stardate 3499.1.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Psi-Jack to Digital Man on Tue Jul 14 2015 07:30 pm

So if you see this reply, that means I didn't crash after posting after a QWKnet packet.
What did happen during that however was:
evnt Finished Importing QWK Network Packet from VERT: (2 msgs) in 1 seconds (2 msgs/sec)

Well, that went through without dropping me.

Next test, inbound FTN messages, as per from sbbsecho.

Results:
Imported: 3 msgs in 0.0 sec (5806.5/min 96.8/sec)

Let's see....

)))[Psi-Jack -//- Decker]

... Whatever it is, it won't work.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Psi-Jack to Digital Man on Tue Jul 14 2015 07:36 pm

Well, that went through without dropping me.

Next test, inbound FTN messages, as per from sbbsecho.

Results:
Imported: 3 msgs in 0.0 sec (5806.5/min 96.8/sec)

Let's see....

Well, that obviously worked.

However, as I was finishing my mail run, and the new scan completed, right at the very same time outbound FTN NetMail was processing and I was dropped from the connection.

I am using primarily Hyper Allocation for my message bases. Though I've considered switching to Fast or Self-Packing, not knowing really what pitfalls each method has..

)))[Psi-Jack -//- Decker]

... That unit is a woman. A mass of conflicting impulses. Spock and Nomad

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Digital Man to Psi-Jack on Wed Jul 15 2015 12:08 am

I am using primarily Hyper Allocation for my message bases. Though
I've considered switching to Fast or Self-Packing, not knowing really
what pitfalls each method has..

The message base allocation method shouldn't make any difference with the problem you're describing.

Since no one else is reporting the problem you're seeing, I suggest trying to "undo" some of your mods. Or just as a test, don't use of the 3rd party mods (e.g. message reader) you might be using.

I plan to do exactly that as well, disable a few things and see. May do that after I migrate to CentOS 7, maybe before. :)

)))[Psi-Jack -//- Decker]

... But, He has not one redeeming vice.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Nightfox to Digital Man on Wed Jul 15 2015 07:29 am

The message base allocation method shouldn't make any difference
with the problem you're describing.

Since no one else is reporting the problem you're seeing, I suggest
trying to "undo" some of your mods. Or just as a test, don't use of
the 3rd party mods (e.g. message reader) you might be using.

I've actually seen what I think he is describing. It seems to happen when I reply to a message using my reader. It happens rarely and randomly, so it's difficult to reproduce. I've noticed it in Windows (I think Psi-Jack is running Synchronet in Linux?).

The other day I made a small JavaScript script that opens a sub-board with a MessageBase object, lets a user post a message while that MessageBase object is open, then closes the MessageBase object. I was trying to reproduce the issue in Linux where if a JS script has a MessageBase open and lets the user post a message in that sub-board, the MessageBase wasn't getting updated as far as the number of messages. I ran that small script on my Linux test setup, and while that script didn't reproduce that issue, I did see it crash Synchronet multiple times after I had saved my message. I tried it in my Windows Synchronet setup, and I didn't see it causing a crash there. This is the code (substitute the value of the subCode variable with a valid sub-board code on your BBS):

load("sbbsdefs.js");

subCode = "loc_general";
var msgbase = new MsgBase(subCode);
if (msgbase.open())
{
console.print("\1n\r\n# of messages (before posting): " + msgbase.total_msgs
+ "\r\n\1p");
var msgHdr = msgbase.get_msg_header(true, msgbase.total_msgs-1, true);
if (msgHdr != null)
{
bbs.post_msg(subCode, WM_NONE, msgHdr);
console.print("\1n\r\n# of messages (after posting): " + msgbase.total_msgs
+ "\r\n\1p");
}
msgbase.close();
}

nice! I will take this code and test to see if I can get it to re-produce it at a faster rate, like setting it up in a loop. :D

I can just create a local test board only I have access to and see what the results end up being.

)))[Psi-Jack -//- Decker]

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Re: Posting a message with messagebase open
By: Khelair to Nightfox on Wed Jul 15 2015 11:02 am

I'm not sure I'd consider 4DOS a limitation, since (I believe) it's
fully compatible with command.com. It's cool to hear that 4DOS is
now freeware. I used to use 4DOS in the early 90s, when DOS was my
primary OS, and I thought it provided some useful enhancements over
the command.com included with MS-DOS.

I loved 4dos. Such awesome enhancements for command.com's pointlessly lobotomized batch features.

I will update that to say, I /love/ 4DOS, and I'm glad it's freeware now and in my Z:\4DOS DOSEMU drive. Totally dependant on it now with my monster setup, making maintenance much better.

)))[Psi-Jack -//- Decker]

... You can tune a piano, but you can`t tuna fish.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Digital Man to Nightfox on Wed Jul 15 2015 09:39 pm

Re: Posting a message with messagebase open
By: Digital Man to Nightfox on Wed Jul 15 2015 16:46:12

I just committed an additional file stream flush to the SMB
"get status" function (in src/smblib). Please try this and if
that doesn't fix it, provide more details or better yet, a
small script to reproduce the problem (e.g. using jsexec).

I hadn't actually commited the flush call yet. I just did. Please
try again.

I updated my Synchronet build on my Linux test system, and that seems
to have fixed the issue I was seeing.

Okay, good. There will probably be other flush-related commits to smblib since this highlights an issue with stale read-data that I was not aware of.

This does not explain any crashes or unintended program exits you were seeing.

I too confirm the issue Nightfox reported is resolved. I just finished migration from Manjaro to CentOS 7, and doing a lot of tests. So far, replied to a lot of forums, without a single hitch since the migration.. Kind of ... Well, very nice.. I'll continue to test and see what happends, along with using Nightfox's earlier script he posted to do more direct tests locally. I can easily hammer it down on a test board to see what causes it, if anything anymore.

But, yes, thank you for fixing this issue as well. I noticed it, but thought nothing of it. It would load, just had to jump back a message, then back forwards before it would show up. Now it's instant as it should be.

I just wonder if the same issue MAY had been causing the disconnects as well. As you say, it doesn't explain it, but if it resolves it at the same time...

)))[Psi-Jack -//- Decker]

... But soft, what light through yonder tagline breaks?

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Digital Man to Psi-Jack on Thu Jul 16 2015 02:39 am

I just wonder if the same issue MAY had been causing the disconnects
as well. As you say, it doesn't explain it, but if it resolves it at
the same time...

I can't think of any logical connection between the two things.

Yeah, definitely isn't. A message or two after this reply I made, I was dropped just while putting in the To, then Subject, then boom, disconnect.

So, Not sure. Definitely still happening, with current CVS codebase.

)))[Psi-Jack -//- Decker]

... Culture is what your butcher would have if he were a surgeon.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Digital Man to Psi-Jack on Thu Jul 16 2015 07:06 pm

I just wonder if the same issue MAY had been causing the
disconnects as well. As you say, it doesn't explain it, but if it
resolves it at the same time...

I can't think of any logical connection between the two things.

Yeah, definitely isn't. A message or two after this reply I made, I
was dropped just while putting in the To, then Subject, then boom, disconnect.
So, Not sure. Definitely still happening, with current CVS codebase.

Does it happen when you don't use the custom message reader mod?

If it's a JS exception causing the termination, there should (normally) be some indication in the log output. I suppose it's possible that the script could handle the exception (whatever it is) and just terminate silently, but that's not normal.

I haven't yet tried, but I'm going to be making my development shell utilize the internal message systems itself, but also I'm trying something else out as well before I finish that.

The problem may actually be SyncTERM itself dropping the connection.. A potential situation I pondered.

The way I'm currently testing this theory is sadly by using NetRunner. I hate this particular terminal app because it's so slow and clunky, and doesn't even support rlogin or ssh like I usually use. heh. And it's slow, even over LAN.

So far, though, using NetRunner, I have yet to see a single disconnect, but I'm going to continue using it for the rest of this week to see.

)))[Psi-Jack -//- Decker]

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Digital Man to Psi-Jack on Fri Jul 17 2015 01:51 pm

The problem may actually be SyncTERM itself dropping the connection..
A potential situation I pondered.

The way I'm currently testing this theory is sadly by using NetRunner.
I hate this particular terminal app because it's so slow and clunky,
and doesn't even support rlogin or ssh like I usually use. heh. And
it's slow, even over LAN.

So far, though, using NetRunner, I have yet to see a single
disconnect, but I'm going to continue using it for the rest of this
week to see.

And you could try different protocols in SyncTERM (e.g. Telnet instead of SSH).

So far, so good with SyncTERM 1.0b 2015-07-15. Not 1 single hangup, yet.. That's using rlogin. I haven't yet changed protocols, because I'm trying to check same-condition different versions, to validate.

)))[Psi-Jack -//- Decker]

... Conscience gets alot of credit that belongs to cold feet.

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Nightfox to Psi-Jack on Fri Jul 17 2015 02:47 pm

So far, though, using NetRunner, I have yet to see a single
disconnect, but I'm going to continue using it for the rest of this
week to see.

And you could try different protocols in SyncTERM (e.g. Telnet instead
of SSH).

That reminds me, several years ago I noticed that when I was connected to my BBS via SSH (using SyncTerm), I would get disconnected from my BBS after some amount of time. It didn't seem to be related to posting messages or anything in particular. But I noticed that seemed to be fixed in more recent builds of Synchronet/SyncTerm.

Well, I seem to be getting less random disconnects with just simply a newer version of SyncTERM. With cryptlib actually supplied in SyncTERM with this method, also finally /allows/ me to use ssh, too. :)

)))[Psi-Jack -//- Decker]

... The English may not like music, but they absolutely love the noise it makes

---
þ Synchronet þ Decker's Heaven -//- bbs.deckersheaven.com

Re: Posting a message with messagebase open
By: Psi-Jack to Nightfox on Sat Jul 11 2015 21:19:17

This is another message from you that has somehow migrated to another board - I saw this in the Sports board on Dove-Net. This was originally in the Synchronet JavaScript board.

Nightfox

---
þ Synchronet þ Digital Distortion: digitaldistortionbbs.com

Re: Posting a message with messagebase open
By: Nightfox to Psi-Jack on Sun Jul 19 2015 11:53 am

Re: Posting a message with messagebase open
By: Psi-Jack to Nightfox on Sat Jul 11 2015 21:19:17

This is another message from you that has somehow migrated to another board - I saw this in the Sports board on Dove-Net. This was originally in the Synchronet JavaScript board.

And he's dumping dupes, apparently from DECKHVN2 to DECKHVNT.

digital man

Synchronet "Real Fact" #81:
Flapuebarg unf vagreany ebg13 fhccbeg sbe fhcresvpvnyyl rapelcgvat grkg.
Norco, CA WX: 69.9øF, 94.0% humidity, 2 mph NW wind, 0.86 inches rain/24hrs

---
þ Synchronet þ Vertrauen þ Home of Synchronet þ [vert/cvs/bbs].synchro.net