Random application errors... three years and counting

My original post about these errors was at the old support web-site before this site existed.   I mentioned them here in this forum back in this 2009 post...

http://community.sagemas.com/t5/Installation-and-Technical/MAS-500-Desktop-Auto-Shutdown/m-p/5178#M173

 

I'm posting to provide a status update.  The situation is that we never saw any similar errors when our Terminal Server was running Windows 2000.   It was not until we installed a Windows 2003R2 Terminal Server a little over three years ago that we began to see these errors.  I worked with Sage and our VAR trying to solve them for several months, but eventually we all basically just decided that we could not solve them and that we would just have to live with them.   In the back of my mind, though, I had been hopeful that perhaps some future upgrade would fix them.  Well, three years and several upgrades later, I just thought I would update you all... they haven't changed a bit.

From the time we started seeing them immediately after upgrading our Terminal Server from Windows 2000 to Windows 2003R2, we have since gone through the following upgrades:

 

7.05 to 7.20.x to 7.30.2 and finally to 7.30.5.

 

Our Dell PowerEdge 1900 Terminal Server was reformatted and rebuilt with Windows 2008 SP2 last summer.

 

And finally a week ago our Dell PowerEdge 2900 SQL Server was reformatted and rebuilt taking it from Win2003R2/SQL2005 up to Win2008R2/SQL2008R2.  Our system is now as current as it can possibly be.

 

None of these upgrades have had any effect on the errors.  With about 15-20 users on the system per weekday, we continue to average about one random application error per day on our Windows 2008 SP2 Terminal Server.

All server firmware and drivers are the latest available.

I would be interested to know if any of you see similar errors in your Application Event Logs on your PCs (workstations or Terminal Servers) running the MAS 500 client.

Here are a few examples of the types of errors I'm referring to.

 

Log Name:      Application
Source:        Application Error
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Description:
Faulting application Apzdj001.exe, version 7.30.0.0, time stamp 0x4ac57cc0, faulting module unknown, version 0.0.0.0, time stamp 0x00000000, exception code 0xc0000005, fault offset 0x850f57c0, process id 0x1450, application start time 0x01cc162063b76c3e.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2011-05-17T12:44:00.000Z" />
    <EventRecordID>34807</EventRecordID>
    <Channel>Application</Channel>
    <Security />
  </System>
  <EventData>
    <Data>Apzdj001.exe</Data>
    <Data>7.30.0.0</Data>
    <Data>4ac57cc0</Data>
    <Data>unknown</Data>
    <Data>0.0.0.0</Data>
    <Data>00000000</Data>
    <Data>c0000005</Data>
    <Data>850f57c0</Data>
    <Data>1450</Data>
    <Data>01cc162063b76c3e</Data>
  </EventData>
</Event>
******************************************************************************

Log Name:      Application
Source:        Application Error
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Description:
Faulting application sozdc001.exe, version 7.30.0.0, time stamp 0x4ac57b59, faulting module MSVBVM60.DLL, version 6.0.98.2, time stamp 0x4791a724, exception code 0xc0000005, fault offset 0x0000a20e, process id 0x11ec, application start time 0x01cc164d77520b49.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2011-05-18T17:52:22.000Z" />
    <EventRecordID>34882</EventRecordID>
    <Channel>Application</Channel>
    <Security />
  </System>
  <EventData>
    <Data>sozdc001.exe</Data>
    <Data>7.30.0.0</Data>
    <Data>4ac57b59</Data>
    <Data>MSVBVM60.DLL</Data>
    <Data>6.0.98.2</Data>
    <Data>4791a724</Data>
    <Data>c0000005</Data>
    <Data>0000a20e</Data>
    <Data>11ec</Data>
    <Data>01cc164d77520b49</Data>
  </EventData>
</Event>

******************************************************************************

Log Name:      Application
Source:        Application Error
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Description:
Faulting application poztl001.exe, version 7.30.0.0, time stamp 0x4ac57ad3, faulting module unknown, version 0.0.0.0, time stamp 0x00000000, exception code 0xc0000005, fault offset 0x007278fe, process id 0x29b0, application start time 0x01cc165d56a1d071.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2011-05-19T19:54:47.000Z" />
    <EventRecordID>34899</EventRecordID>
    <Channel>Application</Channel>
    <Security />
  </System>
  <EventData>
    <Data>poztl001.exe</Data>
    <Data>7.30.0.0</Data>
    <Data>4ac57ad3</Data>
    <Data>unknown</Data>
    <Data>0.0.0.0</Data>
    <Data>00000000</Data>
    <Data>c0000005</Data>
    <Data>007278fe</Data>
    <Data>29b0</Data>
    <Data>01cc165d56a1d071</Data>
  </EventData>
</Event>

 

 

 

 

 

  • We get them also.  At least in some of our cases they are directly related to customizer issues.  I also believe some number of them are associated with unhandled errors in the MAS 500 code (either from Sage or one of our VARs).  A few others are likely caused by system slowdowns (ie timeout errors).

     

    John

  • Thanks for the comment, John.  I don't believe ours are related to customizer, since we see them in modules that have no customizer changes whatsoever.  Probably not timeout errors either.  "Unhandled errors in the MAS 500 code from Sage" seems like a possibility though.

  • Thanks, Robert. As I've stated before, I've rebuilt the servers more than once in trying to solve this (and have even tried a different physical server for the Terminal Server), so if it's a corrupt registry, the corruption would have to be being caused by something early on in the build process (like drivers or MAS), since I've built it and temporarily run with absolutely NOTHING on it except Windows, drivers, and MAS, and we still saw the errors.

    It's not malware.

    It's not faulty RAM.

    It's unlikely that it's faulty drivers, since both servers have been rebuilt with newer versions of Windows and drivers (most drivers have actually been updated several times).

    It's unlikely that it's Windows security features, since at one time I had some users running as local admins on the Terminal Server as a test, and we still saw the errors.

    I also doubt that it's incorrectly installed or updated software.

     

  • Unfortunately, it's not a reproducible scenario.  I understand that no computer errors are truly random, but these errors appear to be random.  We see them in modules that have a few customizations, but we also see them in modules that have no customizations at all.   Usually the tasks that the users are in when the errors occur are tasks with no customizations at all.  I am aware of a few other customers who see these types of errors.

    Probably no sense in opening a support case at this point.  Neither our VAR nor Sage Support were able to solve the errors after several months of trying back in '08.

  • We also get these errors running in a Citrix environment. I have to agree with John's assessment that some of this is likely due to unhandled exceptions in source code. More specifically, exceptions that occur and are unrecoverable from within a terminal services or Citrix type environment. Some of these ARE very likely due to memory or processor issues.

     

    Given that we have a very heavy user load (roughly 250 - 300 on any given day), I may be able to lay out a scenario in which "some" of these events occur. Occasionally someone will run an application within Mas that generates a block transaction. If the block isn't cleared right away, from a "citrix" point of view, the user has the option to either "disconnect" a Mas session or "log off" a session. If they choose to "disconnect", it will leave their transactions running in the background and give them the appearance that they have closed out of Mas 500. When the user tries to log back in, Citrix will try and recover the applications that user left up and running and there are MANY times, these windows will not recover correctly even though the application they left open STILL has a process running in the background. The user has no way to recover the open window at that point and I believe is responsible for some of these errors. 

     

    Many times, our users will get locked up due to to a transaction that is being run by another user and will attempt to terminate their connection to their host Citrix server and it will not log off as expected. We usually have to log off these users as in the Citrix Management Console and this will terminate "running" background applications abnormally also. I believe that this again will generate the errors you are seeing.

     

    Just my 2 cents...