Base Monitor Performance issue

SQL Server performance monitoring and alerting

Moderators: eddie davis, priyasinha, Adam, chriskelly, Chris Lambrou, Chris Spencer

Base Monitor Performance issue

Postby JoeGT » Mon Jul 08, 2013 11:47 pm

I have been running SQL Monitor v3.4.1.4 in two very similar environments for without issues but in the past 4 days (post applying some Windows OS patches to the base monitor server in both environments), I am seeing issues with the performance of the :

RedGate.Response.Base.Service

Which are affecting the overall performance of the server hosting the bases monitor. In one of the environments the memory and CPU usage of the service has started to be what looks to be excessive. Frequently the process is using 90% plus of the CPU and the memory usage climbs to up to 30GB. A reboot of the host server or a restart of the service only results in a temporary reprieve and then the problem starts again.

In contrast in the other similar environment (with no issues) the CPU usage for the process will hover around the 2-10% mark and the memory usage is significantly less - around 2GB.

Below are indicative screenshots of both the bad and good base monitor processes (woth of which had been running for the same length of time when the screenshots were taken :


Image

Image


Can you please advise if this is something you have seen before and what steps I can take to attempt to remedy this issue ?

Cheers

Joe
JoeGT
 
Posts: 40
Joined: Wed Jun 13, 2012 3:37 am

Postby JoeGT » Tue Jul 09, 2013 11:25 pm

Just to add some more information to this one. As of today, without any intervention on my part, the CPU and memory consumption on the problem base monitor has lowered - now the process is consumin approx 8GB memory and running between 5 and 20% CPU

Perhaps not coincidentally this morning I also received a number of very delayed emails for alerts that were generated 36 hours ago within SQL Monitor. We have confirmed from our SMTP message transfer logs that the emails were only delivered to the message transfer agent this morning despite the details of the alerts showing that they were from 36 hours ago.

This is reflected in what I now see in the "Alerts" page in SQL Monitor - these alerts were not visible here until this morning.

So this suggests to me some form of problem within the SQL Monitor application.

Interested to get some feedback from some at RedGate soon as we place a form of reliance on seeing these emails alerts for specific SQL events/issues and not having them delivered in a timely fashion obviously does not allow us to meet our customers expectations.

Cheers

Joe
JoeGT
 
Posts: 40
Joined: Wed Jun 13, 2012 3:37 am

Postby JasonC » Wed Jul 10, 2013 10:14 am

Hi Joe. To help with diagnosis, can I ask 2 questions:
1) How much memory is installed in each machine?
2) Are any other intensive applications running on each machine (e.g. a SQL Server), or is SQL Monitor the primary application?
Thanks!
Jason Crease
Red Gate Software
JasonC
 
Posts: 84
Joined: Wed Aug 16, 2006 1:58 pm

Postby JoeGT » Wed Jul 10, 2013 10:40 pm

Hi Jason

Both of the base monitor servers I mentioned have the following specifications :

IBM X3550 M3 2.13Ghz 32GB RAM running Windows Server 2008 R2 Enterprise

They are "Jump" servers that are used by a number of our staff to grant them access to our production environments. The only tools that run on them aside from SQL Monitor are SSMS and a number of other bespoke internal very low impact applications.

They do not have SQL Server or any SQL instances hosted on them.

Cheers

Joe
JoeGT
 
Posts: 40
Joined: Wed Jun 13, 2012 3:37 am

Postby JasonC » Thu Jul 11, 2013 1:47 pm

Hi Joe. Thanks for the extra info.

As for the memory usage: this sometimes happens when SQL Monitor is doing something CPU intensive on a machine with lots of memory. When no other intensive processes are on the machine, the CLR sometimes grabs massives of memory for itself to amortize garbage-collections over very long periods, in order to free up the CPU. Generation 0 can become several gigabytes. This is just the CLR trying to be efficient by using all the resources available to it. Once the CPU is freer, it'll then hand the memory segments back to the OS.

As for the CPU: that is more worrying. What I believe is happening is that SQL Monitor was performing a large purge (i.e. deleting old monitoring data in your database). The purging algorithm is a bottleneck, since it has to sort and delete gigabytes of data. We'll investigate further.
Jason Crease
Red Gate Software
JasonC
 
Posts: 84
Joined: Wed Aug 16, 2006 1:58 pm


Return to SQL Monitor 3

Who is online

Users browsing this forum: No registered users and 0 guests