Bugs in 2.3.1

Is Virtual Radar Server not behaving itself? If so then please report it here.
dsfh2992
Posts: 156
Joined: Tue Oct 06, 2015 9:57 pm

Re: Bugs in 2.3.1

Post by dsfh2992 » Tue Jan 26, 2016 4:29 pm

At first it appeared that the JSON feeds (with 1 second updates) were working much better, but this morning, the queue is pegged at 200,000. My three feeder feeds' "Total Messages" are incrementing at a rate of about 500-1000/sec (not being queued), but the merged feed "Total messages" is only incrementing by 5-10 messages per second. Not sure what's up with that....

BTW, if you need a boatload of data to test with, I am happy to send you a feed.

dsfh2992
Posts: 156
Joined: Tue Oct 06, 2015 9:57 pm

Re: Bugs in 2.3.1

Post by dsfh2992 » Tue Jan 26, 2016 4:33 pm

FYI, closing and re-launching VRS cleared that problem, and things went back to normal. Largest queue high-water mark is now 3,800 or so. The level of traffic is exactly the same as it was (a bit higher now actually) a few minutes prior when everything was gummed up prior to the close and re-launch.

Not sure why that would be, but just FYI...

dsfh2992
Posts: 156
Joined: Tue Oct 06, 2015 9:57 pm

Re: Bugs in 2.3.1

Post by dsfh2992 » Tue Jan 26, 2016 6:15 pm

Data point - Just ground to a halt again, same symptoms, queue nailed up to 200,000, only updating maybe 5-10 messages per second on the merged feed.

Quitting and restarting VRS got things back to normal.

Checked the log file. Absolutely no entries except the shutdown and restart of VRS.

Not complaining, just a data point for troubleshooting.

Thanks,
Dan

dsfh2992
Posts: 156
Joined: Tue Oct 06, 2015 9:57 pm

Re: Bugs in 2.3.1

Post by dsfh2992 » Tue Jan 26, 2016 7:04 pm

It seems like it processes very few messages for a couple seconds, then a big burst goes through, it's slow again for a few seconds, another big burst, etc....

It seems as if eventually something happens where it has more slow periods than fast periods and it just doesn't ever catch up....

dsfh2992
Posts: 156
Joined: Tue Oct 06, 2015 9:57 pm

Re: Bugs in 2.3.1

Post by dsfh2992 » Tue Jan 26, 2016 7:31 pm

Increasing the time between JSON messages from the feeder instances from 1 sec to 5 sec did not seem to make a difference either. It takes the queue longer to get to 200,000, but it still gets there.

It seems like it may also have something to do with how many viewers are on the site as well. Increasing the minimum refresh rate for clients from 1 second to 3 seconds seemed to help quite a lot, but even at 1 second, the CPU utilization never goes above 50%.

agw
Posts: 2241
Joined: Fri Feb 17, 2012 3:20 am

Re: Bugs in 2.3.1

Post by agw » Wed Jan 27, 2016 10:01 pm

I've just removed some of the more egregious locks from the aircraft list which I'm hoping will lead to less blocking on the receivers from the merged feed message processing thread. I'm running it here for a bit before I do a preview installer, but if you want to give it a try then let me know and I can upload a copy for you.

If you could let me have a feed then that would be handy, I can merge that and the ADSBHub feed together and see what VRS does with it :) You can either PM or email me the details if you're still up for it.

dsfh2992
Posts: 156
Joined: Tue Oct 06, 2015 9:57 pm

Re: Bugs in 2.3.1

Post by dsfh2992 » Wed Jan 27, 2016 10:42 pm

Thanks, PM sent.

dsfh2992
Posts: 156
Joined: Tue Oct 06, 2015 9:57 pm

Re: Bugs in 2.3.1

Post by dsfh2992 » Thu Feb 04, 2016 6:16 am

Any ideas what these mean?

http://www.evernote.com/l/ALx-fe775rJMr ... np7hgE_MQ/

These appeared on servers with a large number of client feeds. I might be able to simply break them down by running multiple VRS instances on these machines, but if there's a cleaner way, great.

Thanks,
Dan

agw
Posts: 2241
Joined: Fri Feb 17, 2012 3:20 am

Re: Bugs in 2.3.1

Post by agw » Sat Feb 06, 2016 4:01 pm

It's saying that it ran out of memory when it built the JSON to send back to the browser. Was this using the release version of 2.3.1 or the current preview?

dsfh2992
Posts: 156
Joined: Tue Oct 06, 2015 9:57 pm

Re: Bugs in 2.3.1

Post by dsfh2992 » Sat Feb 06, 2016 8:30 pm

Current preview version.

would it be possible to have a switch that simply ends the VRS process when these errors come up? i have a routine that will restart it.... the error could be logged and VRS quits.

Locked