Bugs in 2.1.0

[eshabi]

1. Download and run Process Explorer from SysInternals (https://technet.microsoft.com/en-gb/sys ... 96653.aspx)
2. Start Virtual Radar Server and wait until its holding on to say 2,000 handles
3. Right-click VirtualRadar.exe in Process Explorer and choose Create Dump | Create Full Dump
4. The .DMP file that it's created will be very large. You might want to compress it to ease the pain of uploading it.
5. Upload the dump file to one of the file sharing sites (DropBox, OneDrive, Google Drive etc.) and PM or email me the URL for it.

I'll then download the dump and take a look to see if I can figure out what's going on.

Just sent you a PM with a link to dropbox for downloading the dump file.

[franzke]

Not sure if this a bug of just a coincidence...
Yesterday i saw a flight that landed about 10-15 km away and first i saw a height of 000, then GND or something like that which meant landed i think. a bit after that it went to 3000+ feet, exactly the same height as the other plane of the same company that was in the air. then it disappeared because it was out of sight or maybe disabled the transponder. It's difficult to reproduce because normally no flight lands there (ANR) with a ADS-B transponder

I've also seen the same thing occasionally with aircraft landing at Heathrow. They'll land just fine and then a few minutes later I get one message that shows them at 20,000 feet, and then nothing. I managed to capture one instance on a recording of an SBS-3 - I was recording both port 30003 and port 30006 from BaseStation, an aircraft landed at Heathrow and then a couple of minutes later a message came out of port 30006 that showed it at 34,000' but there was no corresponding message on port 30003. I presumed that BaseStation's sanity checker had recognised that the aircraft could not have moved from 0' to 30,000' in a couple of minutes and dropped the message, but allowed it through on port 30006 because it passed the parity check.

VRS does have a sanity checker for things like that, but it's not very good when more than about 30 seconds have elapsed between messages. I think basically I need to write a better sanity checker.

What i meant with it that this 'flight' seemed to have copied the data of another flight of the same operator since it had the exact same height... That would be a bug and not a bad reception or something like that in my eyes. I had sorted the flights by flag at that moment. I've seen other weird stuff but that was all due to errors in reception, this one lust looked too strange to catalogue as bad data
I have a flightfeeder from flightaware btw...

[ssfer]

I'm having this issue and it occurs randomly.



VirtualRadar

Code: Select all

Version: 2.1.0.42181
Plugins: DatabaseWriter
Receivers:
               Name: Allwiner
               Format: AVR or Beast Raw (beast is the receiver's mode)
               Connection: Network (no push, send keep-alive packets)
Rebroadcasters:
                Name: Basestation Server
                Receiver: Allwiner
                Format: BaseStation
                No push
                Port: 30003
                Unrestricted
--
               Name: ModeBeast
               Receiver: Allwiner
               Format: Pass-through
               Port: 30005
               No push
               Unrestricted
--
               Name: Virtual Radar
               Receiver: Allwiner
               Format: Compressed VRS
               Port: 33001
               No push
               Password protected
               Unrestricted

 - Piaware+faup1090 is feeded by ModeBeast rebroadcaster;
 - fr24feed_x64 is feeded by Basestation Server rebroadcaster;
 - Two remote VRS are feeded by Virtual Radar rebroadcaster.

SO

Code: Select all

3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Mono version

Code: Select all

Mono JIT compiler version 3.2.8 (Debian 3.2.8+dfsg-4ubuntu1)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

I'm using Ubuntu headless server. For X11 support, I use XVfb.

[obj]

Not sure if this a bug of just a coincidence...
Yesterday i saw a flight that landed about 10-15 km away and first i saw a height of 000, then GND or something like that which meant landed i think. a bit after that it went to 3000+ feet, exactly the same height as the other plane of the same company that was in the air. then it disappeared because it was out of sight or maybe disabled the transponder. It's difficult to reproduce because normally no flight lands there (ANR) with a ADS-B transponder

I've also seen the same thing occasionally with aircraft landing at Heathrow. They'll land just fine and then a few minutes later I get one message that shows them at 20,000 feet, and then nothing. I managed to capture one instance on a recording of an SBS-3 - I was recording both port 30003 and port 30006 from BaseStation, an aircraft landed at Heathrow and then a couple of minutes later a message came out of port 30006 that showed it at 34,000' but there was no corresponding message on port 30003. I presumed that BaseStation's sanity checker had recognised that the aircraft could not have moved from 0' to 30,000' in a couple of minutes and dropped the message, but allowed it through on port 30006 because it passed the parity check.

VRS does have a sanity checker for things like that, but it's not very good when more than about 30 seconds have elapsed between messages. I think basically I need to write a better sanity checker.

What i meant with it that this 'flight' seemed to have copied the data of another flight of the same operator since it had the exact same height... That would be a bug and not a bad reception or something like that in my eyes. I had sorted the flights by flag at that moment. I've seen other weird stuff but that was all due to errors in reception, this one lust looked too strange to catalogue as bad data
I have a flightfeeder from flightaware btw...

I have been tracking a similar problem in dump1090.

The most likely cause I've found is when there are two aircraft within receiver range that have ICAO codes that differ by only a small number of bits - most commonly by single bit (it happens more often than you'd think!). Then a DF0/4/16/20 response that has damage to the Address/Parity data can easily be attributed to the wrong aircraft, as both the correct and damaged residual CRC syndromes look like plausible ICAO addresses.

The approach I'm looking at using in dump1090 is (a) sanity checks on altitude changes and (b) ignore DF0/4/16/20 altitude data when we have heard DF17/18 ES positions from the aircraft (semi-recently); DF17 is going to be much more reliable since it's got full CRC coverage.

[agw]

I'm having this issue and it occurs randomly.

It looks like a connection to a rebroadcast server may be going down and the main display is trying to fetch the local and remote endpoint addresses for it before it has been cleaned up.

I've fixed the exception that you were hitting, I can send you a link to the preview installer for it if you want to try it out.

[YSWG]

I think the handle issue is to do with the "Custom Content" plug in, disabling it seems to have done the trick. Three days and the handles are at 2,725, where as before it was around 5,000+.

[agw]

2725 is still pretty high. When I looked at your dump file the excessive handles were all native unmanaged thread handles. VRS doesn't allocate those directly, it allocates threads via the .NET framework, and in the dump file there were only something like 20-ish managed threads (i.e. threads that VRS allocated via .NET and still had references to). In eshabi's dump file his excessive handles weren't thread handles, they were something else and so far (touch wood) the preview version - which didn't work when you tried it - is working for him, the last I heard from him his handle count is normal.

I did notice in your dump file that some 3rd party things have hooked themselves into the VRS process, namely an ATI DLL and an anti-virus DLL. I don't think that either of these would be allocating lots of threads and leaving them dangling, but I'm wondering if something that VRS is doing might be antagonising one of them. I'd be surprised if it was that though.

The custom content plugin is just exposing functionality that is built into VRS, injecting content into pages is done by VRS itself and not by the plugin. The custom content plugin is only a GUI that exposes those bits of VRS' internals. Content injection happens even if you don't have the plugin installed; it happens if you have bundling turned on, which you do by default. That's not to say that there isn't something in the custom content plugin that could be triggering this, but given that it's just setting up more of what VRS already does, and that I've not seen excessive unmanaged dangling threads when I use it, I suspect that might be a red herring.

However, if you could send me the HTML files that you were injecting via the custom content plugin and the plugin configuration file (Help | About, click the link to the configuration folder and then it's the PluginsConfiguration.txt file from there... also if you could send me Configuration.xml, VirtualRadarLog.txt and ConnectorSnapshots.txt from that same folder as well, they might hold some clues) then I can use those to configure the same setup here and see if it triggers the problem for me. Those files should all be small enough to send via email to andrew@virtualradarserver.co.uk, you don't need to use Dropbox for them.

[ssfer]

I'm having this issue and it occurs randomly.

It looks like a connection to a rebroadcast server may be going down and the main display is trying to fetch the local and remote endpoint addresses for it before it has been cleaned up.

I've fixed the exception that you were hitting, I can send you a link to the preview installer for it if you want to try it out.

The issue above has been solved on the preview version. But now, from time to time the feeder stops sending data, it shows as connected and no error is reported. I have to right click on the feeder and reconnect it.

This is just a heads up. I have disabled the option "Send keep-alive packets" and lets see it happens again.

[ssfer]

I'm having this issue and it occurs randomly.

It looks like a connection to a rebroadcast server may be going down and the main display is trying to fetch the local and remote endpoint addresses for it before it has been cleaned up.

I've fixed the exception that you were hitting, I can send you a link to the preview installer for it if you want to try it out.

The issue above has been solved on the preview version. But now, from time to time the feeder stops sending data, it shows as connected and no error is reported. I have to right click on the feeder and reconnect it.

This is just a heads up. I have disabled the option "Send keep-alive packets" and lets see it happens again.

It happened again. This time send keep-alive packets was unchecked. It still losing connection, actually it show as connected, but no data is transferred. Just a detail I'm using OpenVPN between VRS and RTL(dump1090).

[YSWG]

However, if you could send me the HTML files that you were injecting via the custom content plugin and the plugin configuration file (Help | About, click the link to the configuration folder and then it's the PluginsConfiguration.txt file from there... also if you could send me Configuration.xml, VirtualRadarLog.txt and ConnectorSnapshots.txt from that same folder as well, they might hold some clues) then I can use those to configure the same setup here and see if it triggers the problem for me. Those files should all be small enough to send via email to andrew@virtualradarserver.co.uk, you don't need to use Dropbox for them.

Well it had been running fine 25th of February to the 5th March. Didn't check the Handles when I got the error, sorry.

Code: Select all

[2015-03-05 11:48:12.385 UTC] [t22] Exception caught during data download: System.InvalidOperationException: Could not download http://www.virtualradarserver.co.uk/Files/FlightNumberCoverage.csv: The remote name could not be resolved: 'www.virtualradarserver.co.uk' ---> System.Net.WebException: The remote name could not be resolved: 'www.virtualradarserver.co.uk'
   at System.Net.HttpWebRequest.GetResponse()
   at VirtualRadar.Interface.WebRequestHelper.<GetResponse>b__0(WebRequest r)
   at VirtualRadar.Interface.WebRequestHelper.RepeatActionOnTemporaryErrorCondition[T,TResponse](T obj, Func`2 action, Boolean suppressRepeat)
   at VirtualRadar.Interface.WebRequestHelper.GetResponse(WebRequest request, Boolean suppressRepeat)
   at VirtualRadar.Database.StandingData.StandingDataUpdater.DefaultProvider.DownloadToStream(String url, Stream stream, Boolean urlIsGzipFile)
   --- End of inner exception stack trace ---
   at VirtualRadar.Database.StandingData.StandingDataUpdater.DefaultProvider.DownloadToStream(String url, Stream stream, Boolean urlIsGzipFile)
   at VirtualRadar.Database.StandingData.StandingDataUpdater.DefaultProvider.DownloadLines(String url)
   at VirtualRadar.Database.StandingData.StandingDataUpdater.Update()
   at VirtualRadar.Database.StandingData.BackgroundDataDownloader.Heartbeat_SlowTick(Object sender, EventArgs args)

From the 5th March to today (12th), running ok and handles are at 6464.

I'll get those files to you when I have some spare time.