[hpsdr] Observations on the new HPSDR firmware/PowerSDR

[Dan Quigley]

Hi Bill,

Yes I am using Process Explorer.  This is an odd one, but I believe it is
germane to problems others have reported.  The length of time appears to be
random from my experience. (after today from 0 to a couple of hours)  I am
90% convinced it is a PowerSDR issue and related to (or exacerbated by)
sample rate but I have not found a reproducible trigger.  There may be a
correlation to the GC - I watched the .NET runtime and its memory heap was
definitely fluid during PowerSDR abnormal operations but that may be a
reaction to something else.  This does smell a bit like a race condition
issue but it is too early to be sure.

Some additional observations:

Last night I chatted for 2 hours on 75m phone using HPSDR without any issue.
Rx was stellar and CPU stayed under 30% for the most part and there were no
unusual audio reports on tx. 

Early this morning I wanted to make some CQWW WPX contacts on 20m.  When I
powered up the rig, I observed the relay chatter and distorted audio
(described by others), the CPU was north of 80% and the system was totally
unusable (Mode or DSP functions did not matter). When I dropped the sample
rate to 48k it settled right down.  An hour later I changed the sample rate
back to 192k and it worked fine for an hour or so.  During contest ops
today, I had to stop PowerSDR three times and restart it, but I never
observed the morning chatter/distortion issues later in the day.  I have not
found it necessary to power cycle the hardware.

This may be two separate problems.  The only time I have seen the
chatter/distortion issue was after this morning's cold start.  The
degradation/pop issue is reproducible, but on a random basis. 

Tonight I am on 75m phone again and have encountered no problems.
Wash-Rinse-Repeat... I'll try 20m tomorrow morning again.

If you can send or point me to PowerSDR release symbol files, my stack
traces may be more helpful...

Best,
Dan

-----Original Message-----
From: Bill Tracey [mailto:bill at ewjt.com] 
Sent: Sunday, May 31, 2009 9:45 PM
To: Dan Quigley; hpsdr at openhpsdr.org
Subject: Re: [hpsdr] Observations on the new HPSDR firmware/PowerSDR

Hi Dan,

Interesting observations -- will have to look 
into it.  How long does it take it to degrade to 
where you're hearing bands and pops?  I've had 
mine running listening to a tone for the  last 30 
mins or so and haven't heard it glitch.   I'm on 
a Quad Core on Vista so have a bit different setup than you do.

A time oriented degradation leads me to the 
perhaps garbage collection issues, but I did not 
think any of our main path (audio buffers) stuff 
relied on garbage collection.

How are grabbing the call stacks - Process Explorer?

Looks like it might be time to learn a bit on .NET profiling.

Cheers

Bill (kd5tfd)

At 04:52 PM 5/30/2009, Dan Quigley wrote:
>***** High Performance Software Defined Radio Discussion List *****
>
>
>Content-Type: multipart/alternative;
>         boundary="----=_NextPart_000_0006_01C9E136.326233F0"
>Content-Language: en-us
>
>I hope this helps.
>
>On my WinXP box running on an Intel 2.1Ghz duo 
>core with 3GB of ram, PowerSDR works fine (read 
>great!) for a "while" then the receive function 
>gradually starts degrading (drop outs/pops etc) 
>until audio eventually stops.  CPU also rises 
>with the incidence of popping/drop outs.  How 
>long it takes to degrade appears to be related 
>to audio buffer size and sample rate 
>settings.  The higher the sample rate the less 
>amount of time it takes for performance to 
>degrade.   Transmitting also appears to 
>accelerate the time it takes to degrade.
>
>Good performance returns when you  restart 
>PowerSDR (first waiting for the previous 
>instance to unload, which can take a while.
>
>When running fine (e.g. no drop outs) , the 
>thread that consumes most of the CPU cycles in 
>the PowerSDR.exe process has this call 
>stack  (or one very similar), at 192K bandwidth.
>
>Thread 5-11% cpu
>ntkrnlpa.exe!NtBuildNumber+0x43
>ntkrnlpa.exe!MmIsDriverVerifying+0xbb0
>ntkrnlpa.exe!MmIsDriverVerifying+0x1492
>ntkrnlpa.exe!PoShutdownBugCheck+0x39e9
>ntkrnlpa.exe!KeSynchronizeExecution+0x30c
>ntdll.dll!KiFastSystemCallRet
>KERNEL32.dll!WaitForMultipleObjects+0x18
>pthreadVC.dll!pthread_exit+0x111
>
>Once performance degrades and drop outs begin to 
>occur frequently.  This thread and call stack 
>floats quickly to the top of the sorted by cpu 
>list.  Then stays at the top.  This thread jumps 
>to  50% cpu  when you close PowerSDR and stays 
>there for a minute or so.  When it eventually 
>terminates, PowerSDR is unloaded from memory:
>
>Thread 34% cpu
>Stack:
>ntkrnlpa.exe!NtBuildNumber+0x43
>ntkrnlpa.exe!MmIsDriverVerifying+0xbb0
>hal.dll!HalClearSoftwareInterrupt+0x342
>DttSP.dll!SetKeyerSampleRate+0x27c8
>DttSP.dll!PolyPhaseFIRF+0x28ca
>
>When the system has totally degraded this thread stays at 15% cpu
>Thread 15% cpu
>Stack:
>ntkrnlpa.exe!NtBuildNumber+0x43
>ntkrnlpa.exe!MmIsDriverVerifying+0xbb0
>ntkrnlpa.exe!MmIsDriverVerifying+0x1492
>ntkrnlpa.exe!PoShutdownBugCheck+0x3702
>ntkrnlpa.exe!KeSynchronizeExecution+0x30c
>ntdll.dll!KiFastSystemCallRet
>KERNEL32.dll!WaitForSingleObject+0x12
>libusb0.dll!usb_set_debug+0x266
>libusb0.dll!usb_reap_async+0xf
>libusb0.dll!usb_bulk_write+0x1e
>JanusAudio.dll!OzyBulkWrite+0x21




 1243833724.0