Results of microphone tests

[David Morrison]

Dear MacVoice readers

Over Christmas, I had a chance to do some testing of microphones with 
iListen and Via Voice.

The results I got were not exactly in line with conventional wisdom 
and reports I have read on this list. I wonder whether other people 
have done similar testing and what their results were.

Note that this is a comparison of microphones over a specific range 
of tests that try to eliminate variations in the software being used. 
It is not a comparison of iListen and Via Voice.

The System:

Mac G4/450DP, using Mac OS 10.2.3. (10.2.3 has some audio fixes, so I 
thought that might help.)

The Microphones:

Andrea 7100 USB mic (a noise cancelling USB mic supplied with Via Voice)
Andrea 600 mic (an active noise cancelling analogue microphone) 
together with an Andrea USB Pod (USB sound adapter) or a Griffin NE 
Mic (analogue Plaintalk adapter)
Shure WH20 wireless analogue mic together with an Andrea USB Pod or a 
Griffin NE Mic. (This microphone has a powered radio receiver with a 
gain control, so the signal level can be adjusted to an optimum 
value.)

The Software:

Via Voice 3.0 (Only supports USB microphones)
iListen 1.5.2 (Supports USB and analogue microphones)

The Environment:

I was at home in a carpeted room. Two outside doors and some windows 
were open. Traffic noise could be heard from about 200 metres away. 
Birds were chirping loudly just outside. My G4's fan was making its 
usual wind-tunnel noise :-(

The tests were done over several days, and the background noise 
varied a little.

The Test Procedure:

For each combination of software and microphone, I did:

Select the appropriate sound input and set the gain to maximum in the 
Sound system preference and the Speech system preference.
Start the software and create a new profile
Set up the microphone in the software
Read the first two stories and get them analysed (Note: The stories 
in iListen are shorter than the stories in Via Voice, so iListen 
could be expected to learn less of my pronunciation and therefore be 
less accurate.)
Dictate a paragraph of 65 words a number of times, with no correction.
Calculate the average number of recognition errors
I also read a portion of a letter (304 words), but only once.


The Results: Via Voice
=============
Andrea 7100 USB mic

Mic setup: sound quality reported as Very Good.
Read the paragraph 5 times: average errors 5.2
Read the letter: 21 errors

Andrea 600/Pod

Mic setup: was not able to get past this stage for several days. 
Everything I tried - tapping the microphone, almost yelling the test 
sentence - yielded a message that the microphone was not connected, 
even when the sound was showing up in the VU meter. Eventually, I 
tried tapping the microphone element quite hard, so that the VU meter 
went almost to its limit, and was able to pass this point. The sound 
was reported as fair, then good when I read the setup paragraph 
again. Seems like a Via Voice oddity that it will not recognise it 
initially.
Read the paragraph 8 times: average errors 8.0
Read the letter: 21 errors

Shure WH20/Pod

Mic setup: sound quality reported as Very Good.
Read the paragraph 9 times: average errors 4.4
Read the letter: 13 errors


The Results: iListen 1.5.2
===============
Andrea 7100 USB

Mic setup: iListen set the gain to 100%, and rated it as satisfactory 
for voice recognition.
Read the paragraph 3 times: average errors 4.0
Read the letter: 54 errors

Andrea 600/Pod

Mic setup: Unable to set the volume automatically. I set the volume 
to the maximum and iListen reported the quality as insufficient for 
voice recognition.

Andrea 600/Griffin NE Mic

Mic setup: Unable to set the volume automatically. I set the volume 
to the maximum and iListen reported the quality as insufficient for 
voice recognition.

Shure WH20/Pod

Mic setup: iListen set the gain to 100%, and rated it as satisfactory 
for voice recognition.
Read the paragraph 8 times: average errors 6.0
Read the letter: 47 errors

Shure WH20/NE Mic

iListen set the gain to 87% and rated it as satisfactory.
Read the paragraph 8 times: average errors 6.2
Read the letter: 49 errors

Observations
========
The Andrea 7100 seemed to perform reasonably with both Via Voice and 
iListen. From other straight audio tests, the signal level is 
relatively low. MacSpeech reports mixed results with this mic and 
iListen. It is the standard mic for Via Voice.

The Andrea 600 could not be made to work with iListen at all, and was 
marginal with Via Voice. Its accuracy was no better than the 7100, 
and in some tests much worse. Other people report that this mic 
provides better accuracy than the 7100 (with Via Voice? Has anyone 
got it to work with iListen?). In audio tests, its signal was around 
25-50% of that of the 7100, which is apparently to be expected 
because of the nature of the noise cancelling mechanism.

The Shure WH20 with the Andrea Pod worked well with Via Voice, being 
more accurate than either of the Andrea mics. With iListen, it was 
comparable to the Andrea 7100.

Conclusions
=======
This test was performed in a moderately scientific way. It is however 
only a single test with a single computer, one speaker and one sample 
of each microphone. The same tests on a different computer (Mac or 
PC), with a different speaker or with different microphones may 
produce totally different results.

That said, it seems clear that the two programs have different 
requirements for sound input. Some microphones seem to work with one 
program but not the other. Some work with both.

I was looking for a microphone that could be used with both programs. 
 From these tests, I cannot say that the Andrea 600 is it. I would be 
interested to know of microphones (analogue or USB) that work well 
with both programs.

Regards

David


-- 
David Morrison, Systems Specialist
Flexible Learning Administrator and Blackboard Project Manager
Communication and Information Services, University Services
The University of Newcastle, Australia
E-mail David.Morrison at newcastle.edu.au
Ph +61 2 49215397   Fax +61 2 49217087