User:Mjb/DAE quality testing
The author of Exact Audio Copy also has a project called DAE Quality, which is the only way you can really test how well your CD drive performs these functions:
- error correction – detecting erroneous data and replacing it with correct data
- error concealment or error hiding – detecting erroneous data and replacing it with incorrect, but hopefully less audible data
- C2 pointer reporting – detecting erroneous data and reporting to the ripper, regardless of whether it has been corrected or concealed
C1 & C2 error correction data is on every CD, and all drives process it. Drives use the C1 data to deal with manufacturing defects and tolerances at the pits-and-lands level. Drives use the C2 data to detect and attempt to fix the byte-level errors in audio data, as may be caused by dust, smudges, scratches, etc.
As audio data is read from the CD, each of the four bytes in each stereo sample in a 6-sample frame are tested against that frame's C2 data. Any byte which fails to validate is subjected to some kind of error correction attempt, such as:
- actual correction: C2 data, if undamaged, allows fixing up to 3 bytes, each having a single corrupt bit (I think)
- interpolation: making an educated guess based on nearby previous samples, or just using the data as-is
- zero-order hold: repeat the last sample value
- muting: replace affected sample with zeroes, maybe adjust nearby samples to do a very quick fade-out and fade-in
Many CD drives can report the outcome of this C2 processing to the ripper...sorta. What they report, for each byte, is a "C2 pointer". This is a flag which indicates that an error was detected (maybe) and corrected or concealed. Unfortunately, it seems drives are free to define "corrected" however they want; it doesn't mean the samples are 100% correct. It is also quite possible that some errors slipped through undetected by the drive's C2 processing system, which by design has a 1 in 1000 chance of missing an error.
Anyway, if a secure (re-read-capable) ripper makes use of these pointers, it can speed up ripping by trusting that the drive has corrected some errors. Using C2 pointers can reduce the likelihood that what appears to be good data is actually consistent errors. I'm still trying to wrap my head around how rippers actually use the pointers to accomplish this, though.
The only way to figure out how good your drive's C2 processing and reporting is is to rip a special CD that will produce erroneous audio data when read. The DAE Quality project allows you to make such a CD and test how well your drive rips it, without any help from the ripper. Below, I'm documenting my adventures doing just that.
Contents
Caveats
- Results can only be compared when the same CD is used.
- There's a thread that suggests C2Extract.exe doesn't handle sync errors very well, which screws up the results. (That thread also says C2 error reporting may be near 100% reliable for all drives, which I have trouble reconciling with everything else I've heard, like Feurio's old FAQ, for example.)
Procedure
- Get the DAE Quality testing software from the maker of Exact Audio Copy.
- Follow the instructions! But here's what the instructions don't tell you:
- Make sure when you print the PDF, unset any resizing options ("fit to margins", etc.)
- The file to run to create the WAV is actually called CreateReference.EXE, not reference.exe.
- Creating the WAV is super slow, even on my fast, modern PC, so I suggest just finding the file somewhere else. I'm keeping a copy for posterity.
- In EAC, when you rip the WAV, make sure you have set Burst mode.
- You're supposed to try out different drive speeds in EAC, because different speeds produce different results, but not all drives allow their speed to be set in EAC, apparently. Mine only lets me choose "Current".
- Likewise, you're likely to get different results if you enable Allow drive speed reduction. Whether it will be better or worse, who knows. Try and see.
- When you run C2Extract.exe, the files it creates will be named for the drive, e.g. PLDSDVD-RWDH16ABSH.wav
- When you run Analyse.exe, the first argument is the .wav to analyze, and you can optionally specify a graph size, e.g.
Analyse.exe PLDSDVD-RWDH16ABSH.wav 1600 1000 - In the instructions, where the
-c2flipnotes say "red and blue graph nearly as high as the red one", I think it is supposed to say "red and blue graph nearly as high as the green one".
Files created by Analyse.exe, when run on Reference.wav ripped by EAC:
Reference.bmpReference.datReference.log
Files created by Analyse.exe, when run on the WAV produced by C2Extract.exe:
drivename.bmpdrivename.datdrivename.logdrivenamec2.bmpdrivenamec2.dat
The .bmp is the graph. The .log is brief text file that's the same as the console output when the analyzer is done. The .dat is data from which the graph and log were derived.
My results
PLDSDVD-RWDH16ABSH
This is my desktop PC's drive.
EAC's rip
I had configured EAC to let the drive slow down to fix errors, so the rip took a long time to get through the worst parts.
- X-axis = minutes; 1-minute increments
- Y-axis = dBFS (A-weighted), a.k.a. dB(A); 6 dB increments; black line = 0 dB
The spikes indicate uncorrected errors. The spaces in between are error-free, either because they were read cleanly, or because the drive corrected them.
The first batch of spikes is the blacked-out part of the disc. The five batches of spikes on the right side of the graph are the scratches. Since each batch is more spikey than not, we can say this drive's error correction is poor to fair, at best. Since the spikes are almost all below -60 dB, except in the worst parts, we can say this drive's error concealment is pretty good.
The log file gives us more clarity:
Errors total Num : 14157098 Errors (Loudness) Num : 75876 - Avg : -26.8 dB(A) - Max : -7.4 dB(A) Error Muting Num : 8639 - Avg : 146.9 Samples - Max : 3584 Samples Skips Num : 2 - Avg : 12.0 Samples - Max : 12 Samples Total Test Result : 65.1 points (of 100.0 maximum)
That is, the rip produced 14,157,098 uncorrected (i.e., badly corrected) samples, which is about 5.4 minutes of potential glitches. To get a rough estimate of how loud the glitches are, the analyzer divided that 14 million into about 75 thousand chunks, and used an FFT to get a volume level for each chunk. The loudest chunk was -7.4 dB(A), but the average was -26.8. Also, 8,639 samples were "muted" (I don't like this term; "held" is better), meaning they were repeating the last good sample. The longest stretch of held samples was 3,584, and the average was 146.9. I'm not sure if that's good or bad. There were 2 skips (sync errors), meaning the drive thought it was reading from a certain spot, but the analyzer reveals it was actually off by some amount, resulting in two 12-sample chunks containing data from slightly ahead or behind where it should've come from.
These numbers will be more meaningful once we compare them to other rips of my specific test disc. Ideally, we'd try different drive speeds, and different drives.
C2Extract's rip
This went faster. It also seems to be better.
Look at that...the spike density is actually increased in the blacked-out part of the disc, but one of the scratches was corrected altogether! And the spikes in the scratched areas are significantly quieter than the ones in the EAC rip.
The log shows even more craziness:
Errors total Num : 344541 Errors (Loudness) Num : 12142 - Avg : -71.3 dB(A) - Max : -31.1 dB(A) Error Muting Num : 587 - Avg : 1.1 Samples - Max : 3 Samples Skips Num : 0 - Avg : 0.0 Samples - Max : 0 Samples Total Test Result : 79.8 points (of 100.0 maximum) C2 Accuracy : 100.0 %
The bad sample count dropped by 75% as compared to EAC's rip! The majority of errors that are left are probably inaudible, or very faint (-71 dB!). And no skips. This is astonishing to me.
C2 accuracy "100.0%" is great, but I'm betting it's misleading. It could be 99.95% or higher, just rounded up. I think this is confirmed by the blue part of the C2 graph:
- x-axis = minutes, as before
- y-axis, each graph = error count, scale unknown but logarithmic (each line is twice as many as the line below it)
- Green = reported C2 errors, both corrected and uncorrected
- Red = corrected C2 errors, both C2-corrected and interpolated
- Blue = uncorrected errors that weren't reported as C2 errors
So the green is showing detected errors, red is the subset that were perfectly corrected, and blue is overlooked errors.
The fact that the blue line has a couple of spikes indicates that C2 accuracy is not truly 100%.
So what does this mean for me? Should I rely on C2 pointers?
EAC rip with no drive speed reduction
Although I told it not to reduce the speed, the rip did get slow at the end. However, I think it still went much more quickly than the first EAC rip.
Yikes. Even more errors than the first go-'round. What's going on, EAC?
Errors total Num : 18205624 Errors (Loudness) Num : 93502 - Avg : -24.7 dB(A) - Max : -5.6 dB(A) Error Muting Num : 12533 - Avg : 5.7 Samples - Max : 3584 Samples Skips Num : 0 - Avg : 0.0 Samples - Max : 0 Samples Total Test Result : 74.2 points (of 100.0 maximum)
cdda2wav rip
I didn't set the drive speed. I just used this command line:
cdda2wav.exe -D 0,1,0 -B -vall
Errors total Num : 363494 Errors (Loudness) Num : 12611 - Avg : -71.0 dB(A) - Max : -28.7 dB(A) Error Muting Num : 467 - Avg : 1.1 Samples - Max : 5 Samples Skips Num : 0 - Avg : 0.0 Samples - Max : 0 Samples Total Test Result : 80.0 points (of 100.0 maximum)
CUERipper rip
It may not be fair to include CUERipper here, because CUERipper's burst mode re-reads sectors with C2 pointers, betting that re-reading will get better results than the drive's correction & concealment strategy. I'd like to see an analysis of how well this actually works. Also, I think this capability was available but removed from EAC because it was allowing some copy protected CDs to be ripped, or something.
Just eyeballing it, it looks like it worked out OK, especially in the blacked-out part of the disc.
Errors total Num : 292734 Errors (Loudness) Num : 10118 - Avg : -70.7 dB(A) - Max : -22.9 dB(A) Error Muting Num : 397 - Avg : 1.1 Samples - Max : 3 Samples Skips Num : 0 - Avg : 0.0 Samples - Max : 0 Samples Total Test Result : 80.5 points (of 100.0 maximum)
