# ABX Test of 320kbps VS FLAC - Results



## Berlioz

I realize that there is another thread just like this one in this sub forum, but I didn't want to hijack the thread, and I was curious enough to do my own test.

 The structure of the test went like this:

 I used the ABX Comparator on foobar2000. It is an optional utility during installation of the program.

*How it works:*

 You choose two files to test. You can pick what section of the song to examine, and how short it is etc.

 There are two sets of two choices each: [A and B] & [X and Y]

 You can compare any of them to one another at any time, as many times as you like. Basically, you need to match them up in pairs.

 One of the [A and B] choices will be lossless, and one will be the 320kbps version.
 Same thing goes for the [X and Y] choices.

 I would compare the A and B tracks, and decide which one sounded better. Then I would do the same with the X and Y tracks. If B sounded better than A, and X sounded better than Y, I would pick the option "B is X". 

 Sorry if that's a little confusing for some people; that's the best way I can word it.

*I chose four pieces:*

*1)* Symphony No. 1 by Dmitri Shostakovitch - As with all of his work, this dynamic piece is packed with energy. I was curious if I could separate the two versions based on transients.

*2)* Gold Rush by Dragonette - Electro Pop that is full of compressed sounds, particularly in the low range. I thought it would be a good test of the type of mastering that most people listen to on the radio.
*
 3)* Rambaldo!...Ah! M'aiutate! - A section of an Italian Opera _La Rondine_ by Giacomo Puccini. Vocals and the backing orchestra constantly strive for top spot across the entire piece. This recording won the 1997 Gramophone Award for "Record of the Year".

*4)* Sunshine of Your Love by Cream - This song got me into music in the first place, so I couldn't leave it out! Clapton's riffs are overdriven yet articulate at the same time. Coupled with his vocals, it's another good test of dynamics in my opinion.
*
 Preparation:*

 For equipment, I was simply using my Shure SE530's connected to the headphone jack on my computer.

 All of the tracks were copied twice using EAC. Once in a 320 kbps .mp3 format using the LAME encoder. The second time, in .flac using the FLAC encoder bundled with EAC.

 There would be 40 replicates of each test, per song, so 160 trials altogether. I could compare the tracks as many times as I liked, and I took breaks about every 10 trials (trust me, I needed them!).

 All of the MP3's are compressed to have a bitrate of 320 kbps.
 The bitrates for the FLAC files are as follows:

 Symphony No. 1 - 427 kbps
 Gold Rush - 1001 kbps
 Rambaldo!...Ah! M'aiutate! - 628 kbps
 Sunshine of Your Love - 818 kbps

 Foobar2000 also calculates your probability of randomly guessing and getting the same results,although it only goes to 1 decimal place. To calculate the values on your own, with more decimal places, use the binomial probability equation.

*Results:*

*Symphony No. 1 -*

 Correctly differentiated them 28/40 times. Probability of guessing: 0.8%

 I was very pleased with this result. It was *extremely* hard to differentiate the selections, even though I focused on a variety of sections of the song. The least complex sections that I tested gave me the most trouble.

*Gold Rush*

 Correctly differentiated them 18/40 times. Probability of guessing: 78.5%

 This made me sad. I have _always_ sworn that Martina Sorbaras voice sounded sexier on FLAC. Oh well. I honestly couldn't make out any difference between the two versions. About a dozen or so replicates in I was already getting discouraged. It was *impossible* to hear any difference.

*Rambaldo!...Ah! M'aiutate!*

 Correctly differentiated them 22/40 times. Probability of guessing: 31.8%

 I had high hopes for this one. At this point, I began to realize what others have been saying. I tried as hard as I could, but I could not discern any differences whatsoever. I would hear A to have crisper vocals than B, and hear the _same_ quality in Y. I would pair A up with Y....and be wrong!
 Oh well, there's still another test left!
*
Sunshine of Your Love*

 Correctly differentiated them 24/40 times. Probability of guessing: 13.4%

 Again, it was impossible to tell the difference. I took a lot of breaks, and felt well rested, but it seemed to make no difference.

*Conclusion:*

 Only one of the results had enough of a difference to be statistically significant. In doing this test, I discovered just how close 320kbps and flac really are. Even with Symphony No.1, I was focusing on the music so ****ing hard, to the point where it wasn't even enjoyable anymore. 

 It's important to note, that this is only one set of tests, and really doesn't mean anything in the big picture. Maybe my hearing sucks. Maybe my equipment is so poor I can't hear all the little nuances. 

 However, the fact alone that it was so hard to tell them apart in the first place...there's not _nearly_ as much difference between the two as I found in sighted "tests".

 I take back any statements I have ever made as to whether I can hear a difference between FLAC and 320kbps MP3s. I didn't know what the hell I was talking about. It's funny, because as soon as I play them again the .flac one sounds so much crisper. But I know it's all in my head now.

 I hope that anyone reading this enjoyed my experiment, and maybe you'll be curious enough to do a test of your own.

 Any comments/questions are welcome.


----------



## krmathis

Nice write-up. 
	

	
	
		
		

		
			




 But to me this result mean that you can indeed hear a difference between 320kbps and lossless.
  Quote:


  Originally Posted by *Berlioz* /img/forum/go_quote.gif 
_Symphony No. 1 -

 Correctly differentiated them 28/40 times. Probability of guessing: 0.8%_

 

But of course not on all kind of music/samples, and you need full focus to differentiate them.


----------



## Berlioz

Oh very true. However, I was straining incredibly hard to focus. If I was in my music listening mood(I'm always in this mood!), it's unlikely that I would be doing this. It wasn't exactly enjoyable.

 Now, I still believe it's possible to discern 320kbps from lossless. However, the difference I experienced between the two was very small. That's what really got me. Before, I could hear huge differences between them. I'd play my FLAC files and think to myself, "Yeah, that's why I spent all that time burning those CD's." It was blissful. In reality though, they are _almost_ exactly the same, at least to my ears - which, at least according to those hearing tests someone posted - are pretty good.

 When I get my home set up complete, I'll test it again. I expect it to be easier to differentiate the two, using better equipment, but it should still be pretty hard. Harder than I thought anyways.


----------



## krmathis

Fair enough!
 In most cases you are not able to hear and audible difference, hence why the increased size may not be worth it to you.


----------



## CodeToad

LAME is good stuff. Most of the problems I've heard with mp3's have been with stuff encoded by WMP (Fraunhoffer?) at 320 CBR.

 With LAME I can usually go with VBR4 and be ok with it on portables or in the car but there's just no way I'd go anything less than FLAC on the 'puter.

 I've played with the f2k abx'er and most of the time I can hear (Flac v. WMP 320 CBR's) a difference but can't always tell which is which. I applaud your 40 round tests...that's dedication.


----------



## jinp6301

You might want to try this with a better source. With the headphone jack off my laptop, I only get around 50% correct. With a high end dac, I can get about 90+% right most of the time

 ^based on previous tests I've done


----------



## mike1127

Quote:


  Originally Posted by *Berlioz* /img/forum/go_quote.gif 
_Oh very true. However, I was straining incredibly hard to focus. If I was in my music listening mood(I'm always in this mood!), it's unlikely that I would be doing this. It wasn't exactly enjoyable._

 

Some people (such as myself) theorize the important differences between components show themselves mainly in a "music listening mood" and in "enjoyment" rather than during straining. 

 (Note: *some* differences do show themselves during extremely close concentrated listening, as we know. However, that doesn't mean that *all* differences do, or that the most important ones do.)


----------



## revolink24

Congrats on your patience, it isn't easy.

 I conducted a test of this on my own. at 192, I was pretty much flawless. At 256, I was only noting subtle differences in transparency and dynamics, and only got around 5/7 of the trials correct. At 320, the differences were almost inaudible: I failed 40% of the trials.


----------



## Earwax

Quote:


 Symphony No. 1 - 427 kbps
 Gold Rush - 1001 kbps
 Rambaldo!...Ah! M'aiutate! - 628 kbps
 Sunshine of Your Love - 818 kbps 
 

It's interesting that higher flac bitrates didn't predict which pieces you'd have trouble ABXing. That's not at all what I would have guessed. 

 I am curious about a couple of things. Can you describe the type of difference you heard between the Shostakovitch tracks? How long did it take you to do all those trials? Was this all done in one day or spread out over time?


----------



## elrod-tom

Quote:


  Originally Posted by *jinp6301* /img/forum/go_quote.gif 
_You might want to try this with a better source. With the headphone jack off my laptop, I only get around 50% correct. With a high end dac, I can get about 90+% right most of the time

 ^based on previous tests I've done_

 

I'd tend to agree. I appreciate the work you put into this, but if it's done on the typical PC headphone jack I suspect that it's not allowing a lot of details to come through that might otherwise be more revealing of differences.


----------



## progo

Quote:


  Originally Posted by *Earwax* /img/forum/go_quote.gif 
_It's interesting that higher flac bitrates didn't predict which pieces you'd have trouble ABXing. That's not at all what I would have guessed. 
_

 

Actually, while the flac bitrates don't mean anything, they might correlate to loudness. In general, hot CDs I've ripped are usually over 1000 kbps but the classical and jazz can be 400-700 kbps. But the numbers don't mean anything in the end. Just the saved amount of disk space.


----------



## ph0rk

Quote:


  Originally Posted by *Berlioz* /img/forum/go_quote.gif 
_Oh very true. However, I was straining incredibly hard to focus. If I was in my music listening mood(I'm always in this mood!), it's unlikely that I would be doing this. It wasn't exactly enjoyable.

 Now, I still believe it's possible to discern 320kbps from lossless. However, the difference I experienced between the two was very small. That's what really got me. Before, I could hear huge differences between them. I'd play my FLAC files and think to myself, "Yeah, that's why I spent all that time burning those CD's." It was blissful. In reality though, they are almost exactly the same, at least to my ears - which, at least according to those hearing tests someone posted - are pretty good.

 When I get my home set up complete, I'll test it again. I expect it to be easier to differentiate the two, using better equipment, but it should still be pretty hard. Harder than I thought anyways._

 

I'd also investigate the mp3 encode - which version of lame, what were the settings? from version to version there are often specific pieces of music that remain non-transparent at even the highest settings - these are called "killer samples".

 I'd try it with whatever the analog for --alt-preset insane (v0 insane, "=b 320" I think?)

 Takes longer to encode, but with todays multicore machines who cares 
	

	
	
		
		

		
		
	


	





 There is supposedly very nearly a negligible difference between high bitrate AAC and MP3 - but they have different killer samples and artifacts. I'd be curious if you could tell between an AAC and FLAC of your identifiable piece (I'd suggest apple's AAC encoder)


 Of course, going with high bitrate stuff is a slippery slope - there isn't much difference between 120ish kbps more than transparency and 400 kbps more than transparency, other than space, as long as your formats are supported.


----------



## prone2phone

i am happy that flac doesnt sound worse. the more audio formats supports the player the better the player is :}


----------



## Headdie

Thank you Berlioz ! That's serious testing, the kind that I like to hear about. I'm very interested by this, because I still have to rip my CD collection. Two questions :

 1. When you heard a difference between MP3 and FLAC, was it a continuous difference (meaning that you could hear it all time), or was it a ponctual difference (meaning that you could hear it at some places/passages only) ?

 2. I understand that MP3-320 is a very good format, but the encoder could be the weak link. It would means that encoding with WMP or LAME would not yield to bit identical MP3 files. Is that tested and confirmed ?

 Thanks,


----------



## krmathis

Quote:


  Originally Posted by *Earwax* /img/forum/go_quote.gif 
_It's interesting that higher flac bitrates didn't predict which pieces you'd have trouble ABXing. That's not at all what I would have guessed._

 

I am not as surprised!
 Cause for lossless the bitrate is not a reflection of the sound quality as all (like for lossy), but just a reflection of the complexity of the music and/or the efficiency of the encoder.


----------



## mike1127

I'm going to try this kind of comparison, but not using foobar2000 because my computer rig is nowhere near as good as my Naim CDP. Instead I'm going to burn a CD, and each track will either come from an MP3 or a WAV. I'm a programmer so I'm going to write software that chooses the source for each track randomly.

 My computer rig is this:

 Audiophile 2496 digital coax out -> CAL Sigma II DAC -> (very long and sh**ty cable) -> DNA Sonett (amp)

 For CD listening I have

 Naim CD5x -> Cardas Neutral Reference -> DNA Sonett


 I ripped a CD to MP3 190 kbs, and using ordinary classical music with no cymbals or any simmilar high frequencies, I must say that the difference didn't jump out at me. (This was using my computer rig.) I would say it's less of a difference than I heard in the cable blind tests I did last week. I know that seems paradoxical, but who knows? Anyway putting it on my main rig may be more revealing.


----------



## Berlioz

Quote:


  Originally Posted by *Earwax* /img/forum/go_quote.gif 
_It's interesting that higher flac bitrates didn't predict which pieces you'd have trouble ABXing. That's not at all what I would have guessed. 

 I am curious about a couple of things. Can you describe the type of difference you heard between the Shostakovitch tracks? How long did it take you to do all those trials? Was this all done in one day or spread out over time?_

 

The differences that I heard were mostly in depth of sound and, I guess, realism. On the less "active" sections of the piece it was impossible to differentiate them. On the complex sections, I found it much easier. I believe I was 14/14 on that test before testing the slower sections of the piece. 

 Violins were the most helpful to me in determining which one sounded more real, but I really had to listen for the very end or very beginning of each note: that was where the differences were located.

 The trials took me quite a while, maybe about 3 hours or so. It couldn't be more; it might actually be a little less, but that time includes all the breaks I took, some of which were rather long. The test was done in one day. I did discover some important things about audio testing in general. 

 The more times you hear the same track, the more easily fatigued you become. Listen to the two versions too many times, and they begin to sound exactly the same, despite a minor difference you may have noticed during the first five replicates. Because of this, I had to constantly test different sections.

  Quote:


  Originally Posted by *progo* /img/forum/go_quote.gif 
_Actually, while the flac bitrates don't mean anything, they might correlate to loudness. In general, hot CDs I've ripped are usually over 1000 kbps but the classical and jazz can be 400-700 kbps. But the numbers don't mean anything in the end. Just the saved amount of disk space._

 

I'd definitely agree with that. The newer the CD is, the higher bit rate I get off of it in FLAC. The only reasonable conclusion is that the higher the bit rate, the hotter the CD is. All of my good CD's: the first pressings, classical music, recordings mastered by the artist, etc. - All of them have relatively low bit rates.

  Quote:


  Originally Posted by *Headdie* /img/forum/go_quote.gif 
_1. When you heard a difference between MP3 and FLAC, was it a continuous difference (meaning that you could hear it all time), or was it a ponctual difference (meaning that you could hear it at some places/passages only) ?

 2. I understand that MP3-320 is a very good format, but the encoder could be the weak link. It would means that encoding with WMP or LAME would not yield to bit identical MP3 files. Is that tested and confirmed ?
 Thanks,_

 

1. I could hear the difference during some small intervals only. Usually less than a second or so in length.

 2. The encoder and program definitely make a difference. At first, I tried some 320 kbps AAC files I had on iTunes. A long time ago, I had burned some CD's using iTunes into that format. 

 Since I still have the same CD's in lossless form, I compared the two. The ABX test was not required. The difference I heard was substantial enough that it was highly unlikely it was all in my head, unless I'm just crazy.


----------



## Arjisme

Quote:


  Originally Posted by *elrod-tom* /img/forum/go_quote.gif 
_I'd tend to agree. I appreciate the work you put into this, but if it's done on the typical PC headphone jack I suspect that it's not allowing a lot of details to come through that might otherwise be more revealing of differences._

 

That is certainly possible. What I'd like to know is if listening via this PC headphone jack is the typical way in which the OP listens to music. That's important because he was convinced there was an obvious difference between the two types of encoding prior to this test.


----------



## Berlioz

Quote:


  Originally Posted by *Arjisme* /img/forum/go_quote.gif 
_That is certainly possible. What I'd like to know is if listening via this PC headphone jack is the typical way in which the OP listens to music. That's important because he was convinced there was an obvious difference between the two types of encoding prior to this test._

 

Normally I don't listen to my headphones through the headphone jack. I have some _horrible_ computer speakers that shall go unnamed. When comparing FLAC files to 320kbps, I heard an audible difference from those speakers.

 That makes little sense, considering that my IEM's are vastly superior, and that one would think the better the equipment, the easier it would be to differentiate the two kinds of formats. Obviously, it was all in my head.

 Most of my listening is off of my portable player. I don't have a home setup yet, but I do commute about 4 hours a day to get to and from school, and I get to listen to music the whole way (lucky me!). All my files on that player are lossless, and it'll stay that way regardless of these test results.


----------



## ironmine

I also ABXed flac files and mp3 file once (256 and 320 kbit/s, lame). I used Tony Joe White songs and played them though my DAC to my big speakers (see my signature). At first, I couldn't tell the difference and failed the comparison test. As I listened more and more, I noticed the difference in the sound of drums and just the overall feel of the music. So, to the end of my listening test, I scored 8/10 or even 10/10. 256 kbit files were easier to tell from FLAC. 320 kbit files were harder to differentiate. 

 My friend refused to be tested when I offered it to him. He was afraid that he won't be able to maintain his status of being a mp3-hating audiophile


----------



## AtomikPi

Quote:


  Originally Posted by *mike1127* /img/forum/go_quote.gif 
_I'm going to try this kind of comparison, but not using foobar2000 because my computer rig is nowhere near as good as my Naim CDP. Instead I'm going to burn a CD, and each track will either come from an MP3 or a WAV. I'm a programmer so I'm going to write software that chooses the source for each track randomly.

 My computer rig is this:

 Audiophile 2496 digital coax out -> CAL Sigma II DAC -> (very long and sh**ty cable) -> DNA Sonett (amp)

 For CD listening I have

 Naim CD5x -> Cardas Neutral Reference -> DNA Sonett


 I ripped a CD to MP3 190 kbs, and using ordinary classical music with no cymbals or any simmilar high frequencies, I must say that the difference didn't jump out at me. (This was using my computer rig.) I would say it's less of a difference than I heard in the cable blind tests I did last week. I know that seems paradoxical, but who knows? Anyway putting it on my main rig may be more revealing._

 


 I think your setup for testing is highly non-ideal. I assume "190 kbs" means 190 CBR. You may know that VBR>ABR>CBR. Therefore, you should probably use V2 to achieve roughly the same bitrate with higher SQ. Also, you may want to consider using V0. Make sure you're using the newest version of LAME (3.98 right now). Also make sure you set your encoder to "slow mode" (if available) to maximize SQ. Odds are the difference will be even less or non-existent for the vast majority of tracks. I personally find very few tracks are non-transparent at v2.


----------



## Saintkeat

Do 320kbps mp3 files really sound better? Take the test! | NoiseAddicts music and audio blog

 try this. it should tell you how revealing your set up truly is.


----------



## Saintkeat

so VBR is on par with 320 and pretty close to lossless?


----------



## krmathis

Quote:


  Originally Posted by *Saintkeat* /img/forum/go_quote.gif 
_Do 320kbps mp3 files really sound better? Take the test! | NoiseAddicts music and audio blog

 try this. it should tell you how revealing your set up truly is._

 

About worthless without a non-lossy control sample though.


----------



## pumbaa32

Quote:


  Originally Posted by *Saintkeat* /img/forum/go_quote.gif 
_Do 320kbps mp3 files really sound better? Take the test! | NoiseAddicts music and audio blog

 try this. it should tell you how revealing your set up truly is._

 

I guessed it right after 3-5 listenings of each clip but this is a quite hard test

 As for high quality mp3 vs flac i haven't taken any test but i think i will fail alot.


----------



## Saintkeat

i think most will fail unless they have a dac paired with customs or some good headphones.

 i'm quite happy listening to alac now. once i started noticing characteristics of lossless i could spot it easily.. like how i get a lot of clarity because of the huge reduction in 'noise'. by that i mean highs and lows bleeding into each other. gotta find more lossless files to rebuild library


----------



## SP Wild

I own SE530's and they are brilliant. They should have been able to highlight how deficient the computer headphone output is and discourage you from using this methodology. Nevertheless, I couldn't differentiate 320 from lossless when I tried it blind - that was a long time ago though.


----------



## Chef

Quote:


  Originally Posted by *krmathis* /img/forum/go_quote.gif 
_About worthless without a non-lossy control sample though._

 

You can set up an ABX test however you want though, you don't have to depend on this guy's test. Just load up Foobar, and test flac against whatever bitrate you're curious of. Most people can tell on 50% of 128 kb/s mp3 samples some minor difference between it and flac. Remember though, that Foobar is not asking you which one sounds better, it's just asking for you to separate them into two categories. If you answer the test trying to say which sounds better, you might get even poorer results.

 I think, thought maybe not on this forum, it's generally pretty well known lossless are archival formats for transcoding into current lossy formats, and that 360 kb/s is extremely bloated. There are a few killer samples, maybe 1 in 10,000 that are noticeable on any level of mp3 encoding (not sure about other encoders), but for the rest of them basically no one can tell the difference between 192 kb/s and lossless on most modern encoders. These days, 160 kb/s probably isn't suffering either, though I haven't really had the opportunity to see many tests on it. 128 is definitely what everyone is fighting for now, to make transparent that is.

 If this all sounds new to you, remember you're on a forum that is always trying to bring back old, obsolete technologies, claim they sound incredible, and then promoting enough demand for a substandard product (based purely on rustic appeal) for a small business to start hiring engineers to start making products they know are inferior because they can sell them for insane prices to some yokels 
	

	
	
		
		

		
		
	


	




 It's hilarious.


 Yeah, there's merit to saying a laptop is not the ideal music player, especially if you don't do any of your music listening on your laptop, but most people here I think make a big enough fuss about the difference between lossy and lossless that this is a good way to make them realise the difference, if any, is smaller than their laptop setup is able to show. If there's no difference on the laptop, that doesn't exactly prove there's a difference on your rig.



 PS: He did do a lossless vs mp3 blog. WAV is even more lossless than FLAC 
	

	
	
		
		

		
		
	


	




 I'm hoping to start a craze on this forum where people start getting elitist about their untaggable WAV files and refuse to use FLAC for SQ reasons. Sound Test: Difference between Wav vs MP3 | NoiseAddicts music and audio blog


----------



## xnor

It indeed is very, very hard to pick out differences between lossless and 320 kbps mp3's and for most files it is impossible to tell A from B.


----------



## krmathis

Quote:


  Originally Posted by *Chef* /img/forum/go_quote.gif 
_You can set up an ABX test however you want though, you don't have to depend on this guy's test._

 

Sure. But this trhead is about FLAC vs. MP3, not MP3 at a certain bitrate vs. MP3 at different bitrate.

  Quote:


 WAV is even more lossless than FLAC 
	

	
	
		
		

		
		
	


	



 

Totally wrong!
 WAV (the audio file format) can contain lossy audio, like MP3, GSM, and others. While FLAC is always 100% lossless to the source.


----------



## Saintkeat

thanks chef! i can tell the difference between lossless and mp3 direct from ipods but once i amp it i can't tell the difference anymore. so i guess sometimes it makes a difference sometimes it doesn't. and since i can afford the space for lossless now, i'll get whatever i can in lossless, and keep whatever mp3 files i have that i can't get lossless for. no biggie =)


----------



## xnor

Quote:


  Originally Posted by *krmathis* /img/forum/go_quote.gif 
_WAV (the audio file format) can contain lossy audio, like MP3, GSM, and others. While FLAC is always 100% lossless to the source._

 

That doesn't make sense. You can encode the data of an mp3 file as wave or flac (actually, people who do that deserve to be punished) and there's no difference. The format cannot dictate the source to be lossless.


 And it should strike you like lightning that the sentence "WAV is even more lossless than FLAC" can only be a joke.


----------



## krmathis

Quote:


  Originally Posted by *xnor* /img/forum/go_quote.gif 
_That doesn't make sense. You can encode the data of an mp3 file as wave or flac (actually, people who do that deserve to be punished) and there's no difference. The format cannot dictate the source to be lossless._

 

Does not make sense?
 WAV is an audio container (not a codec). It can contain both lossy encoded audio data (like MP3 and GSM) or uncompressed audio data (like PCM). So if the WAV file is lossy or not all depend on the encoder user or not used on the audio data stored within it.

 FLAC on the other hand always encode the input audio stream without any data loss.

 Do not believe me? Please read:
WAV - Wikipedia, the free encyclopedia
Free Lossless Audio Codec - Wikipedia, the free encyclopedia

  Quote:


 And it should strike you like lightning that the sentence "WAV is even more lossless than FLAC" can only be a joke. 
 

Sure! Cause it makes no sense at all...


----------



## xnor

Afaik, WAV is used commonly for PCM data only (and I was talking about the encoding) in which case it's not an apples / oranges comparison like it is in the case of comparing WAV as a container with FLAC.

 And any way, it doesn't matter if I take an mp3 and encode it into PCM WAV or "embed" it directly into a WAV file because I can also encode the same mp3 with FLAC.

 Clearly this is a semantics thingy. By saying "FLAC is always 100% lossless to the source" we're talking about the encoding, which you cannot compare to WAV as container format.


----------



## Chef

Quote:


 the most common WAV format contains uncompressed audio in the linear pulse code modulation (LPCM) format. The standard audio file format for CDs, for example, is LPCM-encoded, containing two channels of 44,100 samples per second, 16 bits per sample. Since LPCM uses an uncompressed storage method which keeps all the samples of an audio track, professional users or audio experts may use the WAV format for maximum audio quality. 
 

WAV (with LPCM) is what people editing and working with audio use because it runs natively and thus is easy to work with. It's also about two or three times as big as FLAC files. When you run a program like EAC, it extracts the PCM files from the CD as WAV, and then if you chosen to, transcodes to a compressed format, such as FLAC or MP3.

 The joke was that it's completely ludicrous to use for regular listening, because the files are gigantic and don't support meta-data, afaik. The idea is that for some reason people think bigger, more annoying files = higher quality, so introducing people to uncompressed music on head-fi sounds like a great experiment to see how dumb people get. Every player supports WAV, so it might seem like a dream come true to some people


----------



## b0dhi

You should tell them that adding 200mb of (redundant) zero-byte padding to the header of the WAV file will give the DAC a chance to "calm down" before playing the audio, and that it results in audio nirvana.

 Do it for the children.


----------



## Crazy*Carl

I do what I can to get lossless files, but I am usually satisfied with VBR V2 and up.

 320 may be almost as good as lossless, but with terabyte hard drives and 5Mbit+ Internet, why would you compress music at all (except for portables)


----------



## Ham Sandwich

Quote:


  Originally Posted by *Crazy*Carl* /img/forum/go_quote.gif 
_320 may be almost as good as lossless, but with terabyte hard drives and 5Mbit+ Internet, why would you compress music at all (except for portables)_

 

Because the lossless world has not (and likely will not ever) standardized on a format that is supported by all portables, all desktop players, and all hardware devices. MP3 works almost everywhere on almost every device and almost every player. The same cannot be said of any of the lossless formats (FLAC, ALAC, WMA Lossless, etc).

 Until the lossless world gets its act together and supports interoperability and broad support it will remain far more convenient to just use MP3. The audio geeks will do lossless, but regular people are going to continue to use MP3 for the convenience of having it just work.


----------



## krmathis

Quote:


  Originally Posted by *Crazy*Carl* /img/forum/go_quote.gif 
_320 may be almost as good as lossless, but with terabyte hard drives and 5Mbit+ Internet, why would you compress music at all (except for portables)_

 

To add features not present in the non-compressed audio formats, like full support for metadata (tags) and artwork (album art).
 Plus the fact that you can use compression to save space without loosing audio data, as long as you use lossless compression (like FLAC, ALAC, WavPack, ..).


----------



## DrBenway

Quote:


  Originally Posted by *Ham Sandwich* /img/forum/go_quote.gif 
_Because the lossless world has not (and likely will not ever) standardized on a format that is supported by all portables, all desktop players, and all hardware devices. MP3 works almost everywhere on almost every device and almost every player. The same cannot be said of any of the lossless formats (FLAC, ALAC, WMA Lossless, etc).

 Until the lossless world gets its act together and supports interoperability and broad support it will remain far more convenient to just use MP3. The audio geeks will do lossless, but regular people are going to continue to use MP3 for the convenience of having it just work._

 

It's true that lossless is not standardized; it's also true that MP3s "just work" on a very wide variety of devices. But I chose my Clip+ in part because it does play FLACs, and FLAC support seems to me to be turning up on more and more devices. Given that it is free and encoders are widely available, I feel pretty secure with the format. More secure, in any event, than I would be with a proprietary format that could disappear from the marketplace (ATRAC, anyone?)


----------



## Crazy*Carl

Quote:


  Originally Posted by *krmathis* /img/forum/go_quote.gif 
_To add features not present in the non-compressed audio formats, like full support for metadata (tags) and artwork (album art).
 Plus the fact that you can use compression to save space without loosing audio data, as long as you use lossless compression (like FLAC, ALAC, WavPack, ..)._

 

Sorry, I said it wrong. What I meant to say is why would you not go with lossless (since FLAC and ALAC are still compressed)


----------



## Young Spade

Wow interesting. And thanks a lot for putting so much time and effort into your studies. If I knew of a great audio converter I would use it and convert my FLAC files to 320kbs :/


----------



## Prog Rock Man

Quote:


  Originally Posted by *Berlioz* /img/forum/go_quote.gif 
_
 ...... Even with Symphony No.1, I was focusing on the music so ****ing hard, to the point where it wasn't even enjoyable anymore........ 
_

 

Get write up Berlioz. I know what you mean about getting fed up trying to hear differences. I have found if I just chill and go for the overall 'feeling' of the music, rather than trying to listen for tiny differences, it is easier and I have been more accurate in picking out the differences.


----------



## xnor

Quote:


  Originally Posted by *jinp6301* /img/forum/go_quote.gif 
_You might want to try this with a better source. With the headphone jack off my laptop, I only get around 50% correct. With a high end dac, I can get about 90+% right most of the time

 ^based on previous tests I've done_

 

And you know that 90% is not statistically significant, or in other words you were not able to reliably tell one from the other, right?


----------



## plonter

first of all,thanks to the op for all the efforts invested in this test.
 I think that most portable players these days do support some kind of lossless format. I never did a direct comparison of mp3 vs flac (or any other lossless format) but since i got into this hobby a few years ago,it was always lossless for me, cd's on my home system and flac on my portable.


----------



## Saintkeat

OK i didn't want to start a new thread so i'm just gonna ask it here. 

 I was getting worried that perhaps it is possible some of my "flac" files were mp3 to begin with. so I tried converting an mp3 file to ALAC and it works. the file was identified as apple lossless and the file size was almost the same as the proper alac file. 

 question is. how do you tell if the files you've got were once mp3 and then converted to flac?


----------



## Chef

Uh, you rip them yourself ^^ I guess you could try Accurate rip checking the file but I don't have a clue if that would be effective. If you're buying online, just make sure you trust the outlet. There's aren't really many reasons someone would do that, except if they were horrendously ignorant, or just trying to be jerks. I've heard about certain online sellers sabotaging their lower bitrates though to encourage people to pay more for 'premium' files.


----------



## plonter

I have a question, I don't want to change this thread's subject but I also don't want to open a new thread just for this,so here's a quicky:
 is there a better lossy format than mp3 320 soundwise? I am considering moving to loosy format because my sansa clip+ runs out of battery too quickly,so I want to know what is my best option.
 what is the absolutely highest quality lossy format (and in what bitrate) that exist in the market? what would be the lossless of the lossy?


----------



## Rip N' Burn

Quote:


  Originally Posted by *plonter* /img/forum/go_quote.gif 
_I have a question, I don't want to change this thread's subject but I also don't want to open a new thread just for this,so here's a quicky:
 is there a better lossy format than mp3 320 soundwise? I am considering moving to loosy format because my sansa clip+ runs out of battery too quickly,so I want to know what is my best option.
 what is the absolutely highest quality lossy format (and in what bitrate) that exist in the market?_

 

Ogg Vorbis -q10 500kbps
 Nero AAC -q 1.0 ~400kbps


----------



## plonter

Quote:


  Originally Posted by *Rip N' Burn* /img/forum/go_quote.gif 
_Ogg Vorbis -q10 500kbps
 Nero AAC -q 1.0 ~400kbps_

 

thanks. I also read in other places that ogg supposed to be better than mp3.


----------



## Crazy*Carl

Quote:


  Originally Posted by *plonter* /img/forum/go_quote.gif 
_I have a question, I don't want to change this thread's subject but I also don't want to open a new thread just for this,so here's a quicky:
 is there a better lossy format than mp3 320 soundwise? I am considering moving to loosy format because my sansa clip+ runs out of battery too quickly,so I want to know what is my best option.
 what is the absolutely highest quality lossy format (and in what bitrate) that exist in the market? what would be the lossless of the lossy?_

 

MP3 LAME VBR V2 usual in the neighborhood of 200kbps

 But listen to your ears. These supposed things you read are probably minuscule differences. Especially with a portable player and headphones, I wouldn't expect to hear much of anything different.


----------



## Tyll Hertsens

Big props to the OP, I've been saying it for years.

 I will add though, in agreement with a few others, that if you use better gear, you will be able to discriminate better.

 Also, when I listen for differences like this, I don't really listen hard to the music. What I do is listen in a relaxed way, and observe carefully my feeling w/regard to the music. In other words, I just let myself experience the music, and I carefully evaluate my response to it. I don't carefully evaluate the music itself.

 Sounds odd, I know, but give it a try.


----------



## nick_charles

Quote:


  Originally Posted by *Tyll Hertsens* /img/forum/go_quote.gif 
_I will add though, in agreement with a few others, that if you use better gear, you will be able to discriminate better._

 

This being the "critical" subforum I would like to challenge this. 

 Whilst I agree that any really crap system will be so riddled with noise and distortion so as to make any discrimination task difficult to impossible, beyond a certain point of technical competency I would imagine it is human abilities that are the limiting (and well known after 100 years of psychophysics research) factors.

 To the best of my knowledge there has never been a single set of controlled DBTs that has ever tested your assertion, i.e that has given listeners the same controlled blind discrimination task and changed only the test kit and described the results, I am open to being proven wrong !

 Anecdotes need not apply


----------



## Tyll Hertsens

Quote:


  Originally Posted by *nick_charles* /img/forum/go_quote.gif 
_Whilst I agree that any really crap system will be so riddled with noise and distortion so as to make any discrimination task difficult to impossible, beyond a certain point of technical competency I would imagine it is human abilities that are the limiting factors_

 

See? We agree.


----------



## Young Spade

Quote:


  Originally Posted by *Tyll Hertsens* /img/forum/go_quote.gif 
_Big props to the OP, I've been saying it for years.

 I will add though, in agreement with a few others, that if you use better gear, you will be able to discriminate better.

 Also, when I listen for differences like this, I don't really listen hard to the music. What I do is listen in a relaxed way, and observe carefully my feeling w/regard to the music. In other words, I just let myself experience the music, and I carefully evaluate my response to it. I don't carefully evaluate the music itself.

 Sounds odd, I know, but give it a try._

 

I did that yesterday. Just laid down and thought about how the music made me feel. I don't do it often but I got goosebumps 5 times in one 18 minute song.


----------



## plonter

Quote:


  Originally Posted by *Young Spade* /img/forum/go_quote.gif 
_I did that yesterday. Just laid down and thought about how the music made me feel. I don't do it often but I got goosebumps 5 times in one 18 minute song._

 

this is the best way to listen to music, absolutely the opposite of analysing and consertrating. you need to really get in the music in order to get pure nirvana,but it doesn't happen often.

 about lossless vs lossy,I DO believe everybody that say that hear a difference. I never did a direct comparison, but I think that the psycological aspect is not less important here than the physical aspect. what can we do?? we are human beings...and humans are effected by psycological things.
 I will keep listening to lossless because i know that i am not compromising the music in any way, no matter if i can hear it or not,and also it maked me enjoy the music more...the same story as with burn-in.

 it is like playing chess..there is a lot of books written about psychology in chess. where is chess and where is psychology?? chess is a 101$ mathematical thing...just like music, but because who plays the game is a human being, than the psychology starts to take place.


----------



## RonaldDumsfeld

I was disappointed to discover that I too am unable to tell the difference between a lossless file and a good quality lossy equivalent. Neither can my friends. Although not all of them have got round to accepting it yet. 

 Like the OP I used foobar +foo - ABX, FLAC vs MP3 VBR V0. 

 I cheered up a bit when I found out that no one else can either. Not in a properly conducted test. So I believe. If anyone knows to the contrary I'd be pleased to hear.

 I still try to keep lossless masters or vinyl of music whenever possible though. Like the poster above. It just feels better that way and you never know......


----------



## Young Spade

^ *Shakes your hand, nods head, then walks away*
 That's what I'm talking about.

 EDIT: RonaldDumsfeld: Hey man, just wanting to ask questions; when people AB or compare formats with themselves and friends I'm always urged to ask questions 
	

	
	
		
		

		
		
	


	




 Do the other people that you tested the files with have any experience in higher quality audio? 
 And what type of equipment did you use? 

 One of these days I'm going to do a blind test with friends to see if they can hear. The last time I did some sort of test was with my T51, RSA Tomahawk and UE Triple.Fi 10s. My friend couldn't' tell the difference between the LO to RSA or straight from the HO out.

 I have a feeling this isn't going to end well lol


----------



## Feanor

I dont have the best dac or anything and I listen through a tube amp, but for what its worth, if I recall correctly (its been over 4 years since that test), I couldn't really notice the difference between lame 320 vbr and lossless in most of my music either. 

 I recall one part of a song where I could though, multiple very sudden high notes crowding up on each other. There I got it almost every time.

 That said, I prefer to store my music in flac simply because space is cheap and flac is transcodable to smaller bitrates while a 320 mp3 would loose more quality with each transcode. Plus I dream of a perfect amp and dac and headphone where it WILL make a difference 
	

	
	
		
		

		
		
	


	




...


----------



## RonaldDumsfeld

Quote:


 Hey man, just wanting to ask questions; 
 

Fire away I've got nothing to hide. 

 Like I said. I was amazed and disappointed when I first found out because I was a fully paid up member of the 'of course I can tell, it's easy' brigade. 

  Quote:


 Do the other people that you tested the files with have any experience in higher quality audio? 
 

Yes. We are all rather old and have been collecting hi-fi gear since the 7Ts.

 One guy in particular has a Linn Sonn, 3xNaim, Linn Iso system from the late 8Ts. He has failed twice and won't do the test any more taking the attitude already expressed that he cannot tell when he is concentrating but can when relaxed. 

 Most have Linn Sons or Regas and middle market (~£1,000) amp and speakers except 1 guy who is an engineer and has known this all along. He wires the cheapest gear he can find together with regular house cabling.

  Quote:


 And what type of equipment did you use? 
 

Foobar with LAME (3.98.2), MOTU UltraLite (same converters as RME) Soundcraft Spirit M4 and either Rogers A75 2 / Mordaunt-Short Pageant 2 (decent) or Kenwood KR-V7040 / TDL RTL 3 (loud) or M-Audio AV40 (desktop nfm).

 I've also done the same experiment with mostly the same people using an MP3 CBR 320 and .WAV downloaded from Beatport, a commercial CD .WAV and a 12" vinyl original. All of the same tune. 

 Variety of cartridges inc. AT 440MLa, Stanton 500AL, Stanton TrackMaster & MC Denon DL-103 with Cambridge Audio 640P (which I'm not happy with btw got any suggestions that will not break the bank?).

 The vinyl is easy to tell apart, of course, but not the digital files.

 I think you might also be very surprised, I was, to find how difficult it is to tell the difference between a vinyl played direct and the same signal passed into the box via the MOTU and direct monitored out again after recording (Audacity).


----------



## nick_charles

Quote:


  Originally Posted by *RonaldDumsfeld* /img/forum/go_quote.gif 
_I
 I cheered up a bit when I found out that no one else can either. Not in a properly conducted test. So I believe. If anyone knows to the contrary I'd be pleased to hear._

 

Generally I cannot tell MP3 VBR 0 from lossless. However I was able to DBT them on one specific track to a P < 0.05 level (14/17 or something like that) , on the track in question the original file was heavily saturated , peaking frequently at 0db and the endoding appeared to add a small extra amount of distortion, the telling them apart was really hard and I had to focus on one small (5 second ) segment and go back and forwards between the two of them many many times. Nevertheless I managed it. In casual listening I would have been unable to tell them apart only with extreme concentration and mind-numbing repetition and only with one tricky sample...


----------



## Young Spade

^Wow alright man thanks for sharing 
	

	
	
		
		

		
			




 I really should AB some files one day. All of the current stuff I get now is ripped to FLAC and used so I'm probably just gonna wait until I'm forced to convert and see what happens


----------



## Chef

I know you probably just mistyped, but please know the difference between AB and ABX testing. The former is not useful in this context.

 I think you'll find if you're ABX testing a random song for the first time, you'll probably even have trouble comparing 128 vs flac. You will be surprised how hard you need to listen, and how much experience it takes knowing where to listen. There are some rare files where it is very obvious, but on the whole lossy encodings are very good and you will find yourself confused.


----------



## Young Spade

^Thanks and yea I meant ABX 
	

	
	
		
		

		
		
	


	




 It was getting a little too late for me


----------



## Horken

chef said:


> I think you'll find if you're ABX testing a random song for the first time, you'll probably even have trouble comparing 128 vs flac.


 
  
 That describes me pretty well! I just tried my first ABX test using foobar2000, my desktop PC (Gigabyte motherboard with good audio), and my Etymotic HF5's. I took some source flac files and converted them to MP3 with various Lame presets. Then I ran them through the foo_abx.
  
 At first, I tried using quality V2 with Radiohead's "15 Step" from In Rainbows. It's a very noisy track with a lot of range. It came out to an average of 187 kbps.  I didn't even bother making any guesses because I couldn't find _anything_ by which to differentiate A from B, and I tried very hard in a number of passages. So, I tried V5 instead (avg 139 kbps). Still nothing. At this point I really wanted to hear a difference so I made a V8 version (avg 98 kbps) and tried again. OK, this time it was obvious right at the start and I got 100% correct after 5 guesses and stopped because it was so obvious.
  
 Then I tried "Satellite Mind" by Metric. It's dense, but not noisy like the last track. I figured it would be harder to tell the difference, and it was, but I had the exact same results: I couldn't tell the difference until V8, but then it was 100%. However, it wasn't nearly as obvious. I had to listen to a couple passages and could only really hear a difference in the cymbals.
  
 I thought it was going to be easier to tell the difference. Boy was I wrong! Lame is a really good encoder, and I have 40 year old ears. Despite having very accurate headphones, which are super crisp and clear, it was a struggle to find any difference until V8!


----------



## Eternal Schism

I've used the same method on Foobar myself. I've done it with Reflections by Tool (the early drumming parts), Helms Deep from the Lord of the Rings Trilogy Soundtrack, Money by Pink Floyd and Bust a Move (quick synth arpeggios) by Infected Mushroom. I couldn't _reliably_ discern the differences between MP3 320 or Flac files using a HD800 powered by a NFB 28 (S/E output).


----------

