# Lossless vs 128kbps mp3 vs 320kbps mp3 blind test



## chewy4

*Statistics have now been gathered, but I will leave the files in case anyone wants to ABX them. The key and statistics are in a spoiler at the bottom of this post.*
   
  Hey everyone,
   
  I converted and volume matched some files for a blind test. I'd appreciate your participation, but please follow these guidelines so we can eliminate some unwanted variables entering the test:
1. Put your answers in SPOILERS. Do not read anyone's answers before listening and responding yourself.
   This is so nobodies answer is influenced by anyone else.
  2. This is a LISTENING test. Not a looking test. No peaking. If you peak you are a bad person.
  3.  Don't do any bizarre equalization or any other fancy tricks. Listen to them like you would normally listen to your music.
   
  When you listen to each group I'd like for you to answer the following:
  - How do you rank each of the files?
  - Were you guessing?
     - If you weren't guessing, what differences did you hear?
     - How difficult was it?
   
  On with the test! 
   
  For each group, one file is lossless, one is a 128kbps mp3, another is a 320kbps mp3. They are randomly scrambled, so one group's "A" might not be the same type as the other group's "A".
   
   
  Group 1 - "Mighty River" by Railroad Earth
  Zipped files: http://www.mediafire.com/download.php?u44qxs3j43azgo7


Spoiler: Individual%20files(for%20those%20paranoid%20about%20zip%20compression%20%3A)%20)



 
  Source: http://www.mediafire.com/download.php?apisoaeabmjpbmn
  A: http://www.mediafire.com/download.php?kwoi8pi8hplea82
 B: http://www.mediafire.com/download.php?kxjbujfi135plpy
 C: http://www.mediafire.com/download.php?wckkgtuk4t3l5c4
   


   
   
  File 2 - "Fahrenheit Fair Enough" by Telefon Tel Aviv
  Zipped files: http://www.mediafire.com/download.php?hpi9n4d2i3544di


Spoiler: Individual%20files



 
  Source: http://www.mediafire.com/download.php?91i462ih79gnwae
  A: http://www.mediafire.com/download.php?j5g3dr31vvzasvw
  B: http://www.mediafire.com/download.php?c521p8rx2myu1c3
  C: http://www.mediafire.com/download.php?5clv3q643do8o2e
   


   
  File 3 - "Falling Awake" by Shpongle
  Zipped files: http://www.mediafire.com/download.php?aib6uc1hw1ng73o


Spoiler: Individual%20files



 
  Source: http://www.mediafire.com/download.php?i9syghdbn62dfhe
 A: http://www.mediafire.com/download.php?ebfuuvya2x3y74q
  B: http://www.mediafire.com/download.php?f7006wt04kc6w4f
  C: http://www.mediafire.com/download.php?mz64z6d5i84tc7j
   


   
   
   
  While I'd appreciate it if you listened to all three groups, if you don't have the time feel free to only give your feedback on one or two. I tried to make the groups pretty different from each other.
   
  Happy Listening!
   
   


Spoiler: Warning%3A%20Spoiler!%20Key%20and%20Statistics!



*KEY*
*Group 1:*

 A.320 B.Lossless C.128 *Group 2:*
 A.Lossless 
 [size=inherit]B.320[/size] 
 [size=inherit]C.320[/size] *Group 3:*
 A.128 
 B.320 
 C.Lossless
 
*STATISTICS*
*Group 1*
  A(320kbps): 4/11(36%) were correct. Only 1 person said they were guessing, and they guessed right.
   
  B(Lossless): 3/9(33%) were correct. 1 person said they guessed, and they guessed wrong. One person said that it was the opposite bitrate.
   
  C(128kbps): 7/10(70%) were correct. Two people said it was the opposite bitrate. 
   
*Group 2*
  A(Lossless):3/9(33%) were correct.  Two of the correct answers said they were guessing. Two people said it was the opposite bitrate.
   
  B(320kbps):1/8(12.5%) were correct. The person who answered correctly said it was a guess. Two of the wrong answers were said to be guesses.
   
  C(320kbps): 4/9(44.4%) were correct. Three correct answers were said to be guesses. One wrong answer was said to be a guess.
   
*Group 3*
  A(128kbps): 7/11(63.6%) were correct. One person said it was the opposite bitrate. 
   
  B(320kbps): 5/10(50%) were correct.

 C(lossless): 5/10(50%) were correct. Four people said it was the opposite bitrate.
   
   
  One answer for each group said they couldn't tell a difference.
   
  I also posted this on another forum, only a few people answered though.
  One person guessed all of group 1 wrong, and they were unsure about their answers.
  Another person guessed all of group 3 right, and they described what differences they heard.
  One person said they couldn't tell a difference.
  One person said they didn't give a schiit.
   
   
   
*Conclusion*
   
  These are some fairly interesting results. Group 2 was a little weird, and not completely fair because I unintentionally lied about there being 3 types of file. But the largest amount of people said that their answers were guesses for this one, so hooray for you folks.
   
  I found group 3 to be the most interesting. Almost half the people who ranked it said that the lossless file was 128kbps! At the same time, it was just behind the first group in the amount of people guessing correctly on the 128kbps.
   
  So, what does any of this actually prove? Absolutely nothing statistically speaking, although there did seem to be a higher percentage of people guessing the 128kbps file right. Hopefully it gave some people a better idea of what compression really sounds like.


----------



## musical-kage

Spoiler: Warning%3A%20Spoiler!



I'm only listening to one right now but will edit later. I need to go to sleep really, haha.
 My choice is 'Fahrenheit Fair Enough' by Telefon Tel Aviv:

 A = 128kbps
 B = 320kbps (guess)
 C = Lossless (guess)


 *2^*
   
  Just from initial listens. I think I'm probably wrong on all counts... Listened a few times again and changed my perceptions based on certain parts of the track.
   
  Really struggled to be honest.
   
*3:*


Spoiler: Warning%3A%20Spoiler!



"Falling Awake" by Shpongle.
 Damn you not providing us with the higher treble parts 

 A = Lossless
 B = 128kbps
 C = 320kbps


   
*1:*


Spoiler: Warning%3A%20Spoiler!



"Mighty River" by Railroad Earth.
 I just hope Winamp is opening them in order of selection

 A = 320kbps?
 B = Lossless
 C = 128kbps


   
  EDIT:
   
  Mine were all mixed up and its hard to correct too, so I've put numbers where each one is


----------



## KamijoIsMyHero

I already know I can't tell the difference between lossless and 320 so I will just pick out what I think was the 128
   
   


Spoiler: Warning%3A%20Spoiler!



128:
  1A
  2C first one that stood out IDK why
  more acoustic guitars  -.- 3A
   


   Mann, you just had to use 2 acoustic guitar and an electronics track, 99% of my music have none of that, its pretty difficult since I am not used to it at all
   
  I didn't double check my picks, I think my ears have grown too accustomed to try and re-examine the tracks


----------



## Strangelove424

Thanks for getting the files together Chewy, I enjoyed the samples and I liked the 1st one so much I looked up Railroad Earth on MOG - very good stuff. Pass or fail, I'm really happy to have some new music to try out. I love folk!
   


Spoiler: Warning%3A%20Spoiler!



 
Group1 [size=12.0pt][/size]
A>B>C, decided based on acoustic reverb and clarity of strumming
Group 2
B>A>C, decided based on airiness and richness[size=12.0pt][/size]
Group 3
B>A>C, decided on acoustic reverb and clarity of strumming[size=12.0pt][/size]


----------



## cer

Spoiler: Warning%3A%20Spoiler!



Only tried the first song - "Mighty River". At first just casually listened to the files in foobar. They seemed reasonably similar, especially A and B. But something seemed to be wrong with C. So I ABX-d B and C. 10/10 and pretty convinced of the difference. Not a massive difference, but detectable when paying attention. 
  I used headphones - Beyer DT880 and E-MU 0202 USB sound card.
  So, C is definitely 128 kbit/s. It just has that familiar impureness of transients, some sort of wrong echo around them. I won't bother with A vs B.
  So it seems, that 128 kbit/s still is different from the original and I would say, in a detrimental way. I wouldn't choose C as the one I would want to listen[size=1em].[/size]
   
  [size=1em]So the ABX result here:[/size]
   foo_abx 1.3.4 report
  foobar2000 v1.1.15
  2013/01/16 10:25:42
   
  File A: C:\Documents and Settings\Owner\My Documents\1_B.wav
  File B: C:\Documents and Settings\Owner\My Documents\1_C.wav
   
  10:25:42 : Test started.
  10:26:20 : 01/01  50.0%
  10:27:21 : 02/02  25.0%
  10:28:14 : 03/03  12.5%
  10:28:52 : 04/04  6.3%
  10:29:55 : 05/05  3.1%
  10:30:37 : 06/06  1.6%
  10:31:23 : 07/07  0.8%
  10:31:58 : 08/08  0.4%
  10:32:11 : 09/09  0.2%
  10:32:39 : 10/10  0.1%
  10:43:41 : Test finished.
   
   ---------- 
  Total: 10/10 (0.1%)


----------



## musical-kage

Yeah thanks for this Chewy, it was really enlightening actually, and some of the best test material I've heard!
  I listened to the whole of Shpongle's album this morning. Couldn't find the Telefon Tel Aviv track on Spotify/Deezer or Rdio though


----------



## chewy4

No problem, glad you guys liked the files. Hopefully enough people take this so that I don't have to wait too long to put the results up.
   
  Sorry I wasn't able to put any metal or the likes in there for you KamijolsMyHero, the closest stuff I had was either not lossless or of not that great recording quality. I mentioned in another thread that I wanted to avoid stuff with any hiss in the background and that's hard to avoid when distorted guitars come into play.
   



musical-kage said:


> Couldn't find the Telefon Tel Aviv track on Spotify/Deezer or Rdio though


   


  The remaining member of Telefon Tel Aviv hasn't exactly been trying to get their stuff out there since the other guy died, so you'll probably have trouble finding it on any of those streaming services(some of their stuff is on lastfm but not of that album, which is completely different than all their others). Good luck finding it lossless legally too, I ended up buying the CD for about $30 used off discogs. I think most is on YouTube though if you want to listen to it.


----------



## musical-kage

Quote: 





chewy4 said:


> No problem, glad you guys liked the files. Hopefully enough people take this so that I don't have to wait too long to put the results up.
> 
> Sorry I wasn't able to put any metal or the likes in there for you KamijolsMyHero, the closest stuff I had was either not lossless or of not that great recording quality. I mentioned in another thread that I wanted to avoid stuff with any hiss in the background and that's hard to avoid when distorted guitars come into play.
> 
> ...


 

 I should have made sure Winamp put the songs in order. With one of the sets, it kept the X1,X2,X3 in Winamp. The other two opened up with the file names so you couldn't see if they were A,B,C etc.
   
  Ah, I know  Even their website doesn't go anywhere as far as I can tell. It just has the announcement and then, it won't go any farther.


----------



## chewy4

You should be able to have it display the actual file name instead of the title IIRC, but I should probably wipe the tags from the files. I'll do that and re-upload tonight so nobody else runs into that problem.


----------



## badhomaks

Spoiler: No%20point%20in%20peeking%2C%20you're%20just%20lying%20to%20yourself



For the first one is seems to be A>B>C
  C had this weird stereo effect like the song was alternating between left and right speakers
  For 2nd one after a few mins I just gave up. The loud squeaking hurt my ears.


----------



## Kaffeemann

Spoiler: Warning%3A%20Spoiler!



Could not hear any difference in group 1 and 2.


----------



## Strangelove424

I can't seem to submit a single post anymore without a typo. In my answer box, the second Group 2 should be labeled Group 3. I don't want to edit my original post though. I also neglected to write down my observation that it was a difficult test, which confirms my belief that well-mastered music sounds good no matter what, and compressed junk sounds bad no matter what.


----------



## chewy4

Quote: 





strangelove424 said:


> I can't seem to submit a single post anymore without a typo. In my answer box, the second Group 2 should be labeled Group 3. I don't want to edit my original post though. I also neglected to write down my observation that it was a difficult test, which confirms my belief that well-mastered music sounds good no matter what, and compressed junk sounds bad no matter what.


 
  It would probably help if you edited since I'll forget about that when it comes time to collect the statistics. 
  Don't worry I'll trust that you didn't change your answers. 
	

	
	
		
		

		
			




   
  I threw the groups in their own zip files so people don't have to click a million links. Probably should have done that in the first place.


----------



## sonitus mirus

Wow!  This is absolutely wonderful.  Thank you so much for making this comparison available.  I love your song choices, too.  If you ever see a post of mine making a recommendation for any music that you might not have heard before, jump on it.  I think our tastes are very similar, or at least eclectic enough to enjoy a little about almost everything.
   
   
  I ranked each file within its group from worst sounding to best sounding.  So anything with a 3 is my guess at the lossless file, 2 would be the 320kbps mp3, and 1 is my guess at the 128kbps mp3.
   
   


Spoiler: Warning%3A%20Spoiler!



 
  Group 1
   
  A-3
  B-1
  C-2
   
  Group 2
   
  A-1
  B-3
  C-2
   
  Group 3
   
  A-1
  B-3
  C-2


----------



## Strangelove424

Quote: 





chewy4 said:


> It would probably help if you edited since I'll forget about that when it comes time to collect the statistics.
> Don't worry I'll trust that you didn't change your answers.


 
  Ok, I edited it.


----------



## KamijoIsMyHero

Quote: 





chewy4 said:


> No problem, glad you guys liked the files. Hopefully enough people take this so that I don't have to wait too long to put the results up.
> 
> Sorry I wasn't able to put any metal or the likes in there for you KamijolsMyHero, the closest stuff I had was either not lossless or of not that great recording quality. I mentioned in another thread that I wanted to avoid stuff with any hiss in the background and that's hard to avoid when distorted guitars come into play.
> 
> ...


 
   
  can you provide an example(e.g. link on youtube) of hiss that you hear or describe it further, I don't hear a hiss at all, BTW the distortion is a characteristic of metal anyway, same way the acoustic guitar is for that genre you gave out


----------



## chewy4

Quote: 





kamijoismyhero said:


> can you provide an example(e.g. link on youtube) of hiss that you hear or describe it further, I don't hear a hiss at all, BTW the distortion is a characteristic of metal anyway, same way the acoustic guitar is for that genre you gave out


 
  I'll post a link on lunch break, but a lot of times you have to have good gear to hear it. Mine is particularly unforgiving of it. It's also generally inaudible unless there is close to silence so it's mostly a nonissue if I chose the right parts but I just played it safe and chose tracks that I couldn't hear any of it in.
   
  I might have been wrong in saying that it is hard to avoid when distorted guitar comes into play, that might just be my particular collection. The stuff I have with the loudest hiss is actually acoustic.
   
  EDIT: Here's an example of something with very noticable hiss(warning: album cover contains nudity but it's art so it's cool): http://www.youtube.com/watch?v=iCxNVYF_7hA
   
  Generally a lot quieter than that, but I was really picky in my selections.


----------



## Maker

When there will be results ?


----------



## chewy4

Quote: 





maker said:


> When there will be results ?


 
  I was thinking in about a week, depends on how much more people participate.


----------



## bigdubs

For #3:
   
   


Spoiler: Warning%3A%20Spoiler!



 
  a. 128kbps
  b. 320kbps
  c. Lossless


----------



## longslong

Here's a stupid question, how do you format/post the spoiler bit? My google fu seems to be week


----------



## chewy4

Quote: 





longslong said:


> Here's a stupid question, how do you format/post the spoiler bit? My google fu seems to be week


 
  In the reply box there is a thing that looks like a black speech box. It's in between the double quotes and text alignment boxes.


----------



## liuk

Spoiler: Group%201



A: 128
  B: 320, guessed
  C: lossless, guessed
   
  could not tell a thing between B and C honestly. Just "feels" B is a bit less precise than C but have no idea in all honesty. A felt with less crispy highs than the source.
   


   


Spoiler: Group%202



A: Lossless guessed
  B: 128
  C: 320 guessed
  after the 16th second A felt more confused, with the highest frequencies less sharp and "nippy". Again, lossless and 320, I have no real idea.
   
   


   


Spoiler: Group%203



A: 320, guessed
  B: lossless, guessed
  C: 128
   
  C felt less dynamic after the 18th second, more confused and the guitars less vibrant. for 320 and Lossless, no my friend, I have no idea at all.
   


   
  here are mine. Was not easy at all, honestly, to distinguish them. Actually, I mostly have no clue to distinguish 320 from Lossless, and actually 128 is not obvious to me also. I also suspect than when confronting the "alleged" 128 with the source, my mind was tricking me.
   
  Overall, I think that, had I bought a CD, or downloaded a bad rip, I would have hardly known.
   
  Also I do not really agree with the chosen music. Some more complex, dynamic piece (e.g. classical, Dvorjak's 9th maybe) would have probably been better.
   
  Great initiative, thought. Thanks for that. Will be very interested in the results.


----------



## jiminy

just had a go, but how do you hide your answers, also agree with the last post, with that kind of music is it not going to be more difficult, and should it not be the same track for each?


----------



## jiminy

Spoiler: Warning%3A%20Spoiler!%20group%20a



in order a, b then c from 128 to lossless, think c was lossless because it sounded the quietest


  very difficult, listened to the first bit of 2 and could not differenciate
   
  sorry, found answer to spoiler box when i started to read ealier posts
   


Spoiler: Warning%3A%20Spoiler!



i have had a go at 3 as i seem to agree on the first, i didnt look at the answers to 3
 A was 320
 b was lossless, hated the second bit
 c was 128

 if im correct with both of mine which i doudt. lossless is the easiest to pick as its the dullest


----------



## liuk

Quote: 





jiminy said:


> Spoiler: Warning%3A%20Spoiler!%20group%20a
> 
> 
> 
> ...


 
  I checked and you are as wrong as I was.


----------



## jiminy

Have you got the answers? Have we mistaken 128 with lossless? I do hope so


----------



## chewy4

I would have probably thrown in a classical track if I had any lossless one's, but I don't. So I just tried to keep it diverse and use well mastered stuff, hopefully showing a varying degree of difficulty between the three groups. There's no way I could have what _everyone_ wants to see with only three tracks, and I felt like any more than that would be too much. 
   
   
   


jiminy said:


> Have you got the answers? Have we mistaken 128 with lossless? I do hope so


   
  Nobody should have the answers yet but some people have given opposite answers so far.


----------



## Taowolf51

Group 1:
   


Spoiler: Warning%3A%20Spoiler!



To me, A sounded the best, with B and C being fairly similar. If I had to choose between B and C, I would say B was the 320 and C was the 128.


   
  Group 2: I invalidated the test by accidentally seeing someone's answers for group 2 when looking at Group 1 answers (after completing the group 1 test).
   
  Group 3:
   
   


Spoiler: Warning%3A%20Spoiler!



I would say C was Lossless, B and A were lossy. If I had to choose between them, I'd say A was 128.


   
   
  The conversions for this are pretty well done. I'd be interested in a test that compared a series of converters. All 3 tracks are 128, one done by LAME, one by iTunes, and one by some other converter. See if people can ABX between that.


----------



## liuk

Quote: 





jiminy said:


> Have you got the answers? Have we mistaken 128 with lossless? I do hope so


 
   
   
   


Spoiler: Warning%3A%20Spoiler!



yes we did. I do not have the answer but guessing the bitrates with a spectrometer is quite easy. And yes, we swapped 128 with lossless in group A.


----------



## Nirvana Woman

Just bumping this because I think it's very interesting and useful.
   
  I've done 128kbps vs 320kbps tests in the past before and save for some very minor things (more distortion on clear high notes with lower kbps files for example), I can't reliably tell a difference. As I have always felt, recording quality is what it's all about, not bitrate. Granted, I am a music listener and not really an audiophile.
  I am sure certain listeners have their ways of finding out structural differences between bitrates though.


----------



## jiminy

Ops, I take back my comments about music choice as I was playing the files direct from my macs download folder and had not realised that it was playing the rubbish I had downloaded last night after each of your tracks.
No wunder they all sounded like dubstep.


Spoiler: Warning: Spoiler!



ive just had another go with the 2nd one and I'm not sure i can tell any differance, this is the hardest one. I'll go with a as lossless and b and c are 128 then 320 but guesses


----------



## GrindingThud

Very nice.


Spoiler: Warning%3A%20Spoiler!



 
  Well, I listened to these for quite a while, and i'm not sure I can tell the two better files apart on my laptop, so I'm only going to make an attempt at the 128K files.  Group 1, file C somehow seems annoying...Group 2, file A seems gritty and B is not much better....I'm not sure I have an opinion on this material after listening for a while.  I like the music....i may be buying this one.    Group C I had to convince myself I heard differences...all versions lacked air and depth needed to tell them apart....it may have been over produced from the start.


----------



## felimz

Spoiler: Warning: Spoiler!



1 - B>A>C

2 - C>A>B

3 - C>B>A


----------



## thepredestrian

Why don't you just post the answers and type them in spoilers? We can check them straightaway, rather than waiting a week and forgetting about this thread


----------



## chewy4

Quote: 





thepredestrian said:


> Why don't you just post the answers and type them in spoilers? We can check them straightaway, rather than waiting a week and forgetting about this thread


 
  Because people cheat. While most wouldn't there probably would be a few people that would.
   
  I'll leave the files and everything once I do post the answers so people can still ABX and the like, but I'd like to get some statistics beforehand. It'll probably be closer to Monday or Tuesday, as the views have really skyrocketed this weekend(there were just over 300 on Thursday I believe, now it's in the thousands). There have been over 200 downloads of the first group too...


----------



## stv014

Spoiler: Warning%3A%20Spoiler!



It is still possible to cheat for those who want to, by simply looking at the spectrum of the files (even the foobar2000 "Spectrogram" visualization is sufficient for this purpose, no audio editor is needed).
  By the way, Group 2 does not seem to include a 128 kbps file (although it could have been intentional to see who "hears" a difference between identical files) ?


----------



## liuk

Quote: 





chewy4 said:


> Because people cheat. While most wouldn't there probably would be a few people that would.
> 
> I'll leave the files and everything once I do post the answers so people can still ABX and the like, but I'd like to get some statistics beforehand. It'll probably be closer to Monday or Tuesday, as the views have really skyrocketed this weekend(there were just over 300 on Thursday I believe, now it's in the thousands). There have been over 200 downloads of the first group too...


 

 it would be great if, alongside with the answers, you publish some statistics per group and file type.
   
  Table 1: will tell us the percentage of users that got the answers correctly either by group and by filetype 

 *GROUP* *128kbps* *320 kps* *lossless* *tot* *participants* 1 n k j n+k+j p1 2 o p y o+p+y p2 3 e d f e+d+f p3 tot n+o+e k+p+d j+y+f     participants p128 p320 ploss    
   
   
  with n,k,j etc being the number of people that got the answer right, and p1,p2, p3 the total of participants for the groups and p128,p320, ploss the number of retrieved answers per filetype (right and wrong)
   
   
  I also assume that 320/LL will present the highest margin of error among users, so another interesting table could answer the question "can users reliably distinguish 128kbps mp3s from higher quality files?
   
  Here we need Table 2, which can actually be derived from Table 1 but for the sake of handiness..

 *GROUP* *right/wrong* *128kbps* *320/lossless* 1 Right n k+j 1 Wrong w1 q1 2 Right o p+y 2 Wrong w2 q2 3 Right e d+f 3 Wrong w3 q3
   
  With a little bit of statistics we can do the math. I can do that, if you provide the table - I'll deliver. Only problem is that we probably have not many data points per group, especially group 2. Actially, Group1 and Group3 are not that diverse as music genres, so if we lack of data (we need at least 10-15 participants per group, not necessarily unique) we can aggregate that.
   
  This way we can add some actual science to the result of the test and conclude (or not conclude) with statistical significance (or no statistical evidence) that people on head-fi are good/bad at detecting sound quality of music files.


----------



## Charzarn

This was pretty awesome. 
   


Spoiler: Warning%3A%20Spoiler!



Alright. Lossless to 128.
   
   
  Group B
  B-A-C
   
  B and A were really close but I felt B had a little bit better separation so I went with B. 
   
  Group C
   
  I really liked this one
   
  C-B-A 
  Finding the Flac seemed easier in this test where B and A stood no chance. 
  Though I did find the A was more enjoyable to listens to then B. In B you could hear more instruments just popping out and the bass notes were more prominent but in return felt muddy. I almost pick 128 for this but i knew it was just one of those moments when a song cant do it all. A was more enjoyable because there wasnt as much bass so there was less to muddy it up. Still not as clear as the highs in B but Sounds nicer. B was just trying to hard to be lossless.


----------



## Maker

Quote: 





chewy4 said:


> Because people cheat. While most wouldn't there probably would be a few people that would.
> 
> I'll leave the files and everything once I do post the answers so people can still ABX and the like, but I'd like to get some statistics beforehand. It'll probably be closer to Monday or Tuesday, as the views have really skyrocketed this weekend(there were just over 300 on Thursday I believe, now it's in the thousands). There have been over 200 downloads of the first group too...


 
   
  skyrocketed urghh? nice, it´s propably becouse i gave it on reddit  http://www.reddit.com/r/headphones/comments/16ti95/lossless_vs_128kbps_mp3_vs_320kbps_mp3_blind_test/


----------



## Tryptamine

I'm not the most active Head-Fi member, but saw this on reddit, hehe. 


Spoiler: Group%201



[size=1em]A: 320kbps[/size]
  [size=1em]B: FLAC[/size]
  C: 128kbps
   
  C sounds crappy, A and B required me to switch headphones a couple of times (from Beyer BT770 to Alessandro MS-1 and back), but it's definitely noticeable.
   


   


Spoiler: Spoiler%202



A: FLAC (guess)
  B: 128kbps (guess)
  C: 320kbps (guess)
   
  Can't really tell. B and C sound too similar to be 128kbps and 320kbps, but if I had to pick I'd go for C. A seems to sound a little tad closer to source than B and C. The source definitely sounds like it already has some compression artifacts too, could be a mistake by OP but might as well be the recording itself and the fact that it's glitchy music and I'm quite high atm.


   


Spoiler: Group%203



A: 128kbps
  B: 320kbps
  C: FLAC
   
  I know those tunes really good since I'm a huge Shpongle fan. The plucks just sound broken in A and B and have plenty of artifacts.


----------



## chewy4

Quote: 





maker said:


> skyrocketed urghh? nice, it´s propably becouse i gave it on reddit  http://www.reddit.com/r/headphones/comments/16ti95/lossless_vs_128kbps_mp3_vs_320kbps_mp3_blind_test/


 
  Well that would explain it then, thanks for sharing it.
   
  I'm just going to post the key now, I figure there is enough to form some statistics. I'm too hungover to make accurate statistics right now so I'll do them Monday night, and I'll try to make them as in-depth as possible. Make sure to not edit your post after this point, as I will consider your answers invalid.
   
  Key:
  
   


Spoiler: Warning%3A%20Spoiler!%20



 
  1: Mighty River - Railroad Earth
  B - lossless
  A - 320
  C - 128
   
  2: Fahrenheit Fair Enough - Telefon Tel Aviv
  A - Lossless
  B - 320
  C - 320
   
  3: Falling Awake - Shpongle
  A - 128
  B - 320
  C - lossless
   
   
  stv014 mentioned that there didn't appear to be a 128kbps file in group 2. If this is the case, it wasn't intentional.I thought I was pretty careful with it... so can anyone elaborate on that? I'm no pro at looking for this kind of stuff.
   
  EDIT: Just re-converted, I did fudge up on group 2. The one I had as 128kbps is indeed just another 320kbps file... Didn't intend for this, so sorry guys. I will post the real 128kbps if people want to ABX it.


----------



## KamijoIsMyHero

Ooh 2/3, I'll give the tracks another listen tonight, first one was really hard

now just 1/3 -.-


----------



## musical-kage

With the Mighty River track, I simply got the 320kbps and Lossless files mixed up but I was right with the 128kbps.
   
  I only got the 320kbps right on Fahrenheit Fair Enough. I got the 128kbps somehow mixed up with the lossless... How I don't know.
  I just opened that up in Audacity though, and its clipped on the Lossless version in that case... Look at the levels... They are clipped compared with the other two. Whether that had something to do with it.
   
  Didn't get any right on Falling Awake  I thought the 320kbps file was 128kbps... haha


----------



## Strangelove424

My answers were no more statistically meaningful than guesses. I don't see any rhyme or reason in them compared to the answer key. Based on these results, I think I can listen to mp3s and MOG now without second guessing the quality.


----------



## chewy4

Well I did mess up on group 2, I swear it wasn't intentional(as tempting as doing something like that was).
   
  I edited the key with the change.
   
   


Spoiler: Warning%3A%20Spoiler!



What I did find interesting is that multiple people said that they guessed on which was 320 and lossless for this group, but said that they _weren't_ guessing on the 128kbps. Which didn't actually exist.


----------



## Strangelove424

Well, disregarding the error in group 2's electronic stuff, I thought the worst sounding files in Group 1 + 3 were C. Group 1's C was indeed 128kbps, but as it turns out Group C's 3 was lossless, and I believe that mistaking 128 for lossless by itself invalidates all of my observations. Those were acoustic samples too, which should be easiest to tell apart.


----------



## Luchico638

Group A


Spoiler: Warning: Spoiler!



Oddly enough, I don't notice a difference betwen the 3. 
I think C may have been the lossless one.


----------



## liuk

Turns out my guesses where indeed no more than random guesses, and my brain did all the "justification" for me (and many of ours did too, I see).
   
  Out of curiosity yesterday I did a second blind test with some of my music with my girlfriend.
   
  Only piece I could get right to be compressed 128 vs Lossless was the 4th movement of Dvorjak's 9th symphony. And I had to take the 128 compression down to "default" quality, because with "high" quality I really could not spot any difference.
   
  Lesson learned: compress everything to 256kbps and you'll never have to complain about anything.


----------



## musical-kage

Mighty River was a lot more complex in sound. 
  I'm interested in maybe creating one myself with a classical track and maybe one other at different parts, with complex and more analogue.


----------



## chewy4

Just remember that you do need to volume match for 128kbps. It wasn't neccessary for 320kbps for the files and encoder that I chose, but 128kbps always came out a little quieter(around a half dB, could vary even more for songs with a high dynamic range).
   
  How I did it was using dbpoweramp's replaygain apply. I converted the files, compared their replaygain levels in foobar, and then converted again(from the source) using the adjustments needed in the replaygain apply DSP. I'm not sure if there is a way to get that DSP to make the adjustments automatically, if there is I didn't figure it out.


----------



## ozarkcdn

Very nice - and love you did this for all the 'I don't listen to anything less than 320kps' crowd.  Maybe in a secret bunker in a deep underground layer... 
	

	
	
		
		

		
			




   
  You did some good work here - what would also be fun is comparing AAC vs MP3 etc, which they do over here
   
  That's where I got the idea to compress my mobile library to 96k AAC files. IIRC, the 96k AAC files sounded almost a little better than 128k mp3... and takes about 33% less room-ish.
   
   
  Anyhow - I'd like to see some others report their report from ABX plugin on foobar, here's mine!
   
  Nice selection of tracks!
   
  My best guesses were on:


Spoiler: Warning%3A%20Spoiler!



 
  #1 - B
  #2 - A
  #3 - C
   
  Hahahahahah.... guess I messed that up!


   
   
   
  Test data:
   
   


Spoiler: Warning%3A%20Spoiler!



Track #1:
  B vs souce was the most reliable
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:23:53
  File A: C:\Users\Geoff\Downloads\Group 1\1_A.wav
 File B: C:\Users\Geoff\Downloads\Group 1\1_Source.wav
  15:23:53 : Test started.
 15:25:09 : 00/01  100.0%
 15:25:56 : 00/02  100.0%
 15:26:56 : 00/03  100.0%
 15:27:02 : Test finished.
   ----------
 Total: 0/3 (100.0%)
  --------------------------------
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:27:45
  File A: C:\Users\Geoff\Downloads\Group 1\1_B.wav
 File B: C:\Users\Geoff\Downloads\Group 1\1_Source.wav
  15:27:45 : Test started.
 15:29:39 : 01/01  50.0%
 15:30:43 : 02/02  25.0%
 15:31:18 : 03/03  12.5%
 15:32:53 : 03/04  31.3%
 15:33:06 : 03/05  50.0%
 15:33:42 : 04/06  34.4%
 15:34:36 : 05/07  22.7%
 15:35:30 : 06/08  14.5%
 15:35:35 : Test finished.
   ----------
 Total: 6/8 (14.5%)
  ----
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:35:51
  File A: C:\Users\Geoff\Downloads\Group 1\1_C.wav
 File B: C:\Users\Geoff\Downloads\Group 1\1_Source.wav
  15:35:51 : Test started.
 15:36:18 : 00/01  100.0%
 15:36:44 : 00/02  100.0%
 15:37:14 : 00/03  100.0%
 15:37:38 : 00/04  100.0%
 15:37:41 : Test finished.
   ----------
 Total: 0/4 (100.0%)
   
  Group 2
  ---
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:41:18
  File A: C:\Users\Geoff\Downloads\Group 2\2_A.wav
 File B: C:\Users\Geoff\Downloads\Group 2\2_Source.wav
  15:41:18 : Test started.
 15:43:05 : 01/01  50.0%
 15:43:16 : 01/02  75.0%
 15:43:43 : 02/03  50.0%
 15:44:27 : 03/04  31.3%
 15:45:11 : 04/05  18.8%
 15:45:18 : Test finished.
   ----------
 Total: 4/5 (18.8%)
   
  --
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:45:40
  File A: C:\Users\Geoff\Downloads\Group 2\2_B.wav
 File B: C:\Users\Geoff\Downloads\Group 2\2_Source.wav
  15:45:40 : Test started.
 15:46:50 : 01/01  50.0%
 15:47:23 : 01/02  75.0%
 15:47:49 : 02/03  50.0%
 15:48:22 : 03/04  31.3%
 15:48:45 : 03/05  50.0%
 15:49:12 : 04/06  34.4%
 15:49:58 : 04/07  50.0%
 15:50:02 : Test finished.
   ----------
 Total: 4/7 (50.0%)
  ---
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:50:20
  File A: C:\Users\Geoff\Downloads\Group 2\2_C.wav
 File B: C:\Users\Geoff\Downloads\Group 2\2_Source.wav
  15:50:20 : Test started.
 15:51:28 : 00/01  100.0%
 15:52:09 : 00/02  100.0%
 15:52:28 : 00/03  100.0%
 15:52:30 : Test finished.
   ----------
 Total: 0/3 (100.0%)
   
  ------
   
  Group 3
  ---
   
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:54:45
  File A: C:\Users\Geoff\Downloads\Group 3\3_A.wav
 File B: C:\Users\Geoff\Downloads\Group 3\3_Source.wav
  15:54:45 : Test started.
 15:55:04 : 01/01  50.0%
 15:55:25 : 02/02  25.0%
 15:56:12 : 02/03  50.0%
 15:56:42 : 03/04  31.3%
 15:57:00 : 04/05  18.8%
 15:57:05 : Test finished.
   ----------
 Total: 4/5 (18.8%)
   
  --
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:57:21
  File A: C:\Users\Geoff\Downloads\Group 3\3_B.wav
 File B: C:\Users\Geoff\Downloads\Group 3\3_Source.wav
  15:57:21 : Test started.
 15:57:58 : 01/01  50.0%
 15:58:09 : 02/02  25.0%
 15:58:31 : 02/03  50.0%
 15:58:41 : 02/04  68.8%
 15:59:10 : 02/05  81.3%
 15:59:24 : 02/06  89.1%
 15:59:26 : Test finished.
   ----------
 Total: 2/6 (89.1%)
   
  --
  foo_abx 1.3.4 report
 foobar2000 v1.2.1
 2013/01/21 15:59:43
  File A: C:\Users\Geoff\Downloads\Group 3\3_C.wav
 File B: C:\Users\Geoff\Downloads\Group 3\3_Source.wav
  15:59:43 : Test started.
 16:00:04 : 00/01  100.0%
 16:00:26 : 01/02  75.0%
 16:00:38 : 02/03  50.0%
 16:01:09 : 03/04  31.3%
 16:01:40 : 04/05  18.8%
 16:01:45 : Test finished.
   ----------
 Total: 4/5 (18.8%)


----------



## chewy4

I posted some statistics in the original post. I stopped at the point where I posted the key, so sorry if you just missed it.

 Thank you to everyone who participated!
   
  Also,
   
   


Spoiler: Warning%3A%20Spoiler!



I'll post a real 128kbps file for group 2 later tonight for those who want to ABX.


----------



## deadlylover

I recognised Shpongle, so that's the one I tested for.

I could only ABX the 128kbs sample, maybe I'll try the 320kbps again later tonight when the house isn't as noisy, but I think I'll fail again as I believe the 320kbps sample is transparent for this track. 



Spoiler: Warning: Spoiler!



foo_abx 1.3.4 report
foobar2000 v1.1.18
2013/01/22 14:39:47

File A: C:\Users\Alice\Desktop\Group 3\3_A.wav
File B: C:\Users\Alice\Desktop\Group 3\3_Source.wav

14:39:47 : Test started.
14:44:13 : 00/01 100.0%
14:45:02 : 01/02 75.0%
14:45:15 : 02/03 50.0%
14:45:46 : 03/04 31.3%
14:46:07 : 04/05 18.8%
14:46:49 : 05/06 10.9%
14:47:58 : 06/07 6.3%
14:48:04 : 07/08 3.5%
14:48:20 : 07/09 9.0%
14:48:38 : 08/10 5.5%
14:48:42 : 09/11 3.3%
14:48:55 : 10/12 1.9%
14:51:43 : 11/13 1.1%
14:51:55 : 11/14 2.9%
14:52:01 : 12/15 1.8%
14:52:44 : 13/16 1.1%
14:52:56 : 14/17 0.6%
14:53:22 : 14/18 1.5%
14:54:04 : 15/19 1.0%
14:54:13 : 16/20 0.6%
14:54:46 : 17/21 0.4%
14:55:08 : 18/22 0.2%
14:55:32 : 19/23 0.1%
14:56:01 : 20/24 0.1%
14:56:23 : 21/25 0.0%
14:56:30 : Test finished.

 ---------- 
Total: 21/25 (0.0%)


----------



## cer

chewy4, what mp3 encoder was used?


----------



## chewy4

Quote: 





cer said:


> chewy4, what mp3 encoder was used?


 
  LAME via dbpoweramp.


----------



## sonitus mirus

I listen almost exclusively to 320 kbps mp3 files using MOG or from ripping a CD with LAME.  In every group in this test, I selected the 320 kbps file as the lossless version.  I think that is what I am used to hearing, so it sounded best to my ears.  But with such a small test sample, I was probably only guessing and the results were most likely a chance happening and nothing more.


----------



## musical-kage

...  Whoops


----------



## pathway

Thanks for doing this - really interesting to blind test myself.
   
  (Kit used - Laptop, Topping D1 Mark II, AKG550)
   
  I got Group 1 100% correct and I was pretty confident about it - there was no guesswork but it did take quite a number of back to back listens. I back to backed 2 tracks and ranked them - then did the same again for another pair and the order fell out quite easily. That said, I would never have been able to tell if it was lossless or heavily compressed (128) without having the chance to listen multiple times back to back.
   
  More interestingly I got group 3 flipped (as it seems a few others did) - ranking the lossless file as 128 and vice versa. I found that group far more difficult to differentiate in general and guesswork crept into it. the 128 track 'felt' more open to me and that is why I ranked it lossless! - perhaps as there was just so much more 'going on' in the track it was harder to pick out details but it is great example of putting too much onus on the numbers
   
  Oh - I didn't try group 2 as read something during the thread about it by accident and didn't feel I would be totally 'blind' to the test.
   
  REALLY interesting experiment - many thanks for providing.
   
  O.


----------



## MpX5

Got new headphones, didn't warm up them properly, so here's my guess:
   


Spoiler: Warning%3A%20Spoiler!



Group 1: sounds pretty same: 
  B sounds sharper for me so let it be flac
  A is 128 cuz I didn't like it first time 
  C - 320
   
  Group 2:
  from flac to 128
  C 
  B
  A
  difference - C sounds wider, like there is more sounds
   
  Group 3:
  no drums here(
  A is 128 because sounds like there is lack of bass
  B & C has not much difference for me
  C is flac


----------



## elwappo99

Great thread! 
   
  I REALLY like the work you put into this. Keeping them the same file size, same sample, and volume equalized must have taken some work. Well done! 
   
  I only went through the first one. Based on my quick listening, I did get the answers correct, but I always love listening to these things, because they show how subtle of a difference all the Flac vs mp3 is, especially when you consider the 320 bit rate. I saw some people's comments about listening to brightness or presentation to pick out the differences. I believe somewhere a study was done and people with ibuds liked 128 the best. I think this leaves out some details in a song, and makes it sound more forward. 
   
  Anyway, the way I can easily pick out the differences are when I hear any kind of moraca or a drum brush (like at the 0:10 mark).  Also something to listen for is the subtle decay of guitars (like the decay in the 0:04 - 0:06). The subtle decay from a brush or the individual beads hitting are lost in 128, and 128 can easily be spotted doing this. 320 and Flac is a little more difficult.


----------



## catspaw

Spoiler: Warning%3A%20Spoiler!%20Personal%20stats



Group 1 i noticed a diffrence from 128 to the other 2 but that was it. Had no idea if was better or not.
  Group 2 i noticed there was more clarity on the first song on the high notes, but didnt know if thats supposed to be good or bad ;D the other two did seem same to me.
  Group 3 Got it right very fast (was actualy the only group that was clear, others seemed very similar in general).


----------



## Sleinzel

Didn't do this Test, but i got some tracks where I can tell the difference over 80% between flac and 128 and around 65% between 320 and flac.
   
  - Pink Floyd - The Dark Side of The Moon - Time
  - Netsky - 2 - Love has gone
  - Hans Zimmer - The Dark Knight Rises - Imagine The Fire (actually a 24/196, because the cd has clipping )


----------



## Jeremy1234

I was quite surprise that my colleague passed me 2 songs which are so detail and nice but they are only 128Kbps..............................
	

	
	
		
		

		
			




   
  And I thought 320Kbps should be better.....................................


----------



## jake120

Quote: 





jeremy1234 said:


> I was quite surprise that my colleague passed me 2 songs which are so detail and nice but they are only 128Kbps..............................
> 
> 
> 
> ...


 

 Yes I agree with this.
   
  I tried an 128kbps vs 320kbps vs flac song.
   
  at some times i could agree there were differences
  the 128kbps was not as loud
  the 320kbps was louder and instruments were clear
  the flac was as loud as the 320kbps but some instruments felt like they followed through until the end more.
   
  and then some songs the differences i should have noticed did not register.
   
  i wonder if its not just equipment but its taste as well that matters...


----------



## KamijoIsMyHero

this thread is interesting since I still hear a difference with 128 vs 320 from illegal DLs, but if the music is being passed from a competent legal source (or one that actually has a ripped copy)  then I don't hear a difference at all


----------



## chewy4

Quote: 





kamijoismyhero said:


> this thread is interesting since I still hear a difference with 128 vs 320 from illegal DLs, but if the music is being passed from a competent legal source (or one that actually has a ripped copy)  then I don't hear a difference at all


 
  Makes sense, you don't know what that file has been through.
   
  It could have been burnt to a CD and re-ripped as mp3 several times, converted between lossy formats, anything really.


----------



## soundstige

All tests were done just now, with about 5 minutes spent on each zip package. They were done with a HiFiMan HE-400 headphone through a PCM2704 DAC and CMOY amp, and played through a regular playlist in Foobar2000. Obviously, they were done before reading the results/statistics. Also before reading the results, I think my Etymotic earphones might have had more chance at dissecting every possible artifact, but the HE-400 are certainly more "real-world" listening devices. 
   
  Group 1:


Spoiler: Warning%3A%20Spoiler!



Track 1_C is obviously the 128 sample. There are at least a dozen audible examples of pre-echo or warbling artifacts. I would say I'm 95% confident on this. I believe I hear a very small amount of artifacts in the cymbal crash around 21s that are similarly present in sample C, in sample B. If I had to guess, B would be 256, C is 128, and A is Lossless.
  So, 1_A Lossless, 1_B 256, 1_C 128!


 Group 2:



Spoiler: Warning%3A%20Spoiler!



I'm very unfamiliar with this type of music. If I must guess, and it is a guess, I'd say 1_B is the 128 sample, only because I THINK some complex parts sounded softer and less pronounced than others. As for the other samples, I can't give any meaningful input, even without knowing the results. I just, basically, don't know what to listen for. The music is nice though!
  So, 2_A ???, 2_B 128?, 2_C ???


 Group 3:



Spoiler: Warning%3A%20Spoiler!



The high E string plucking on 3_C around 22s sounded like it got a little mangled towards the end of its resonation. A slightly warble. Other than that, I'm not confident of any difference I heard between the tracks. If a gun were to my head, I'd say C is 128, A is 256, and B is Lossless. Not much real confidence about the guess or track as a whole.
  So, 3_A 256?, 3_B Lossless?, 3_C 128


   

  Cheers and thanks for the test!
   
  Edit: after reading the results, my impressions on my impressions:
   


Spoiler: Warning%3A%20Spoiler!



The only one I was truly confident about, Group 1, I was right! (only on the 128, but that's all I'd consider to count lol). The rest, all guessing. Though I was kind of right on 2! looool fun test


----------



## ozarkcdn

Quote: 





> OriginaI agree with this.
> 
> 
> 
> ...


 
  yo chewy... think you could rig a test to record outputs of a sample run through different DACs? I think results would be very interesting...


----------



## stv014

Quote: 





ozarkcdn said:


> yo chewy... think you could rig a test to record outputs of a sample run through different DACs? I think results would be very interesting...


 
   
  It has been done already 
	

	
	
		
		

		
		
	


	




 Check this, this, or this thread.


----------



## JUKILOP02

A is the best, B sounds like 128 and C 320 Group 1


----------



## OmarSy

Great thread, I'll participate when I get home!


----------



## Prowler

Even though the test was concluded (results were posted in a spoiler, really I didn't peek, didn't read any other posts, didn't even notice just how old the first post was, really!), here are my results. I made a notepad++ document and I'll paste it into a spoiler.
   
  Before doing this test I was previously doing my own LAME V2, V0, lossless ABX testing, so I was already a bit familiar with mp3 artifacts (all thanks to recently buying a headphone amp for my Titanium HD, which I should have done when I built the PC, but hey, I had no idea it'd help that much).
   
  I was using: Foobar2000 -> Sound Blaster Titanium HD -> FiiiO E09k -> Sennheiser PC 350 (modded).
   


Spoiler: Warning%3A%20Spoiler!



Group 1
 A - 320 kbps mp3
 B - lossless
 C - 128 kbps mp3

 Group 2
 basically random guessing, couldn't hear any differences
 A - 128 kbps mp3
 B - lossless
 C - 320 kbps mp3

 Group 3
 A - 128 kbps mp3
 B - 320 kbps mp3
 C - lossless
  
  above is what I write in notepad++ after I concluded testing, below are comments added at the time of this post after seeing the spoiler.
  I spent about an hour on this, way more than I thought till I looked at the clock.
   
  My testing methodology was to first identify the 128 kbps, relatively easy for groups 1 and 3 but I pretty quickly skipped over group 2 as I struggled with it.
  I had an easier time identifying group 1 than group 3. Group 3 took a bit more time but eventually I had an ABX test < 10%, enough for my conclusion in this test.
  Finally I re-visited Group 2 but somehow my mind was so fixated on the fact there must have been a 128kbps encoding present then I committed to the above. After I saw the results for group 2 I just laughed.
   
  Yeah, the group 2 results are actually what I expected (320 kbps vs lossless),so  I feel fine using LAME V0, especially for a portable player. The differences between 320 kbps and lossless are so minute that without the foobar ABX tool it'd be virtually impossible to identify.
   
  I think I'm dome with lossy vs lossless testing... unless someone sets up another one of these tests and I find out about it before it ends 


   
  Thanks for the creating the test chewy4.


----------



## MrLazyAnt

Hah! Got it right! I only listened to the Shpongle one, but I'm well stoked that I got it right! I have a pretty decent stereo, albeit very dated (speakers form the 70's and an amp from the late 80's), but my soundcard is an onboard laptop cheapie. So yay me! =P


----------



## sonitus mirus

You really need to ABX any 2 of the samples at least 15 times before getting excited. If you just are just trying to guess which file is which, you might be right, but can you repeat that or was it just a guess?

Someone might correctly choose how many fingers I am holding up behind my back, but that doesn't mean they are psychic.

You got it right on one song. The next step is to prove to yourself that you can pick the right files every time.


----------



## MrLazyAnt

Quote: 





sonitus mirus said:


> You really need to ABX any 2 of the samples at least 15 times before getting excited. If you just are just trying to guess which file is which, you might be right, but can you repeat that or was it just a guess?
> 
> Someone might correctly choose how many fingers I am holding up behind my back, but that doesn't mean they are psychic.
> 
> You got it right on one song. The next step is to prove to yourself that you can pick the right files every time.


 
  Pardon my ignorance, but what's ABX?


----------



## Brooko

Quote: 





mrlazyant said:


> Pardon my ignorance, but what's ABX?


 
   
  It's a double blind comparison test - basically comparing two different file containers (same song), removing the effect of placebo (blind) so you can see what you actually hear.
   
  Quick link (http://www.head-fi.org/t/655879/setting-up-an-abx-test-simple-guide-to-ripping-tagging-transcoding) to a simple guide I wrote that covers ripping, tagging, transcoding AND setting up an abx in Foobar.


----------



## MrLazyAnt

Quote: 





brooko said:


> It's a double blind comparison test - basically comparing two different file containers (same song), removing the effect of placebo (blind) so you can see what you actually hear.
> 
> Quick link (http://www.head-fi.org/t/655879/setting-up-an-abx-test-simple-guide-to-ripping-tagging-transcoding) to a simple guide I wrote that covers ripping, tagging, transcoding AND setting up an abx in Foobar.


 
  Isn't that what Chewy was doing? I mean, I get that I may have "cheated" by only doing one song, but isn't the whole point of this thread to do a double blind audio test?


----------



## sonitus mirus

Quote: 





mrlazyant said:


> Pardon my ignorance, but what's ABX?


 
  Please don't take my comment as a denigration.  That was not my intent.
   
  What I was attempting to show was that this test is about a collection of results from a group, and not about any individual achievements.
   
  It is like having 50 people guess a number between 1 and 10.  If 5 out of 50 people guess correctly, it doesn't mean those 5 people are extraordinary unless those 5 people can repeat the test with similar results multiple times. 
   
  http://en.wikipedia.org/wiki/ABX_test


----------



## Brooko

Quote: 





mrlazyant said:


> Isn't that what Chewy was doing? I mean, I get that I may have "cheated" by only doing one song, but isn't the whole point of this thread to do a double blind audio test?


 
  Yes
   
  But to compare the tracks properly - you need to use something like Foobar 2000s abx tool.  Setting it up is in the 5th post of that link I sent you.
   
  How did you perform your double blind?  How many iterations?  Or did you just close your eyes, do it once, and called it good?


----------



## sonitus mirus

Quote: 





mrlazyant said:


> Isn't that what Chewy was doing? I mean, I get that I may have "cheated" by only doing one song, but isn't the whole point of this thread to do a double blind audio test?


 
  Sure, but if I guess heads on a coin flip and get it right, should I be excited about my coin-flip-guessing ability?  This test is about many people guessing the result of a coin flip, and seeing what the overall results might be.


----------



## MrLazyAnt

Quote: 





brooko said:


> Yes
> 
> But to compare the tracks properly - you need to use something like Foobar 2000s abx tool.  Setting it up is in the 5th post of that link I sent you.
> 
> How did you perform your double blind?  How many iterations?  Or did you just close your eyes, do it once, and called it good?


 
  I listened to the three samples 6 times on shuffle-repeat wrote down thusly: 1 reference 2 reference 3 reference 4 lossless 5 320 6 128 7 128 etc..... when I was done I checked the playing history using the foobar console, and wrote down next to each selection A B or C, then verified with chewy's answers, and the last 3 sets (7-18) were consistently correct.
   
  EDIT: But, to be fair, I have the original track.


----------



## Brooko

Quote: 





mrlazyant said:


> I listened to the three samples 6 times on shuffle-repeat wrote down thusly: 1 reference 2 reference 3 reference 4 lossless 5 320 6 128 7 128 etc..... when I was done I checked the playing history using the foobar console, and wrote down next to each selection A B or C, then verified with chewy's answers, and the last 3 sets (7-18) were consistently correct.
> 
> EDIT: But, to be fair, I have the original track.


 
   
  OK - again - suggest you go to my post, set up an actual blind abx, compare just 2 of the files - run 15 iterations, and post the logs.  Then do the next two files.  Etc.  Takes some time but worth it in the end.


----------



## MrLazyAnt

Quote: 





brooko said:


> OK - again - suggest you go to my post, set up an actual blind abx, compare just 2 of the files - run 15 iterations, and post the logs.  Then do the next two files.  Etc.  Takes some time but worth it in the end.


 
  Maybe some other time...... Right now I'm just too damn lazy to download and install the necessary software. Not to mention my PC is a 7 year old hunk-o'-junk that I bought on a budget, it can't even run Cubase through ASIO smoothly any more and I don't know how many more programs/plug-ins I can install before it crashes once and for all. It's already dying the poor thing. But maybe one day
   
  I did succeed in the "Mighty River" one, and didn't bother with the "Telefon Tel-Aviv" one having read that 2 of them were 320kps, and being a lazy **** who just wants to listen to his music.


----------



## HE1RO

I honestly dont think its the bit rate that makes songs sound crappy, but how an what it was compressed with. I rip CD's all the time and the program i use has tons of options to choose from. I could run a lossless file through it and not alter its bit rate, but change a few settings and you would think it was an AM radio transmission lol
   
  Imo, its the overall file size in comparison to its format and compression and what options were chosen upon digitizing it that makes a difference. If its a 320kbps MP3 song thats 3minutes~ long and its 5.0mb+ in size, its probably going to sound just as good as if it was straight from the CD itself.


----------



## yesman94

wow..
   
  thankyou chewy this is a great way to test and learn to differ quality songs. I can tell a different between the lossless or not on the accoustic one, the second song, since i dont like techno or whatever is that, i cannot enjoy and tell a different. It is easy to tell that lossless or lossy but its hard to tell between 128kbps and 320kpbs.
   
  here's my answer, from greatest quality to most poor :


Spoiler: Warning%3A%20Spoiler!



 
   
  1. B>A>C
  2. C>B>A
  3. C>A>B
   


   
  A little missed from the answer what Iam happy that i can tell between lossless or not.
   
  Thanks again for chewy   
   
  Sorry for my poor english.


----------



## chewy4

Quote: 





he1ro said:


> I honestly dont think its the bit rate that makes songs sound crappy, but how an what it was compressed with. I rip CD's all the time and the program i use has tons of options to choose from. I could run a lossless file through it and not alter its bit rate, but change a few settings and you would think it was an AM radio transmission lol
> 
> Imo, its the overall file size in comparison to its format and compression and what options were chosen upon digitizing it that makes a difference. If its a 320kbps MP3 song thats 3minutes~ long and its 5.0mb+ in size, its probably going to sound just as good as if it was straight from the CD itself.


 
  Well a 3 minute 320kbps file is always going to be the same size(7.2MB) regardless of settings. Bitrate is just a ratio of time and file size.
   
  But I did indeed use different than default settings for these files. There is a quality slider for LAME compression that isn't by default set to the highest quality. Not sure how big of a difference it makes, but it's simple enough to change it.


----------



## Sound Quest

Sorry for digging this thread back up again. I see the OP said that he/she used the LAME encoder.
   
  Out of interest, which version of the LAME encoder did you use? And what speed did you have the encoder set to?
   
  Also, did you use joint stereo or full stereo mode?


----------



## chewy4

I used the most recent release of LAME. 3.99 I believe, as it hasn't been updated since October 2011. I can double check to make sure when I get home but I didn't install the software until about a month or so before the test so I'm 99% sure here.
   
  It was set to slow speed, high quality. Full stereo. Basically I didn't change anything from default other than the quality slider.


----------



## Sound Quest

Quote: 





chewy4 said:


> I used the most recent release of LAME. 3.99 I believe, as it hasn't been updated since October 2011. I can double check to make sure when I get home but I didn't install the software until about a month or so before the test so I'm 99% sure here.
> 
> It was set to slow speed, high quality. Full stereo. Basically I didn't change anything from default other than the quality slider.


 
   
  Would it not be better to do an ABX comparason using Joint Stereo mode instead?
   
  The "Joint Stereo" mode in LAME is mathematically lossless at 128kbps and above. It will allow the encoder to save bits where both channels are 100% identical. It can then make use of those extra bits in the bit reservoir.
   
  In other words, you'll get better sound quality, making your ABX test even more challenging.


----------



## chewy4

Quote: 





sound quest said:


> Would it not be better to do an ABX comparason using Joint Stereo mode instead?
> 
> The "Joint Stereo" mode in LAME is mathematically lossless at 128kbps and above. It will allow the encoder to save bits where both channels are 100% identical. It can then make use of those extra bits in the bit reservoir.
> 
> In other words, you'll get better sound quality, making your ABX test even more challenging.


 
  That would make sense if it does help save space, I wasn't really aware of this setting before though.


----------



## Sound Quest

Quote: 





chewy4 said:


> That would make sense if it does help save space, I wasn't really aware of this setting before though.


 
   
  There's a fair bit of confusion surrounding Joint Stereo, thats because there are 2 types of Joint Stereo. There's the newer version of Joint Stereo and there is the older version which is now known as "Intensity Stereo".
   
  Intensity Stereo is lossy and LAME uses it for bitrates under 128kbps. But for bitrates at* 96kbps* and above, it uses the newer version of Joint Stereo, which contains the same lossless algorithm that lossless formats such as FLAC and APE use to help save space.
   
  Full Stereo is pretty much obsolete, as there is no quality advantage of using it over Joint Stereo.


----------



## Sound Quest

Also, it only saves space if you use it in VBR mode. You'll get the same sound quality that you would if you used Full Stereo, but the file sizes will be smaller.
   
  If you use it in CBR mode however, the file size will be the same as it would if you used Full Stereo, but the quality will be greater.


----------



## Sound Quest

Just for comparison, here is a track encoded at 128 stereo and 128 joint. (both created from lossless source)
   
*Stereo*
   

   
*Joint Stereo*
   

   
  On both spectrograms, the frequencies cut off somewhere between 16khz to 16.5khz. But in the Joint Stereo spectrogram, it contains more frequency information within that limit.
   
  You may have to enlarge the pictures to see the difference.


----------



## xnor

Quote: 





sound quest said:


> Intensity Stereo is lossy and LAME uses it for bitrates under 128kbps. But for bitrates at 128 and above, it uses the newer version of Joint Stereo, which contains the same lossless algorithm that lossless formats such as FLAC and APE use to help save space.
> 
> Full Stereo is pretty much obsolete, as there is no quality advantage of using it over Joint Stereo.


 
  Source? Afaik this is wrong.
   
  Intensity stereo is only used for extremely low bitrates that nobody uses for music anyway.
  Joint stereo means that the encoder can switch between L/R and M/S (mid/side) stereo coding.
  This is the default for <= 128 kbps and VBR quality > 4.
   
  Higher bitrates / VBR quality uses normal stereo (L/R).
   
  Joint stereo in M/S mode only works well if the signal has little stereo separation and in general if the encoder doesn't have to switch between the modes too often. Otherwise it can cause audible artifacts, i.e. worse quality.
   
  And on spectrograms: don't use them to judge audio quality. Use ABC/HR for that.


----------



## Sound Quest

Quote: 





xnor said:


> Source? Afaik this is wrong.
> 
> Intensity stereo is only used for extremely low bitrates that nobody uses for music anyway.
> Joint stereo means that the encoder can switch between L/R and M/S (mid/side) stereo coding.
> ...


 

 I wasn't trying to use the spectrograms to prove my point, I was simply showing that using Joint Stereo allows the encoder to retain more frequency information below the cut off. The spectrogram clearly shows this.
   
  Joint Stereo is lossless. Joint Stereo can never sound worse than full stereo, regardless of channel seperation. The encoder doesn't make any guesses when converting from Stereo to JS, it uses the same mathematical precision that lossless formats such as FLAC and APE use to save space.
   
  The only time it isn't 100% precise is when you use a bitrate below *96kbps*, which will activate Intensity Stereo, which is lossy.
   
  Think of "Joint Stereo" vs "Full Stereo" like "FLAC" vs "WAV".
   
*EDIT Sorry, mean't to say that intesity stereo is used at 96kbps and under.*


----------



## xnor

Ahm, joint stereo in the lame encoder means that the encoder can switch between L/R and M/S coding.
   
  When you say lossless, you mean that coding a L/R signal as M/S can be reversed, right? Because lossy compression is applied just the same as is with L/R coding.
   
  When choosing the joint stereo option you do not force lame into M/S coding. It only switches to M/S coding when the stereo separation is low enough to save space because then most of the signal is in the M channel.
   
  If you force L/R at low bitrates then the stereo image will be preserved at the cost of audible distortion.
  If you force M/S at low bitrates then the stereo image there will be less audible distortion but the stereo image will collapse.
   
   
  The output of lame should show you how many frames were L/R and how many M/S coded. Try the joint stereo option with different bitrates (128, 192, 320) or even VBR modes (5, 2, 0) and look at the number of M/S frames.


----------



## Sound Quest

I've done this before, but i'm afraid I don't quite understand your point.
   
  When you use the Joint Stereo Mode in LAME at bitrates 96 and above, it converts from stereo to joint stereo losslessly, before applying any lossy compression. LAME encodes each channel independently.regardless of whether you use L/R or M/S mode.
   
  The encoder will take the lossless stereo file, then encode it to a lossless joint stereo file, then it will encode it to MP3. (Encoding the mid channel and the side channel independently).
   
  The number of M/S frames in an mp3 file is meaningless. You could have a track thats 95% Stereo and only 5% Mono, but using Joint Stereo over Full Stereo would still make perfect sense. It just means that 95% of the frames will be in the side channel and 5% of the frames will be in the mid channel. This will either save you a small amount of space (if VBR is used), or will give the encoder extra bits to improve the quality (if CBR is used).
   
  Let me ask you, if you think Joint Stereo is lossy, then what makes it different than Intensity Stereo? Why would there be 2 types of Joint Stereo, if they are both lossy?


----------



## xnor

I was not saying that M/S coding is lossy, read my post again, I was asking if you understand what lossless means in this case, which you seem to do.
   
  Again, the encoder does not convert the whole file to M/S unless you force (option -m f) it to.
   
  I did a simple test:
  -V 5:
     kbps        LR    MS  %     long switch short %
   138.8       64.4  *35.6*        81.7   9.9   8.5
   
  -V 2:
     kbps        LR    MS  %     long switch short %
   199.0       79.1  *20.9*        81.7   9.9   8.5
   
  -V 0:
     kbps        LR    MS  %     long switch short %
   259.7       85.5  *14.5*        81.7   9.9   8.5
   
  -b 320:
     kbps        LR    MS  %     long switch short %
   320.0       86.8  *13.2*        83.3   9.3   7.4
   
  As the bitrate gets higher, lame chooses to convert less and less frames to M/S for reasons mentioned above (preserving stereo image is more important).
   
  Btw, my lame version (3.99.5) always chose -m j as default. So the documentation seems to be outdated.


----------



## Sound Quest

Ok, I see where you're coming from now. Apologies for the misunderstanding. I would still however recommend Joint Stereo mode at birates as low as 128kbps/-V5 CBR. Not forced, but simply set by default.
   
  At higher bitrates 192kbps/-V2 and above, full stereo should be perfectly fine in most cases anyway.


----------



## Audio-Omega

Spoiler: Warning%3A%20Spoiler!



All three tracks in Group 1 sounded very similar.
  1_B sounded fuller.  The music had more body.


----------



## sonitus mirus

I'm not sure I understand what the descriptions of sounding fuller or having more body actually mean when comparing a lossless file to a lossy version.  So, what, the upper-bass frequencies are louder?  For any that make such claims, would it be possible to ABX the lossless file and the 320 kbps file 15 times and post the results?  Being fuller and having more body seems applicable when eating too much pizza, but I don't see how it applies to sound quality with compressed audio unless, perhaps, the bitrate is exceptionally low?


----------



## Audio-Omega

http://www.head-fi.org/a/describing-sound-a-glossary


----------



## D3athangel

Spoiler: Warning: Spoiler!



1. A = 128kbps mp3
 B = Lossless
 C = 320kbps mp3
  
 2.
 A = 320kbps mp3
 B = 128kbps mp3
 C = Lossless
  
 3.
 A = 128kbps mp3
 B = 320kbps mp3
 C = Lossless
  
 Done using my MacBook Pro's internal DAC though, through DT770 Premium 250ohm.  


 Was really hard xD


----------



## TegraStorm

Spoiler: Warning: Spoiler!



1) i am pretty shure they are the same audio, they sound the same to me





Spoiler: Warning: Spoiler!



 





Spoiler: Warning: Spoiler!



maybe ill be able to tell the difference when i upgrade from my HDJ-1000 to Crossfade M-100 (huge upgrade)


 BTW nice recording heheh


----------



## dakanao

Hey guys, I have noticed something about FLAC files. Is it just me, or do FLAC files push the vocals of songs behind the rest of the music? I tested out 2 FLAC files vs the same 320kbps mp3, and I could swear the mp3 had more forward vocals.


----------



## bufferoverflow

You have it all backwards ( or upside-down) :
 FLAC does not change ANYTHING but the size of the resting file, by compressing it .
 MP3 on the other hand works by data-reduction, ie throw away all the stuff you can't hear anyways -
 Except that you apparently *can* hear the difference


----------



## dakanao

bufferoverflow said:


> You have it all backwards ( or upside-down) :
> FLAC does not change ANYTHING but the size of the resting file, by compressing it .
> MP3 on the other hand works by data-reduction, ie throw away all the stuff you can't hear anyways -
> Except that you apparently *can* hear the difference


 
 I'm definitely noticing more clarity with FLAC files than with 320 kbps mp3's...


----------



## KamijoIsMyHero

Then abx them and show your results


----------



## dakanao

Spoiler: Warning: Spoiler!



Group 1A=FLAC file Group 1B=320 kbps mp3
 Group 1C=128 kbps mp3
  
 I didn't notice a lot of difference between A and B, only that A was slightly more clear sounding. I did however notice a clear difference between A and C. C was just a bit muddier in comparison to those other 2.


 Am I right?


----------



## KamijoIsMyHero

The result is on OP


----------



## dakanao

kamijoismyhero said:


> The result is on OP


 
 But im noticing the difference between 320 kbps mp3 and flac clearly with my own music. Flac sounds more clear and also slightly better detailed than mp3


----------



## Brooko

dakanao said:


> But im noticing the difference between 320 kbps mp3 and flac clearly with my own music. Flac sounds more clear and also slightly better detailed than mp3





What you are likely comparing is either two different masterings - OR improperly volume matched files. When transcoded - there is often a volume difference introduced between the original and the output.

If you are really interested in testing yourself (blind abx so that you can take placebo out of the equation) - have a read through this link .....
http://www.head-fi.org/t/655879/setting-up-an-abx-test-simple-guide-to-ripping-tagging-transcoding


----------



## dakanao

brooko said:


> dakanao said:
> 
> 
> > But im noticing the difference between 320 kbps mp3 and flac clearly with my own music. Flac sounds more clear and also slightly better detailed than mp3
> ...


 
 Well, maybe you're right, and I'm listening to differently remastered music. BUT what I found weird is that EVERY FLAC song I replaced with my 320 kbps songs, just sounds cleaner. And I know it's not my imagination, because I listened to those songs for over 100 times.


----------



## Brooko

dakanao said:


> Well, maybe you're right, and I'm listening to differently remastered music. BUT what I found weird is that EVERY FLAC song I replaced with my 320 kbps songs, just sounds cleaner. And I know it's not my imagination, because I listened to those songs for over 100 times.


 
  
 Again - if they weren't from the same source - then you're listening to different masterings.  Follow the link I left you.  The software is all free.  All it takes is time.  The truth is there is you really want to find it


----------



## AnakChan

The following is kinda related to this thread but not specific to MP3's. In my case, what I've found is that with my early ripped iTunes 4.x lossy 160kbps are (almost?) indistinguishable from a re-rip of the very same set of CDs on iTunes 11 at lossless. The reason why I'm specifying the iTunes version is because somewhere along the lines around iTunes 7 (or possibly 5 or 6?) it seems the algorithm changed that iTunes v7.x->11.x lossy 160kbps does sound worse than lossless as we expect.
  
 I have my highly subjective conspiracy theory that somewhere along the lines of iTunes 5 or 6 onwards, lossy algorithms was purposefully redesigned to make room for online sales of 256kbps music.
  
 I had a few of my HF friends in Tokyo even compare the 2003 160kbps iTunes 4  ripped vs the latest re-rip of the same CD at lossy and they were hard-pressed in hearing the difference. But iTunes v7.x->11. lossy 160kbps they could tell pretty much right away. (*NB*: _Gear used was my Invicta DAC feeding to both my ZD & AD's, and Electra & SR-009/007Mk1_).
  
 In other words at least in the AAC world, it's my personal belief lossy doesn't always mean it's bad and wish Apple could bring back the old algorithm used in iTunes 4.0.


----------



## shaolin95

brooko said:


> Again - if they weren't from the same source - then you're listening to different masterings.  Follow the link I left you.  The software is all free.  All it takes is time.  The truth is there is you really want to find it


 
 So good to read stuff like this...its sickening to read about people that feel they are super humans able to hear things nobody else can yet when tested they completely fail and of course, they blame the test procedure 
 Good stuff!


----------



## ljhhh

This test perhaps has concluded some results but it is still pretty much inconclusive, mathematically and physically wise.


----------



## skyd171

Lol. Shure SE 846's with a DAC/AMP and addons galore...couldn't tell the difference between any items in group one. I actually thought the 128 was the lossless and the lossless was the 128. Quite funny.


----------



## luberconn

i got them all right.  what do i win?


----------



## KamijoIsMyHero

Experience


----------



## Dark_wizzie

I've been fooling around with OGG recently. I got to like 75kbps with OGG, and it sounded SHOCKINGLY close to my flac originals. I was blown away, and the size of many of the tracks are 1 mb or under! I didn't know this type of feat would be possible. Definitely something mp3 cannot replicate. Going to that final compression setting for mp3 on Foobar kills the sound quality. All of a sudden now I have more than enough space for all of my tracks in my crappy cell phone. MP3's not really "the best" for audio quality at any bitrate/file size AFAIK, so why test with mp3 besides the fact that it's recognizable?


----------



## Head Injury

dark_wizzie said:


> so why test with mp3 besides the fact that it's recognizable?


 
 Precisely this reason. You want to test using a format people are familiar with, otherwise they either won't learn anything from the test or they'll just have another "flaw" to pick at ("Sure I couldn't tell the difference between these files, but the differences between my FLACs and 320 kbps MP3s are like night and day!")
  
 So in most cases this means testing with MP3s (the most widely available and used format, for better or worse) or AAC (iTunes, for worse 
	

	
	
		
		

		
			





). Or, for personal tests like yours, whichever format you want to use!
  
 I'm thinking of re-encoding my portable songs to mid-level OGG files to save space, because while it's still plenty transparent to me I don't feel comfortable going below V2 with LAME.


----------



## CharlesC

If I said that I can tell the difference between 256k aac and alac what I would mean is that sometimes I can tell the difference. Doesn't a test like this require you to be able to tell the difference consistently to pass? It seems that even if you can only tell the difference 10% of the time then its worth it to rip in a lossless format.


----------



## CharlesC

One more thing - anyone with a $2000 mp3 collection is also going to be biased in that they want to believe that their files are just as good as lossless files. Blind testing does nothing to mitigate that bias, so any statistics derived from this will necessarily be pessimistic.


----------



## skyd171

That's assuming anyone participating actually has a $2000 mp3, or very expensive, mp3 collection. I think it's safe your concern is mitigated by the fact that most here are audiophiles and want to experience the best sound, and would want to hear a difference. I personally did. But as mentioned I have a $3k+ setup, SE846's with audioquest victoria interconnects,dac, amp, tube amp, etc etc.... I did several listens and actually guessed that the 128kbps was the lossless and lossless was 128. I fancy myself an audiophile and have been a musician since my teen years.


----------



## CharlesC

skyd171 said:


> That's assuming anyone participating actually has a $2000 mp3, or very expensive, mp3 collection. I think it's safe your concern is mitigated by the fact that most here are audiophiles and want to experience the best sound, and would want to hear a difference. I personally did. But as mentioned I have a $3k+ setup, SE846's with audioquest victoria interconnects,dac, amp, tube amp, etc etc.... I did several listens and actually guessed that the 128kbps was the lossless and lossless was 128. I fancy myself an audiophile and have been a musician since my teen years.


 

 If someone starts off as a teenager spending $200 a year on mp3/aac files then ten years later they have a $2000 collection.  If at that time they become interested in improving their sound quality and invest in better listening equipment, it's not that hard for me to imagine that they have a bias toward hoping their collection is top notch and that they don't have to think about repurchasing music.  This doesn't seem terribly farfetched to me. My point is simply that there is an assumption here that only the lossless lovers have bias but it seems wrong to suppose that the lossy lovers don't have any.
  
 Personally,  I'm very confused by the whole thing.  Mark me down as one who does terribly on AB testing but continues to believe there is a difference. True, my confidence is not all it was before but it has lead me to think that there may be some problems with AB testing - such as: 1) you should only test music where you already think you can hear a difference - what's the point of testing anything else, why should you be expected to hear a difference on a piece of music where you never claimed to hear any difference in the first place?  2) listening to a small sample of music over and over again may well blind you to subtelties - I don't think you can listen to something 6 times in a row and expect to hear anything useful.  It's like food:  the first time you taste something it may be spectacular, but each time you eat the same thing over and over it loses its special quality - and not because the food has changed.  We're talking about a subjective experience of an intangible phenomena.


----------



## luberconn

kamijoismyhero said:


> Experience


 
  
 i actually didn't get very many of them right.  i pretty much just mess with 320kbps via google play all access.  good enough for this guy.


----------



## Dark_wizzie

I can't successfully abx 320kbps MP3 vs FLAC but I'm still storing my music in FLAC because I can. Life is good.


----------



## Brooko

charlesc said:


> Personally,  I'm very confused by the whole thing.  Mark me down as one who does terribly on AB testing but continues to believe there is a difference.


 
  
 Don't worry - this is very common among the community here.  Despite the actual evidence when we test properly, there are still many people who want to believe there is still an audible difference. I guess some of it is because of the large numbers of people on this site who claim lossless as superior - without actually testing properly.  Crowd/peer think.  The desire to conform.
  
 My advice - if you are more comfortable with lossless - go lossless.  BUT ...... when threads come up comparing the two, or asking if there is a difference, simply refrain from posting. One of the biggest issues remains that people continue to perpetuate a myth (audible difference for proper high bit-rate lossy transcodes) despite evidence to the contrary.  The difference between Head-Fi and sites like HydrogenAudio is that over there  - the expectation that differences are audible is extremely low - and burden on those that claim to hear differences must be proven.  Over here - anecdotal evidence runs rife sadly.
  
 Ultimately though - listen in the format that makes you happy.  Just be careful with any claims to others.


----------



## Tempo

The 1st set is the easiest one. Perhaps, it is because the most of instruments used in this song are acoustic.
 The 3rd set is the a little bit harder.But by comparing the very end of this song, You can tell the difference easily wih various perfomance at the high pitch.
 The 2nd setis the most difficult one to me. Maybe it is  because everything is electronics. By comparing 3 files from 0:14 to 0:25, I can only say one of them generate a firmer sound. However, I think I find out the pattern of the answers.
	

	
	
		
		

		
		
	


	



  


Spoiler: Warning: Spoiler!



1. b>a>c
 3. c>b>a           
 2. a>c>b
 see! "[size=1em]randomly scrambled"!![/size]


----------



## itszvz

Lol, I can't tell a difference between any of them. I tried group 3.


----------



## shanky

damn i mixed up group 3's 320 and lossless. got the rest though ! woohoo!


----------



## bzippy

just found this older thread. i tried testing with foobar2000 w/ ABX Comparator plugin and i admit i can't hear any difference at all. i didn't even bother completing the tests because i knew i would just be guessing.
  
 so is it my equipment? my ears? is it simply not possible to hear the difference? i do have mild tinnitus so maybe that's the issue? well whatever the deal is, i may have wasted my time and memory re-ripping all my music lossless.
  
 my rig:
 Asus Xonar DX sound card (Cirrus-Logic CS4398 DAC) in HiFi mode (i.e. all processing off) LINE OUT >> FiiO E12 >> Sennheiser HD600
  
 maybe next i'll load them onto my FiiO X3 and see if that changes anything.


----------



## krismusic

bzippy said:


> so is it my equipment? my ears? is it simply not possible to hear the difference? i do have mild tinnitus so maybe that's the issue? well whatever the deal is, i may have wasted my time and memory re-ripping all my music lossless.



I apologise if this is off topic. I am thinking that the fact that I have Tinnitus effectively excludes me from making audiophile judgements. The interaction between equipment and my Tinnitus is likely to be a greater factor in my perception than other subtleties of sound.


----------



## sonitus mirus

krismusic said:


> I apologise if this is off topic. I am thinking that the fact that I have Tinnitus effectively excludes me from making audiophile judgements. The interaction between equipment and my Tinnitus is likely to be a greater factor in my perception than other subtleties of sound.


 
  
 Why don't you take the Philips Golden Ear challenge?  This could at least provide you with the knowledge that you can identify basic parameters of sound qualifications with some reference.
  
 https://www.goldenears.philips.com/en/introduction.html
  
 How individuals hear is subjective.  Our brains decipher what we hear, and there are a multitude of factors that could influence what we hear and how we describe it to others.   You can always qualify your opinion by stating that you have Tinnitus, but you might want to include the frequency of the tone that inflicts you.  It probably does not matter, but at least this gives the reader a fundamental benchmark.


----------



## krismusic

^^^ Good call. Thank you.


----------



## krismusic

Doesn't seem to work on an iPhone.


----------



## sonitus mirus

krismusic said:


> Doesn't seem to work on an iPhone.


 
  
 I can get the challenge tests to work on my Android LG G2 with the Chrome browser.  Perhaps the iOS version of the Chrome browser will work for you on the iPhone?


----------



## krismusic

Yep. I'm using Chrome. Either I am being dense or it doesn't work on IOS.


----------



## krismusic

Ah ha. Working. It's official I have tin ears!


----------



## DiscoProJoe

I was using my V-Moda M-100 headphones plugged into my laptop, and played the test files through Winamp.  I was using a slight treble boost on the Winamp EQ with an equivalent precut (as I normally have it set), and that's it.
  
 Anyway, my guesses were as follows:
  
  


Spoiler: Warning: Spoiler!



1:
 128
 FLAC
 320
  
 2:
 FLAC
 320
 128

 3:
 128
 320
 FLAC


  
 I correctly guessed the FLAC file on each set, and got the 3rd group right. But the acoustic guitar in group 3 made it really hard for me to make out any difference, so I was basically guessing on group 3.
  
 I felt somewhat confident about my answers on group 1, but got the MP3 bitrates wrong.
  
 Group 2 I almost got right, but didn't realize it had a trick to it.
  
 Prior to this test, I thought I'd be able to make out the 128 file somewhat easily, but couldn't always guess it right.
  
 Anyhow, I am _not_ an audiophile and am proud of my music collection of almost 10,000 songs, nearly all of which are encoded as MP3s at my chosen bitrate of 192 kbps, so this test doesn't ruffle my feathers in the least!


----------



## Vorpax

My results:
  
  
  


Spoiler: Warning: Spoiler!



first group: A 128kbps                  B 320 kbps 
                  C flac
  
 second:      A flac
                  B 128 kbps
                  C 320kbps
  
 [size=1em]third:   A 128kbps[/size]
 [size=1em]          B 320kbps[/size]
 [size=1em]           C flac[/size]
  


 I messed up the first group entirely it seems 
	

	
	
		
		

		
		
	


	




 
  
 On the second group, the flac was pretty obvious to me. More impact, juicy sound. The 128 and 320 were guesses, but both clearly inferior to the flac, they sounded thinner.
  
 About the third group, the 128 kbps one was easy. More liquid and opaque than the other two. Getting flac right was a bit difficult on this one, but I don't think it was a guess, the low level detail sounded more rich. Flac overall not too different from 320kbps like in the second group but still.
  
  
 I'm using hisoundaudio Crystal, onboard sound card (Realtek ALC892) and windows media player LOL!
  
The first group aside, I'm happy that lossless is not placebo effect for me even with no external DAC nor high-end headphones.


----------



## katulu

Spoiler: Warning: Spoiler!



In order of what I think is increasing quality (128, 320, lossless):

 1 - C, A, B
 2- Cant tell, music sounds like noise...
 3 - A,B,C

 Wonder how I did...


 ok...?
  
 Looks like I got groups 1 and 2 right, at least. For me, the "tell" on 1 was the snare, on 3, higher note picking at the end.
  
 I kind of cheated in a sense - I always listen to my music EQd to the headphone I am using - don't know if that counts as a "bizarre" EQ.  Chain was Stream 7 Tablet, Jetaudio, Equalizer APO with custom settings, Normal USB to Chord Hugo, Norne Draug 2 cable, HD800s.


----------



## katulu

After reading some responses, some more thoughts:
  
 Like most, experience is a key factor in telling differences in files.  If you have read/experimented a little, you may know some of the usual flaws of lossy files - watery cymbals, smeared instruments. But one thing that baffles me is that in these tests, the factor of familiarity is never considered. Let me explain what I mean:
  
 I like steak, generally cooked medium. But I know when it's undercooked, or overcooked. All the music I know and love is my steak.
  
 Now, someone serves three plates of Dodo meat, and asks me to say which is overcooked, undercooked, or just right. What???
  
 I can use the principles of my previous experience to try to tell, but such as I have experienced before may not be present - thus I fail the test.
  
 It amuses me to think about creating some music which would purposefully sound liquidy and smeared, and which perhaps could somehow "clean up" with the linearization of lower bitrate! Experts would be baffled at realizing the "worse - sounding" file is the most correct one...
  
 If I don't know exactly how a file is *supposed* to sound, I can't really be expected to know which one is more correct. One is more readily able to tell quality differences when one has a baseline, I believe. I seriously think tests results would be quite a bit different with songs "everyone" knows...
  
 And this is without going into the details of music which may not be losing that much even when coded into lossy format... can the listener then be blamed? I saw one such test online a while ago, trying to prove no one could figure out where a file was lossy or lossless, and it proves nothing, in my view... I did a brief comparison of Morbid Angel's Covenant album 320 and FLAC, which I know well, and to my surprise the difference was fairly obvious - other more complex extreme metal is even more abhorrent of lossy files. I have likewise been surprised at how good some 192 kbps music has sounded too.
  
 Oh, I also forgot to mention I have tinnitus and hearing loss of higher frequencies in my right ear... and apparently I can still tell differences in SQ.
  
 Just some cents... Cheers, everyone.


----------



## gixxerwimp

katulu said:


> After reading some responses, some more thoughts:
> 
> Like most, experience is a key factor in telling differences in files.  If you have read/experimented a little, you may know some of the usual flaws of lossy files - watery cymbals, smeared instruments. But one thing that baffles me is that in these tests, the factor of familiarity is never considered. Let me explain what I mean:
> 
> ...


 
  
 +1 ... my feeling on these tests exactly. Haven't taken this one yet, but taken a whack at many others. Without being familiar with the music, it's really hit and miss.


----------



## Sonic Defender

gixxerwimp said:


> +1 ... my feeling on these tests exactly. Haven't taken this one yet, but taken a whack at many others. Without being familiar with the music, it's really hit and miss.


 
 Nope, shouldn't matter, if the differences are audible they are audible. If you need to study and train your brain to detect minute differences then for all practical reasons there is no difference. People don't train their ear, they listen to music. I still suspect that even with music you are very familiar with in multiple trials the results would be only marginally better. Today at our head-fi meet I had seven people do multiple trial tests between 320 and ALAC and nobody did better than 50%, and this included three people that swore they had proven in the past the could hear the differences. Hopefully nobody comes along with the standard claim that the gear must have not been resolving enough, the gear wasn't the issue.
  
 One participant did his 5 trials with the test gear and then wanted to use his T1 and PHA3 for two more trials. By this point he was quite familiar with the piece and big surprise, 50% accurate again. We will be doing this again to develop a bigger sample size, but with such a low rate of detection (50% is just guessing) the results are extremely unlikely to change. People would need to be detecting the differences at about 90% accuracy to really be in the area of statistical confidence.


----------



## katulu

sonic defender said:


> Nope, shouldn't matter, if the differences are audible they are audible.


 
  
 It does matter, because that 50% is not whether people *can* detect a difference (yes or no, 50% probability - right?), it's whether they can identify the "better" quality given they say they detect a difference. If a person isn't familiar with the music, there is no baseline to say which is better quality - which makes that 50% number quite explainable. See my previous post for reasoning. Thus a test where the listener does not know the piece as it's supposed to sound is heavily biased in my view.
  
 And of course, it's BS to use musical pieces where encoding has negligible effect to "prove" bitrate doesn't matter/can't be detected. Not all music is equally affected


----------



## Brooko

katulu said:


> It does matter, because that 50% is not whether people *can* detect a difference (yes or no, 50% probability - right?), it's whether they can identify the "better" quality given they say they detect a difference. If a person isn't familiar with the music, there is no baseline to say which is better quality - which makes that 50% number quite explainable. See my previous post for reasoning. Thus a test where the listener does not know the piece as it's supposed to sound is heavily biased in my view.
> 
> And of course, it's BS to use musical pieces where encoding has negligible effect to "prove" bitrate doesn't matter/can't be detected. Not all music is equally affected


 
  
 Seeing as how we are in the Sound Science section - and because you are so confident, how about a little test (if you'd be willing to participate, and if we could set it up)?
  

You provide the files - whatever music you like
You provide own headphones
We arrange a DAC and amp that you're happy with
We arrange a third party observer - give him/her instructions to properly dither and resample the file to aac256
We use Foobar2000, volume match precisely, and blind abx - observer present to ensure everything above board
2 lots of tests 15 - in each.  Because it's music you know well, and headphones you are familiar with, you should ace it easily.  We publish results
  
 If all of this could be arranged - would you be game?


----------



## Sonic Defender

katulu said:


> And of course, it's BS to use musical pieces where encoding has negligible effect to "prove" bitrate doesn't matter/can't be detected. Not all music is equally affected


 
 So you know what music is appropriate to use? Can you describe what the fundamental characteristics are and how this has been demonstrated? The music piece we used is a fantastic recording by Holy Cole, One Trick Pony. The song has smokey female vocals, nice drums, acoustic bass and very nice guitars. It is extremely well recorded and dynamic so I rather doubt the piece of music wasn't up to snuff. As well, people were asked what version the preferred not whether or not the could hear a difference. The logic there is that everybody who assumes that a 320 file is sonically different implies it is different in a negative sense; therefore, you would of course by that logic prefer the lossless file.


----------



## castleofargh

there is nothing wrong with trying to succeed with the proper files, the proper headphone, and taking all the time needed to train and get used to whatever it is that can be heard. maybe that will end up working great, maybe not. as an experiment I see no problem with this.
 now as a representation of reality, of course it's total nonsense. what matters to me as an individual is to know if while I'm on a train or walking on the street with my IEMs that rolls off after 12khz, I would ever notice that the music is "wrong" because it's 320kbps mp3? and I believe I've answered that question for myself years ago.
	

	
	
		
		

		
			




  
 when at home on the computer, with a relatively silent room and speakers giving me more trebles than I can hear, I fail my ABX tests. so while walking on the street it would be real misplaced pride to pretend like 320mp3 ruined my sound.
 now at home again(just when I thought I was out, they pull me back in), I tend to listen to flac, because storage is not an issue so why not? but TBH I often don't pay attention and end up playing mp3 too. never killed me.
  
 IMO increasing sound quality comes from kicking the butt of whoever ordered the sound engineers to mess up records for the sake of being loud. or from finding the signature that is neutral to us, or getting rid of channel imbalance and audible hiss. not so much from worrying about what happens 60 or 70db below the loudest sound of a song. or that one instrument I don't have in 99.9% of my music that's apparently messed up in mp3 at a frequency I can't hear anymore.
  
 mp3 and AAC are practical codecs, and for all intended (space saving) uses, they do a fine job IMO.


----------



## cjl

katulu said:


> It does matter, because that 50% is not whether people *can* detect a difference (yes or no, 50% probability - right?), it's whether they can identify the "better" quality given they say they detect a difference. If a person isn't familiar with the music, there is no baseline to say which is better quality - which makes that 50% number quite explainable. See my previous post for reasoning. Thus a test where the listener does not know the piece as it's supposed to sound is heavily biased in my view.
> 
> And of course, it's BS to use musical pieces where encoding has negligible effect to "prove" bitrate doesn't matter/can't be detected. Not all music is equally affected


 

 This is why an ABX test is better than a "which is better" style blind test. If you are given two reference samples, A and B, and then you have to match your unknown sample to one of them, you don't have to know whether A or B is the better one, and therefore you don't need to have any particular familiarity with the piece. Just listen to A and B until you're confident you know what to listen for to tell the two apart, then listen to X vs both of them and see which it matches.


----------



## sonitus mirus

When I ABX now, I make a 96kbps mp3 to compare with a lossless version first.  I find where a specific artifact can be identified, and then I try and listen for that same artifact at the resolution that I typically use.  I just waste my time scouring a track for any differences, otherwise.
  
  I'm assuming that any problems would be amplified at a lower bit rate, but my understanding of the encoding process is vague at best.  I say this as I was able to increase the bit rate in steps and the anomaly I was hearing at 96kbps could still be identified in an ABX test up to 160kbps, at 192kbps it was not statistically valid, but the results suggested I probably was not guessing, and at 320kbps I could not hear any differences in that particular section of the song.
  
 When trying to ABX 2 very good files, I just want to focus on a few seconds of a track that I know have problems at a lower bit rate.


----------



## Satsugai

My take on group 1:
  


Spoiler: Warning%3A%20Spoiler!



1_A: 320 kpbs. Sounded very good, but not quite as good as 1_B. Sounded slightly more "flat" when compared with 1_B, to my ears.
 1_B: Lossless. The instrument seperation seemed much cleared to me on this file. Strum quality was quite clear and natural sounding.
 1_C: 128 kbps. Sounded more flat, with degradation of the highs when there were multiple instruments playing at the same time (most noticeable at 0:16-0:20)


----------



## watchnerd

Which MP3 codec was used and which options were selected when making the two files?


----------



## sonitus mirus

watchnerd said:


> Which MP3 codec was used and which options were selected when making the two files?


 
   
 Here are the details:
  
 Quote:


> http://www.head-fi.org/t/646411/lossless-vs-128kbps-mp3-vs-320kbps-mp3-blind-test/75#post_9362033
> 
> I used the most recent release of LAME. 3.99 I believe, as it hasn't been updated since October 2011. I can double check to make sure when I get home but I didn't install the software until about a month or so before the test so I'm 99% sure here.
> 
> It was set to slow speed, high quality. Full stereo. Basically I didn't change anything from default other than the quality slider.


 
  
 dBPoweramp was the application, but I didn't see which version was used.


----------



## watchnerd

sonitus mirus said:


> dBPoweramp was the application, but I didn't see which version was used.


 
  
 Did you use any of the LAME presets (e.g. V0, V2, etc) or did you just set it all by hand?


----------



## starcraft2

watchnerd said:


> sonitus mirus said:
> 
> 
> > dBPoweramp was the application, but I didn't see which version was used.
> ...




I can't tell the difference in ABX tests between 320mp3 and lossless files. Should I feel bad?


----------



## hotteen

This is difficult!


----------



## ExtremeGamerBR

starcraft2 said:


> I can't tell the difference in ABX tests between 320mp3 and lossless files. Should I feel bad?


 
  
 Nop. This is almost impossible in many many cases.


----------



## musical-kage

Depends on the ripped source though.

For example I've used Spotify on max (320kbps ogg) for years.

On a trial right now of Tidal lossless, and there appears to be a massive difference


----------



## starcraft2

musical-kage said:


> Depends on the ripped source though.
> 
> For example I've used Spotify on max (320kbps ogg) for years.
> 
> On a trial right now of Tidal lossless, and there appears to be a massive difference




How do you tell the diffence? Soundstage?


----------



## sonitus mirus

starcraft2 said:


> How do you tell the diffence? Soundstage?


 
  
 It's simple.   One is $9.99 and the other is $19.99.


----------



## Brooko

musical-kage said:


> Depends on the ripped source though.
> 
> For example I've used Spotify on max (320kbps ogg) for years.
> 
> On a trial right now of Tidal lossless, and there appears to be a massive difference


 
  
 Probably different masters being used.


----------



## NaiveSound

How does one get free or heavily discounted tidal


----------



## chef8489

naivesound said:


> How does one get free or heavily discounted tidal


 
  
 Serving in the military or being a student is one way.


----------



## NaiveSound

chef8489 said:


> Serving in the military or being a student is one way.





I'm currently a student at a technical school... Wish I could use it but it don't work. 

I feed Tidal hifi to mojo... I wonder if a cheaper alternative might be just as good? No?


----------



## chef8489

naivesound said:


> I'm currently a student at a technical school... Wish I could use it but it don't work.
> 
> 
> 
> I feed Tidal hifi to mojo... I wonder if a cheaper alternative might be just as good? No?



 

No, but you can register for tidal as a student for 9.99 a month.


----------



## castleofargh

this topic is about mp3 bitrate in a blind test, if you don't mind going there http://www.head-fi.org/t/780051/tidal-vs-spotify/120 to keep discussing tidal/spotify matters, that would be nice for future readers.
 thanks


----------

