Sunday, 26 July 2015

I read the news today, oh boy

Earlier this morning, I wanted to catch up a bit on news, and as usual, it took entire seconds to find delightful examples for horrible software defects that should never exist to begin with. The most interesting parts of this (my personal selection, YMMV) mess for today are the following.

Fiat Chrysler Automobiles (FCA)

Ever since our beloved car manufacturers decided to include software in their products, they bless us with wonderful tales of software bugs, almost on a day to day basis. Last week, FCA recalled 1.4 million cars in the US alone, because it turns out that essential functions of these cars can be remotely controlled which includes taking over steering and braking allowing for the possibility of killing the passengers. Mass recalls of so many vehicles is certainly an expensive undertaking, so for us customers, for the sake of transparency, it would be interesting to know how many percent of the price of a new car goes into software incompetence.

ERGO insurance group

One of Europe's largest insurance groups, ERGO, recently admitted that they have been using faulty  life insurance calculation software for decades and now put substantial effort into correcting the calculation for a noticeable number of customers, 350000 so far (German only article, sorry). According to this article, some calculations were off by a 5-digit figure. Peanuts, eh'?


It always saddens me to read of screw-ups committed by Apple, because I am positively biased towards their products, I have to admit. This makes it much more emotional than reading about issues at, say, ERGO, because I would blindly expect them to screw up all their software. In any case, Apple makes it twice into today's list.
  1. A recently published exploit by Stefan Esser allows any user on an OS X 10.10 "Yosemite" to elevate the own permissions to root. The fun part here is that
    1. The exploit is so tiny, it fits in a tweet, and is a mere shell command, and
    2. There is no indication from Apple that they intend to fix it in Yosemite.
  2. After the rant about music streaming I published yesterday, I was secretly hoping for Apple Music to suck less than the other services when it comes to the software. After reading how Apple Music (or rather quitting it) may destroy the rest of a user's library, I should probably bury my hopes six feet under.

Saturday, 25 July 2015

Music streaming sucks - Summer 2015 edition

Music streaming sucks. I'm back to buying (CDs).

Abstract ("TL;DR" nowadays)

Music streaming services could be a wonderful thing for the customer. But they aren't. Stupidity, greed and incompetence from various stakeholders in the service delivery chain ensures that the experience for the customer becomes insufferable. After two years of paying for streaming, sending bug reports over and over, scripting workarounds for shortcomings whilst hoping for improvement, I'm done with streaming until someone gets the service right.


It must have been about 15 years ago when my flatmate showed me this new, "awesome" software called Napster. He looked up a track via some keywords, found it in the list, started the download and only 25 or so minutes later (internet was that slow back then) he had an goddamn-horribly-terribly-awfully-sounding 128kbit mp3 file on his computer (mp3 encoders were in their infancy back then). Apparently I was ahead of my time: "Dude", I said to him, "this is shit. I can't listen to this. When it comes to music, quality matters to me. Why don't the music companies have this as a service to pay for and provide decent quality? I guess 15 DM per month would be fine with me." (We had the Deutsche Mark currency back then.). Then it took the music industry only a decade to come up with this business model that was completely obvious to every student in the world 15 years ago. Well done guys! After some early adopter's years, we are at the top of the hype cycle and have more streaming services that we need, and the whole business stinks. Here is why:

Part 1: Streaming services reviews suck

I don't know how many reviews I have read in the past years regarding streaming services. I suppose it is somewhere in the range of 40-60. Not a single one was actually helpful in determining whether I should subscribe to a particular service or not. Some reasons why they are not helpful and hints for improvement:
  • Reviews typically state that "all tested services had 25-30 million tracks available, so there is no real reason to complain" and that's about the depth of the streaming catalogue analysis. That assessment could not be more wrong. In fact, all streaming catalogues suck (see below), so the reviewers should really do their homework before over-simplifying the situation to the disbenefit of their readers.
  • Most (English-language) reviews seem to intend to address the whole world as potential readers. In this case, it does not make sense to consider services that are only available in the US. If the article is intended for the whole world to read, compare services that are available close to everywhere. Wikipedia keeps a list of those:
  • Most reviews draw conclusions after about a week of usage. This is too early to discover longer-term issues. For instance, changes in the streaming catalogue usually happen less frequent, so do updates to the client software. I'd argue that you really cannot draw useful conclusions with less than 3 months of testing. Hence, the 3 months trial that you can get from Apple Music and TIDAL are intriguing and deserve to be mentioned here.
Having said that, there are of course reviews that are extremely helpful in understanding the business or particular parts of it, especially when reviewers have received insider information about a service. From those we learn, for instance, that TIDAL has no plans of improving its sucking Mac/Windows client UI in the near future. Awesome! Let's move on.

Part 2: The streaming catalogues suck. Yes, they do.

Why on earth should a statement like "we have 30 million tracks" be relevant for any streaming service decision? How many songs do you really like? If you are like me, you are not even in the 10000s range. Or maybe you are, but you certainly don't tell your friends "Well, there are 12554199 songs I love, and I am going to enumerate them for you...", right? If a service has 30 million songs and 29.995 million of them are Mongolian medieval chanting, then the service is probably not the right one for me. So the absolute number is basically irrelevant. But, in fact, most services have a decent coverage of contemporary music. This, I suppose, is what the reviewers (see above) mean when they state "you cannot complain." But most relevant, in my opinion, are two questions:
  1. How well does the service cover rare tracks, special tracks and whole albums? and
  2. How frequent is content that you love removed from the catalogue?
With respect to those two questions, I am sad to say that all streaming services I have looked at suck utterly and completely. Let me elaborate. I suspect most people opting for a streaming service do so under the same premise I did: I am totally happy to pay you guys 10 bucks each month for the rest of my life if I never have to buy music again. Streaming providers support this idea of having everything you love readily available in their client software: You can add tracks, albums or artists to My Library, My Music, or whatever your streaming provider's software chooses to call it, the concept is always the same. It is the cloud-/streaming-service based representation of what used to be your stack of vinyls, cassettes or CDs in your living room. But it isn't. Because streaming services provide horrible answers to aforementioned questions 1 and 2. Specifically:
  1. While fragmentation and exclusive offers over the different services is clearly stupid (why is Coldplay's Mylo Xyloto not on Google Play Music, but on Spotify? Do music industry executives seriously believe that everyone who wants to listen to Paradise once a year subscribes to a second service just for that?), things are, in fact, much worse: In 1993 I bought Paul McCartney's Off The Ground, which is available on essentially all streaming services today; with the exception, of course, of the album's most popular track, Hope Of Deliverance, which is not available on ANY streaming service. (It is also not on Spotify, my dear Spotify friends. When you play the track that is wrongfully listed as Hope Of Deliverance on Off The Ground, it will play some live version of the track, not the original studio recording. More on that later in the section on sucking software.) Same for Rise by Gabrielle, a huge hit in Europe from the year 2000, and several others.
    It does not get any better when searching for tracks that you can only remember as hits from childhood, for example The Captain Of Her Heart by dutch band Double. Apparently the music industry hates this track so much that they want to eradicate if from existence. You can't buy it or stream it anywhere today, except via 20-years-old second-hand-CDs.
  2. Removal of albums from the catalogue does happen. Frequently. All the fu**ing time. In fact, although I have only some 100 albums in "My Library" on Google Play Music, I am still waiting for the first four-weeks-period since I started using the service in which my library is not affected by album removals. This is not only annoying (because tracks/albums just silently disappear from playlists as well as "My Library", without notification whatsoever, see the section on sucking software below), it is entirely unacceptable and renders "My Library" into "Someone else's library that can change any time whenever this someone else feels like it". This simply sucks, and there is no way to talk a paying subscriber into believing that this is a good thing. What would you say if you came home one day and two CDs from your stack were gone? Probably something like "burglary". And what would you say if a sampler-CD you have burned for your old car stereo spontaneously decided to contain only 18 tracks instead on 20 because it is the first of August? I mean, WTF??? I am paying the monthly subscription fee with the intention to listen to Monaco's What Do You Want From Me? whenever I want, not whenever that spooky somebody else does. Sometimes, admittedly, after weeks or months, albums return to the catalogue, at which point it is expected of us subscribers to discover that on our own and add them manually again (see the section on sucking software below). That recently happened to Platinum Collection by Genesis; do I have to listen to the remastered Paperlate now for 5000 times in the next couple of weeks before it gets removed again or can I relax about that?
  3. Strategic removals also happen all the time. An album that was thought to be a flop but then gets streamed more often than expected? Happened to Mark Knopfler's Tracker only a few weeks after its release earlier this year. It's then removed from the streaming catalogue, so maybe more people buy the super-duper-deluxe-limited-digital-sleeve-bonus-tracks edition for 30 bucks (and never listen to it). You get the picture. This business model is well known from narcotics sales. The first fix is free (in this case: part of the monthly subscription fee that you are already paying), but when you become addicted, it starts to cost extra.
Fact is, the volatility of the streaming catalogues entirely void the concept of "My Library", "My Music" or whatever your streaming provider wants to call it. It's not YOUR library, it's someone else's, because it's not YOUR decision when albums get removed from it. Unless there comes along, no not Mary, but a commitment from the music industry and streaming providers that My Library will not lose tracks without my specific request, it remains unacceptable for the music lovers among us who invest time, and, obviously, money, to build up their libraries.

Part 3: The software sucks

I would talk about how the frontend software sucks, but that would be only half of the story; because the backend software sucks, too. I have not checked all the services yet, so the lamerosity-points you find below are restricted to Google Play Music, Spotify and TIDAL (including its predecessor WiMP), and to a tiny extent, Deezer, which is the service I would like to begin with:
  • After selecting Deezer Music and clicking on "install" in Google Play Apps, I am presented with a request from the app that truly stuns me. Hidden in "Other [permissions]", Deezer requests the permission to delete other apps from my phone. Are you guys insane? Is this some kind of twisted reinterpretation of the second commandment? Thou shalt have no other music player before me? Sorry, Deezer, but first you need to give me an Android app that implies some mental sanity in your R&D team, then I might consider test driving your app.
  • We have talked about consistency before: Google Play Music and TIDAL just don't have Paul McCartney's Hope Of Deliverance, while Spotify thinks it has it, but doesn't. This implies that there is something wrong with the identification of unique tracks at the back-end. I would love to spend some time with the dev team of one of the services (Spotify et al: You hear the cry. Give me a call!) because at this point I only have an educated guess: Not all music providers have a persistent, unique mapping for the relationship of all their songs to the list of albums. Let me explain what I mean:
    Consider an artist who has released two albums, and had 3 huge hits. Subsequently there are ten best-of-releases over the next 20 years. Also the artist will be featured on movie sound tracks, best-of-year-20xx editions, and so on. Essentially it's always the same song, and one could simply link it to all releases where it is used, e.g. via meta-data. Presumably the music industry just does not do that consistency, so the R&D teams of the streaming providers face two alternatives:
    1. They can just leave it alone and don't do anything. The track will not be displayed as part of a specific album, even if it is part of another album in the catalogue. In Paul's Hope Of Deliverance case, this is what you want. For other tracks that are part of the catalogue, this may cause an issue: It might not be listed on all the albums that should contain it, but only those where the incomplete links work. This seems to be what Google and TIDAL do.
    2. They can match patterns. For example speculating something like "if two tracks are named <TrackName> by <Artist> and are <N> seconds long, then it is the same track." and use this information to add the missing links. I believe this is what Spotify does. The catalogue looks more complete, because there are less missing tracks e.g. in best-of-collections. In most cases this works well, but occasionally the matching is too fuzzy and you get served a wrong track, as in Paul's Hope Of Deliverance case.
    Again, this is speculation. I have not seen the code of any of the backends. I am just inferring from the symptoms. Nevertheless, for me, a streaming service user with a mild OCD, the implications are sad:
    • The availability of a track on an album does not guarantee that it's really the track it is supposed to be. One has to listen to them to figure it out. Otherwise, one might get disappointed by an offline playlist on a train.
    • The unavailability of a track on an album does not tell me that the track is not part of the catalogue. Again, I have to search and listen to multiple candidates and maybe find it.
  • Playlist and Library management is physically painful. That applies to all services I tested. I have discussed this with several friends of mine who repeatedly told me that this is a point almost nobody cares about, except those OCD people like me. Presumably I am a dinosaur on the verge of extinction, so it might not be a problem for you. Judge for yourself:
    The clients do not deal gracefully with the removal of tracks from the catalogue. With Google Play Music, when an album gets removed from the catalogue, either one of the following can happen:
    • The album disappears from "My Library", and all tracks that were part of some playlist disappear from said playlist. No notification, no way of knowing other than either
      • checking your stuff manually every day, notice that a playlist that yesterday contained 87 tracks now only contains 86 tracks. With Google Music, the only "official" way to find out what happened is to use a web browser and a mobile device and exploit a bug in the mobile app: The mobile app updates its view of the world via incremental syncs. It seems Google has not implemented a command for "this track was in the catalogue yesterday, but now it's gone". Thus, the mobile apps do not notice when an album is removed, and it would still show the 87 tracks playlist. Then, you can compare the track listing on the mobile device with the web browser and identify the missing tracks. Of course, you then have to manually remove that track from the mobile device's playlist to get your data consistent over the various devices you own. If you have 10 mobile devices, have fun doing that every fu**ing week. Oh, and don't use iOS. Google Music for iOS does things to your playlists, especially the "Thumbs Up!" list that not even your worst enemy will do to you.
      • writing a python script that detects these changes for you using Simon Weber's excellent google music API on github. That's what I did after the n-th time the problem occurred.
    • The album disappears from "My Library" but no tracks are removed from playlists. In this case, you find out by more advanced scripting and Simon Weber's google music API. Or, of course, you play your favourite playlist and instead of playing one of the tracks in question, it just tells you "Can't reach Google Play Music at the moment." Great stuff Google, that's how library management is done!
    • Nothing disappears, neither from playlists nor from My Library. Scripting does not help here, manual checking also does not help. The only thing that helps is playing an album and noticing that every track is skipped until an error message pops up.
    • While I have scrutinised Google Play Music the most, I observed similar annoyances in Spotify and TIDAL (and its predecessor in Europe, WiMP). And while there seems to be no problem changing the UI all the time, no action has been taking in fixing the defective playlist / library management functionality for years.
      The bottom line is this: If you give a rat's ass about your playlists and don't have an eidetic memory, you better write scripts. Otherwise it is just a question of when, but eventually all your playlists will be almost empty.
    • TIDAL seems to be interested in maximising the number of clicks on their client UI. Supposedly simple tasks, like rearranging the play queue, or adding multiple tracks to a playlist, require a ridiculous number of clicks. It is almost like they don't want the users to use those features. And thanks to the quite useful article by James Murphy ( we know this UI is going to stay sucking for a while. Bad news for TIDAL subscribers here.
  • Offline playback (not referring to local files here, only to files that have been 'made available for offline use' via the respective apps) on mobile devices is painful. It seems to come as a surprise to streaming software services, but mobile internet is nowhere near where it should be. We are in desperate need for a great offline mode, which is nowhere to be seen. For instance:
    • Google Play Music always attempts to fetch artists's latest pictures and cover art whenever it identifies something that resembles an internet connection. In Germany, this means whenever I am in a car outside a major city, and I have only 2G connection, I have to wait for a minute network timeout each time I click on an artist or album, before I can press the Play button, even if that album is already available offline. This is an app for the patient friends among us.
    • TIDAL (when put into its obscure "offline mode") is significantly reduced in functionality. "My Music" is unavailable, most interesting functions are missing. The app feels a bit too minimalist. But this has one huge advantage: Access to stored offline playlists and albums is quick and painless. The disadvantage: It won't play your local files and you have to switch to a different app for that.
  • Caching of streamed files is not handled well in all services. This is something where Google's app shines. It caches, reducing network load and (maybe more importantly) is easier on your mobile quota. Also it allows to manually clear the cached content in case you need some space for something else. For the latter feature there has been a petition at Spotify's customer base for years now with no chance of ever being implemented. TIDAL's app seems to do no useful caching whatsoever, hitting your mobile plan hard.
  • Local files handling is insufferable. This may not be a problem for the overwhelming majority of iOS users, as non-jailbreaked iOS tries hard to take the burden of "files" away from the user by pretending they don't exist. But on Android, files do exist and users have file managers, so there are ways to keep your device (at least partially) organised as you wish. So when I choose to put my own music files onto my mobile device, I bloody damn expect the music player apps to play them. Android comes with a bunch of codecs for that. Of course, most streaming services apps don't. Why would they? It would be great for the users, but they wouldn't see additional money for it. Why implement a feature that does not get you any additional money? The notable exception here is Google Play Music which happily plays local files. Playlists consisting of both local content and streaming content works seamlessly as well. Big plus points if you ask me.
  • Mixed business models don't take place. From product management 101 some may remember that exploiting opportunities via obvious low hanging fruits is usually worthwhile. Let's get back to the 'incomplete album' example from above. Remember that Gabrielle's Rise album is available on all major streaming services except for the title track, Rise itself. Now Google has not only a streaming service but also a store where you can purchase tracks and albums. In said music store, you can actually buy Rise. The whole thing. Now when I am playing the album through my streaming subscription, the title track Rise is simply not listed. Wouldn't it make sense to have it listed (but striked through or otherwise indicated that it can't be played via subscription) alongside a one-click-buy-button to sell additional peace of mind to the customer? "Hey, dear customer. Click here, and you will have all the tracks from this album! Also, when we remove it from the streaming catalogue, you will still be able to listen to it! It's going to happen soon!" Additional sources of revenue, isn't that what the music industry wanted? Why is this not happening?

Part 4: The sound quality sucks - says who?

This is actually a part I am siding with the streaming providers. I have been a sound quality nerd all my life, but I never made the transition to the hifi-voodoo camp: I don't hear a difference between loudspeaker cables just lying on the floor compared to them being clamped into tiny iridium-platin bridge poles preventing them from touching the floor, and I find it hard to believe somebody else does.
Anyway, my point is: Despite interesting discussions on how bad digital watermarking techniques impair sound quality (e.g., the sound quality on Google Play Music and TIDAL leaves no reason to complain in my opinion. I did not test Spotify at high quality in the last couple of months, so I can't be sure about them, but especially TIDAL's lossless subscription sounds great, at least with the content I am usually listening to. (Most complaints against watermarking artefacts I found on the web are from customers who predominantly listen to classical music.)
But the fire of criticism gets additional fuel from a famous musician: Neil Young ( pulls his songs from streaming platforms because streaming is "[...] the worst quality in the history of broadcasting or any other form of distribution", Young says. I am not sure whether that's really what he means. This claim states that the standard 256kbit MP3 from Amazon's store sounds better than TIDAL's or qobuz's16bit/44.1kHz lossless FLAC-based streaming. Without suggesting that this was a marketing stunt geared towards his Pono player and the Pono Music ecosystem, that calls for a series of double-blind listening tests.
By the way, it is totally conclusive that professional musicians find it easier to distinguish lossy compressed music from the lossless source; but not because they have an inherently better hearing ability than other people. They were subjected to loud music in their lives, lots and lots of hours of it. A double blind test by the German c't magazine conducted in 2000 already demonstrated that impaired hearing sometimes helps in these tests because the impairment changes how sounds are perceived. Hence, the psychoacoustic model of lossy codecs do not match the perception of the hearing impaired candidate. (German only, sorry:


After almost two years of paying for monthly subscription to music streaming, it's time for me to pull the plug and go back to buying CDs. The painful truth is that none of the fundamental flaws of music streaming has changed. The client software is still horrible without any indication of somebody fixing the bugs. The volatility of the streaming catalogue and the strategic removals of content to squeeze more money out of already paying customers causes the streaming subscription model to taste bitter.
I'll probably keep on testing services every now and then, hoping for improvement. As soon as one service gets it right, I am all yours and happy to pay the monthly fee again.

Wednesday, 15 July 2015

New home!

As a new home for the hashtag #SoftwareÜberfordertDieMenschheit (German, lit. software is over-challenging mankind) which I have been using for quite a while on Google+ to tag stories about software-related issues that are just not supposed to happen (but obviously do), I wanted to quickly find a new suitable story. Given the terrifying state of some of today's building blocks of our information age, it took about 20 seconds to extract more than one suitable example: