Waiting for ages, and it finally happened!

Now this comes as a surprise. After years of demonstrating to the world that they can screw up software in any product, it's finally here:
Samsung executives blame lackluster software for company’s problems.
Admittedly, the timing comes as a surprise. Yes, the phone market has been less fortunate for Samsung compared to previous years, but still ok compared to some competitors. Anyway, recognising and acknowledging a problem is the first step towards solving it.

Browser source tarball size again (hopefully transient)

While I still hope that this is just a minor transient glitch in the tarball's creation process, chromium's source tar size has recently increased from ~327 MiB to 1367 MiB in 47.0.2526.106. Yes, that's correct, we are talking 1.33 GiB now. A quick look indicates that it contains lots of binary files for multiple architectures. Hopefully this gets removed again in upcoming releases. Source tarballs should be just that: source.

Now I am relieved

Praise the magnificent creativity at Google's Android team! Recently, we learned from this article that Android has been an emotional wasteland. Not any more! Thanks to the new and awesome minor Marshmallow release bump (6.0.1) we have more than 200 new emoji. Now I can really express myself. This is the most compelling reason to buy a Nexus device in Europe for a very long time! I'll  quickly browse to Google's device store asap and hope I get one of these precious things before it's sold out!
Sorry, I really could not resist this one.

Music streaming sucks - Winter 2015 edition

Some of you may remember that about half a year ago I wrote a rather lengthy rant about music streaming. The bottom line was: "I go back to buying music."

So far, I didn't change my mind and was quite happy to buy albums as carefully selected as before.

Having said that, it seems a bit as if the major streaming providers are trying hard not to earn any money from customers: Next to the albums I am happily purchasing through classic distribution channels as well as some online stores which sell (Hi-Res) FLAC files, I continue to enjoy streaming without paying a dime so far.
First, I received a mail from my friends at TIDAL, indicating how sad they were to see me leave the service and they are happy to give me another 3 months for free, in which I may consider staying with them.
Then there was a Chromecast offer to test Deezer premium for 3 months. From my last posting it was clear that I had not tested Deezer so far. So I saw this as a good opportunity to test the service and compare it to the ones I have been using for the last two years. So, after two months of using Deezer almost daily, here is my review:

• Summary: Run away!!! Of all services I tested until today, this is by far the worst overall experience.
• Noteworthy pros
• They recognise how important non-music elements can be for a portfolio: There is a recommendation system in pace for audiobooks, which I appreciate very much. Also they offer live streaming (audio only) of Bundesliga matches, which my lady appreciates.
• It streams to Chromecast.
• Noteworthy cons
• Playlist management is barely existent. It is so bad in fact that my current hypothesis is that they want the users from maintaining playlists. This is actually understandable, because if you don't have any playlists, you don't notice the volatility of the content catalogue (which is still my main point why streaming in general sucks).
• The Apps (Android, iOS) are the worst kind of software engineering that I have seen for years. Spontaneous bluetooth disconnects, playing different songs than displayed on the UI, displaying "Pause" mode when something is in fact playing are totally normal. Offline content is totally dysfunctional: There is even a "repair file" entry in the context menu because downloading files for offline use is so broken that more often than not skips and holes disturb enjoying the downloaded files. Of course, you do not notice this before you are in a train without internet connection, which is why you downloaded the album in the first place. Apparently, the Deezer dev team never heard of checksums before, as this would help to ensure that a downloaded file isn't corrupt.
• Summary repeated: Run away!!! Of all services I tested until today, this is by far the worst overall experience.

Next test in the line: Spotify just offered free 3 months premium testing since it now supports streaming to Chromecast as well.
It seems, the streaming providers really do not want any money from us these days ...

Probably our phones belong to some evil supervillain anyway

I really don't feel like writing an elaborate rant about how tech companies do not learn from others' mistakes and instead opt for doing the same mistakes on their own. Please judge for yourself whether a browser should have administrative privileges on a machine in the first place. Because if it has, this is eventually going to happen: http://www.theregister.co.uk/2015/11/12/mobile_pwn2own/
Given what our mobile devices know about us, it is seriously unsettling to know they have this level of security. Anyway, it's a weekend. I'll have a beer.

The software updating process

Hey folks! Recently updated your <putYourFavouriteDeviceHere>? One should think that this process of updating a "self-contained" device (I mean something that is usually used without being connected to some other device such as a PC) is a completely solved problem. Just take a decent up-to-date smart phone. At some point in time it will simply notify the user that a firmware update is available. One button click later, the update is being downloaded and installed upon the next device reboot. Easy as pie.
Even older, dumber devices, for instance, my 8 year old music player, have reasonable update procedures: I simply copy a firmware image into the root folder of the device (which mimics as USB mass storage device), and it will install the update upon the next device reboot / power cycle. Again, easy as pie.
And now try to update the firmware of a Sony mirrorless APS-C camera, manufactured in 2014:

• Of course it features WiFi, which would make over-the-air updates (like smartphones, see above) totally feasible; except Sony does not want to do that.
• Then it features USB (the camera can do various USB modes, including USB mass storage) which would make the "older, dumber" (see above) way of upgrading the firmware totally feasible; except Sony does not want to do that as well.
• Then it features an SD card slot, which would allow for another variant of the "older, dumber" way: The user could just copy the firmware onto the SD card, put it in, turn the camera on, and it updates the firmware; except Sony does not want to do that either.

Instead the canonical way Sony wants us to upgrade those devices is:

• Download an almost 200 MiB exe or dmg file. Of course, besides Mac OS and Windows there are no other operating systems for PCs in existence. Thanks for reminding me of that.
• Set the camera to USB mass storage mode (so, somehow it does use a standard transfer mode for files that should be available on any OS).
• Connect USB cable, run the downloaded installer. On Mac OS, this requires root because the installer wants to install a kernel driver (to transfer files??? what-the-f***... seriously Sony, were you thinking???).
• Navigate thyself through the awesome installer UI until the upgrade it complete.

Seriously Sony, this is really an antiquated and terrible way to do it. Please consider one of the three described options above which are ALL easier and more straightforward for the customer than this horrible installer thing.

The golden October in software

I have been somewhat busy over the past few weeks. It's not that nothing has happened in the world (and especially the software world), it's just that business. Anyway, without further delay some news that touched my interest in the recent past:

• The whole Volkswagen emission cheating story is certainly a globally interesting topic, not only for software people. Most of the public rage regarding the incident seems to be quite undirected noise a la "Oh, these evil manufacturers. How could they?!" but won't people stop from buying the new car anyway. So my personal prediction here is that VW won't see any dramatic drop in worldwide sales figures.
  An interesting point, however, is made by Bruce Schneier: Executives will learn their lesson from this, and the lesson is that you can get away with cheating for a long time. In a world where executives' incentives are often fixed to quarterly results, this obviously favours short-term profits by cheating over long-term strategy and brand integrity. We will certainly see more of that in the future. Also I completely support Bruce's statement on transparency and verification: Manufacturers must provide the source code (and a way to build / install it) to the public for independent review and verification.
• Chromium's tarball size has increased noticeably. Again. We are now at 327 MiB for chromium-46.0.2490.80. The earlier cry for feature freeze and cleanup remains (*surprise*) unheard.
• None of my (private) Android devices (Nexus 4, Nexus 7 2012) will receive an official update to Marshmallow from Google. This is very sad news, given that even the (original!) batteries in these devices are still doing surprisingly well.
• Flash. For the longest time, I lack the words to describe how much I loathe this prime example for how-not-to-write-software. I do use an older computer as a media center, but due to usage patterns (Chromecast, mpd, and so on), sometimes I happen not to start a browser on this thing for a week or so. The result: Every single time, and I am speaking literally here, not figuratively, I start that browser and point it to a page where flash is used, it prompts me to an upgrade. The whole flash situation has been ridiculous for years now. Please, for crying out loud: Let flash die.

Software complification [update]

You may remember my rant from a couple of weeks ago, regarding the ever-increasing complexity of modern browsers. There are two bullet points with respect to that topic:

• There are no big news, except
• Despite xz-compression, the chromium source tarball has finally broken warp three: It's now 306 MiB in size. An updated version of the illustration of the different browser size tarballs is given below.

The Windowsification of Unix

For quite a while, Microsoft Windows has exhibited a property that seems to be common in living organisms while being quite uncommon in the world of operating systems: Its ability to age and the subsequent inevitable need to die and get reborn.
To put this in more technical rather than biological terms: After a couple of months up to a low one digit number of years (in a base 3 number system) the continuous degradation of perceived performance in a Windows system, plus the ever-increasing clutter of what-the-hell-is-this-files on the C: partition becomes such a PITA that the easiest solution is to lock oneself in at home on a lonely Friday night with a sixpack of beer and reinstall Windows and all apps.
This of course was completely unknown to Unix-like folks who never really needed to reinstall their machines, mostly due to the Unix philosophy to do as much as possible with simple tools and use simple text files for configuration of the whole OS. For instance, the last time I installed my home server from scratch was more than 6 years ago and the reason was relocating overseas, not a software issue.
And then came Android. "Did you try factory reset?" is a standard question to all kinds of problems with Google's great approach at transferring the much-loathed concept of ageing software from Windows on top of a Linux-based software stack. In the past weeks, my 18-months old Android mobile phone became gradually suspiciously slow and apps started to crash more often. Yesterday it decided to give up even trying to start them: The error message "Unfortunately, the process com.google.process.gapps has stopped." appeared in very quick repetition such that it became hard to operate the device at all in order to attempt all the salvage recipes that the internet offers. Before you ask: No, I did not attempt the ones that involved sacrificing a virgin when the moon is waning gibbous; only the ones that seemed to be sensible from a technical point of view: Clearing caches, resetting apps to initial state and reapplying updates, this kind of thing, and - of course - reboot the device a bazillion times. What helps on Windows shall help on Android, right? The fuck it did. So I finally gave in, did a factory reset, and will probably spend the next weeks noticing small things that were not properly restored by Google's glorious backup-and-restore feature, starting with the local music files. Thank you Google for breaking Linux to the point that it behaves like Windows 95.

Software complification - Please make it stop

Today I have something more visual than my usual text rants: An image! Yay! Because an image is more expressive than words. Have a look a this:

This graph lists the sizes of the source tarballs of different browsers hitting the FreeBSD ports tree over the past decade. (By the way, Git, Python and Google docs were my friends in creating this.) It illustrates two major points I want to talk about:

• Starting with Firefox in 2004 (red line colour) and a tarball size of 29.7 MiB, the code base gained weight very slowly until September 2009 where it started to gain a little faster (in the graph this is roughly where the leading '0' of '01/01/2011' is placed.
  Coincidence? I don't think so. Less than a year earlier, in December 2008, Google released a public beta of Google Chrome and it became blindingly clear to most developers how powerful web apps can become if powered by the right technology. Chrome's release changed the game. It fuelled the development of new features in browsers to a velocity that we haven't seen before and sparked the development of browsers that are quite possibly the most complex piece of software you have installed on your devices.
  Then, after the open source release of Chromium (Chrome's non-commercial code parts) things really took a spin and the code base size of the leading browser implementations increased close to exponentially over the next 4 years to the point of my blog post today, at which Chromium's source can be downloaded xz-compressed into a 279.8 MiB tarball, with Firefox's tarball measuring 170 MiB in size and Seamonkey's 193.6 MiB. Without Chrome sparking this trend, for all we know we could still be at less than 40 MiB for Firefox and the slow JavaScript interpreter we all learned to hate. So far so good.
• Now comes the issue with this. Apparently for a while now we are locked into a mindset in which each new browser release has to be filled with an ever-growing list of new, cool, features. Features that, I'd argue, in part only serve edge cases and more often than not do not concern the average user. Pushing features at every cost has its price. Firefox's recent vulnerability which allowed malicious PDF documents (!) to read local hard drive files is but one example that justifies the question What the hell is missing in current browsers? Do we really need a PDF viewer built into the browser? I have no problem downloading the PDF file and open it in a PDF viewer app. The same goes for other features: Ever-increasing numbers of supported audio and video codecs, offline-applications, "Pocket" (never used it), built-in instant messaging apps, you name it. Focusing development teams (or communities) of a finite sizes strongly to features comprises the danger of omitting the necessary code refactoring, purging the code base of unused or rarely-used components, and old-fashioned maintenance releases which may improve performance, memory efficiency, security without delivering new exciting features. I am not alone here. Enjoy this well-written statement by Paul-Peter Koch who explains a lovely edge-case feature to us: Navigation transition. Quite possible that you have never heard of it before. Read the post and be stunned that this kind of stuff actually exists in your browser.
• Tiny extra point: Seamonkey, in a simplified approximation, is Firefox plus E-Mail client plus IRC client plus HTML-Editor. The graph above also shows (comparing Firefox with Seamonkey) how huge the browser code base is compared to the rest of Seamonkey's components.

So please, please, please, for heaven's sake: You Googles, Mozillas, Microsofts and players in the industry. Stop complicating things and strive for the lean, lightweight, small, fast, secure software that we really need and give the dev teams the chance to implement them.

Learning from mistakes

Learning from mistakes, I was raised in the firm belief that this is a fundamental concept of mankind to acquire knowledge. The canonical example:

• Kid is told not to touch the hot plate because it's too hot.
• Kid touches it anyways.
• Kid burns his fingers.
• Kid realises the mistake she just made.
• Kid cries in pain.
• Kid does not touch the hot plate again.

When and why did the corporate world un-learn this concept? A recent ludicrous example (links only in German, sorry). Some of you may use a Motorola Moto G phone, praised for its very good bang-for-the-buck-ratio. Unfortunately for some users, it does not come with a charger for a standard power outlet.
Last year, a few German discounters wanted to close this gap and just bundled a cheap charger with the Moto G. Too cheap, as it turned out, because the damn thing was prone to dangerous overheating and was subsequently recalled. "That happens", you might say, and you may be right.
This year, German discounters sell the successor of last year's device, the Moto G (2nd gen.), and, again, this device does not come with a charger for a standard power outlet. Again, the discounters wanted to close the gap and bundle a cheap charger with the Moto G (2nd gen.). I am sure by now you see it coming. A few days ago, these chargers have also been recalled because of - you guessed it - overheating issues. Good old friend Spock would probably right now raise his eyebrow and make his trademark statement: Fascinating.

Fasten you seat belt Dorothy, 'cause your PDF is going bye-bye

You may have heard of this already, because it is already a couple of days old, and probably you have already received a security update that patches the vulnerability that allows Firefox's built-in PDF reader to read any file that you have access to, but the fact that an exploit for this has been spotted in the wild last week chills me to the bone. We all use web browsers, on a daily basis I guess, and many developer hours go into making them such that remote sites cannot, under any circumstances, break out of that sandbox the browser provides to a web app. The fact that a malicious PDF file can be used to read any file I have permission to tells me something in that area went terribly wrong. Since it is unlikely that this is the last such vulnerability to be discovered and used against the users, you should always keep reminded that the only reason nothing evil has happened to you so far is that most of the time you are browsing sites that don't want to do anything evil to you.

It's Android's media lib again

News about one of my favourite pieces of horrible horrible terrible horrid software: Android's built-in media management has the wonderful capability of rendering your favourite phone pretty useless, just by placing a broken mkv file somewhere onto said device. The flaw has been reported to Google three months ago, but of course no fix in sight. Great. Just great.

Be sure to make everything wifi-enabled, and be sure to screw up

There seems to be a widespread obsession nowadays to make everything "smart", which, in many cases unfortunately is nothing more than a bloated management buzzword for a wifi connection. Of course, as you would naturally expect, in 99.99999975% of all cases, the vendors screw it up, allowing random third parties to do virtually anything with the device by accessing one of their countless remote exploits. This funny case, brought to my attention by my dear friend Daniel, reports a remote hack of a wifi-enabled rifle to shoot targets at the will of the remote attacker.

I read the news today, oh boy

Earlier this morning, I wanted to catch up a bit on news, and as usual, it took entire seconds to find delightful examples for horrible software defects that should never exist to begin with. The most interesting parts of this (my personal selection, YMMV) mess for today are the following.

Fiat Chrysler Automobiles (FCA)

Ever since our beloved car manufacturers decided to include software in their products, they bless us with wonderful tales of software bugs, almost on a day to day basis. Last week, FCA recalled 1.4 million cars in the US alone, because it turns out that essential functions of these cars can be remotely controlled which includes taking over steering and braking allowing for the possibility of killing the passengers. Mass recalls of so many vehicles is certainly an expensive undertaking, so for us customers, for the sake of transparency, it would be interesting to know how many percent of the price of a new car goes into software incompetence.

ERGO insurance group

One of Europe's largest insurance groups, ERGO, recently admitted that they have been using faulty  life insurance calculation software for decades and now put substantial effort into correcting the calculation for a noticeable number of customers, 350000 so far (German only article, sorry). According to this article, some calculations were off by a 5-digit figure. Peanuts, eh'?

Apple

It always saddens me to read of screw-ups committed by Apple, because I am positively biased towards their products, I have to admit. This makes it much more emotional than reading about issues at, say, ERGO, because I would blindly expect them to screw up all their software. In any case, Apple makes it twice into today's list.

1. A recently published exploit by Stefan Esser allows any user on an OS X 10.10 "Yosemite" to elevate the own permissions to root. The fun part here is that
1. The exploit is so tiny, it fits in a tweet, and is a mere shell command, and
2. There is no indication from Apple that they intend to fix it in Yosemite.
2. After the rant about music streaming I published yesterday, I was secretly hoping for Apple Music to suck less than the other services when it comes to the software. After reading how Apple Music (or rather quitting it) may destroy the rest of a user's library, I should probably bury my hopes six feet under.

Music streaming sucks - Summer 2015 edition

Music streaming sucks. I'm back to buying (CDs).

Abstract ("TL;DR" nowadays)

Music streaming services could be a wonderful thing for the customer. But they aren't. Stupidity, greed and incompetence from various stakeholders in the service delivery chain ensures that the experience for the customer becomes insufferable. After two years of paying for streaming, sending bug reports over and over, scripting workarounds for shortcomings whilst hoping for improvement, I'm done with streaming until someone gets the service right.

Introduction

It must have been about 15 years ago when my flatmate showed me this new, "awesome" software called Napster. He looked up a track via some keywords, found it in the list, started the download and only 25 or so minutes later (internet was that slow back then) he had an goddamn-horribly-terribly-awfully-sounding 128kbit mp3 file on his computer (mp3 encoders were in their infancy back then). Apparently I was ahead of my time: "Dude", I said to him, "this is shit. I can't listen to this. When it comes to music, quality matters to me. Why don't the music companies have this as a service to pay for and provide decent quality? I guess 15 DM per month would be fine with me." (We had the Deutsche Mark currency back then.). Then it took the music industry only a decade to come up with this business model that was completely obvious to every student in the world 15 years ago. Well done guys! After some early adopter's years, we are at the top of the hype cycle and have more streaming services that we need, and the whole business stinks. Here is why:

Part 1: Streaming services reviews suck

I don't know how many reviews I have read in the past years regarding streaming services. I suppose it is somewhere in the range of 40-60. Not a single one was actually helpful in determining whether I should subscribe to a particular service or not. Some reasons why they are not helpful and hints for improvement:

• Reviews typically state that "all tested services had 25-30 million tracks available, so there is no real reason to complain" and that's about the depth of the streaming catalogue analysis. That assessment could not be more wrong. In fact, all streaming catalogues suck (see below), so the reviewers should really do their homework before over-simplifying the situation to the disbenefit of their readers.
• Most (English-language) reviews seem to intend to address the whole world as potential readers. In this case, it does not make sense to consider services that are only available in the US. If the article is intended for the whole world to read, compare services that are available close to everywhere. Wikipedia keeps a list of those: https://en.wikipedia.org/wiki/Comparison_of_on-demand_streaming_music_services
• Most reviews draw conclusions after about a week of usage. This is too early to discover longer-term issues. For instance, changes in the streaming catalogue usually happen less frequent, so do updates to the client software. I'd argue that you really cannot draw useful conclusions with less than 3 months of testing. Hence, the 3 months trial that you can get from Apple Music and TIDAL are intriguing and deserve to be mentioned here.

Having said that, there are of course reviews that are extremely helpful in understanding the business or particular parts of it, especially when reviewers have received insider information about a service. From those we learn, for instance, that TIDAL has no plans of improving its sucking Mac/Windows client UI in the near future. Awesome! Let's move on.

Part 2: The streaming catalogues suck. Yes, they do.

Why on earth should a statement like "we have 30 million tracks" be relevant for any streaming service decision? How many songs do you really like? If you are like me, you are not even in the 10000s range. Or maybe you are, but you certainly don't tell your friends "Well, there are 12554199 songs I love, and I am going to enumerate them for you...", right? If a service has 30 million songs and 29.995 million of them are Mongolian medieval chanting, then the service is probably not the right one for me. So the absolute number is basically irrelevant. But, in fact, most services have a decent coverage of contemporary music. This, I suppose, is what the reviewers (see above) mean when they state "you cannot complain." But most relevant, in my opinion, are two questions:

1. How well does the service cover rare tracks, special tracks and whole albums? and
2. How frequent is content that you love removed from the catalogue?

With respect to those two questions, I am sad to say that all streaming services I have looked at suck utterly and completely. Let me elaborate. I suspect most people opting for a streaming service do so under the same premise I did: I am totally happy to pay you guys 10 bucks each month for the rest of my life if I never have to buy music again. Streaming providers support this idea of having everything you love readily available in their client software: You can add tracks, albums or artists to My Library, My Music, or whatever your streaming provider's software chooses to call it, the concept is always the same. It is the cloud-/streaming-service based representation of what used to be your stack of vinyls, cassettes or CDs in your living room. But it isn't. Because streaming services provide horrible answers to aforementioned questions 1 and 2. Specifically:

1. While fragmentation and exclusive offers over the different services is clearly stupid (why is Coldplay's Mylo Xyloto not on Google Play Music, but on Spotify? Do music industry executives seriously believe that everyone who wants to listen to Paradise once a year subscribes to a second service just for that?), things are, in fact, much worse: In 1993 I bought Paul McCartney's Off The Ground, which is available on essentially all streaming services today; with the exception, of course, of the album's most popular track, Hope Of Deliverance, which is not available on ANY streaming service. (It is also not on Spotify, my dear Spotify friends. When you play the track that is wrongfully listed as Hope Of Deliverance on Off The Ground, it will play some live version of the track, not the original studio recording. More on that later in the section on sucking software.) Same for Rise by Gabrielle, a huge hit in Europe from the year 2000, and several others.
   It does not get any better when searching for tracks that you can only remember as hits from childhood, for example The Captain Of Her Heart by dutch band Double. Apparently the music industry hates this track so much that they want to eradicate if from existence. You can't buy it or stream it anywhere today, except via 20-years-old second-hand-CDs.
2. Removal of albums from the catalogue does happen. Frequently. All the fu**ing time. In fact, although I have only some 100 albums in "My Library" on Google Play Music, I am still waiting for the first four-weeks-period since I started using the service in which my library is not affected by album removals. This is not only annoying (because tracks/albums just silently disappear from playlists as well as "My Library", without notification whatsoever, see the section on sucking software below), it is entirely unacceptable and renders "My Library" into "Someone else's library that can change any time whenever this someone else feels like it". This simply sucks, and there is no way to talk a paying subscriber into believing that this is a good thing. What would you say if you came home one day and two CDs from your stack were gone? Probably something like "burglary". And what would you say if a sampler-CD you have burned for your old car stereo spontaneously decided to contain only 18 tracks instead on 20 because it is the first of August? I mean, WTF??? I am paying the monthly subscription fee with the intention to listen to Monaco's What Do You Want From Me? whenever I want, not whenever that spooky somebody else does. Sometimes, admittedly, after weeks or months, albums return to the catalogue, at which point it is expected of us subscribers to discover that on our own and add them manually again (see the section on sucking software below). That recently happened to Platinum Collection by Genesis; do I have to listen to the remastered Paperlate now for 5000 times in the next couple of weeks before it gets removed again or can I relax about that?
3. Strategic removals also happen all the time. An album that was thought to be a flop but then gets streamed more often than expected? Happened to Mark Knopfler's Tracker only a few weeks after its release earlier this year. It's then removed from the streaming catalogue, so maybe more people buy the super-duper-deluxe-limited-digital-sleeve-bonus-tracks edition for 30 bucks (and never listen to it). You get the picture. This business model is well known from narcotics sales. The first fix is free (in this case: part of the monthly subscription fee that you are already paying), but when you become addicted, it starts to cost extra.

Fact is, the volatility of the streaming catalogues entirely void the concept of "My Library", "My Music" or whatever your streaming provider wants to call it. It's not YOUR library, it's someone else's, because it's not YOUR decision when albums get removed from it. Unless there comes along, no not Mary, but a commitment from the music industry and streaming providers that My Library will not lose tracks without my specific request, it remains unacceptable for the music lovers among us who invest time, and, obviously, money, to build up their libraries.

Part 3: The software sucks

I would talk about how the frontend software sucks, but that would be only half of the story; because the backend software sucks, too. I have not checked all the services yet, so the lamerosity-points you find below are restricted to Google Play Music, Spotify and TIDAL (including its predecessor WiMP), and to a tiny extent, Deezer, which is the service I would like to begin with:

• After selecting Deezer Music and clicking on "install" in Google Play Apps, I am presented with a request from the app that truly stuns me. Hidden in "Other [permissions]", Deezer requests the permission to delete other apps from my phone. Are you guys insane? Is this some kind of twisted reinterpretation of the second commandment? Thou shalt have no other music player before me? Sorry, Deezer, but first you need to give me an Android app that implies some mental sanity in your R&D team, then I might consider test driving your app.
• We have talked about consistency before: Google Play Music and TIDAL just don't have Paul McCartney's Hope Of Deliverance, while Spotify thinks it has it, but doesn't. This implies that there is something wrong with the identification of unique tracks at the back-end. I would love to spend some time with the dev team of one of the services (Spotify et al: You hear the cry. Give me a call!) because at this point I only have an educated guess: Not all music providers have a persistent, unique mapping for the relationship of all their songs to the list of albums. Let me explain what I mean:
  Consider an artist who has released two albums, and had 3 huge hits. Subsequently there are ten best-of-releases over the next 20 years. Also the artist will be featured on movie sound tracks, best-of-year-20xx editions, and so on. Essentially it's always the same song, and one could simply link it to all releases where it is used, e.g. via meta-data. Presumably the music industry just does not do that consistency, so the R&D teams of the streaming providers face two alternatives:
1. They can just leave it alone and don't do anything. The track will not be displayed as part of a specific album, even if it is part of another album in the catalogue. In Paul's Hope Of Deliverance case, this is what you want. For other tracks that are part of the catalogue, this may cause an issue: It might not be listed on all the albums that should contain it, but only those where the incomplete links work. This seems to be what Google and TIDAL do.
2. They can match patterns. For example speculating something like "if two tracks are named <TrackName> by <Artist> and are <N> seconds long, then it is the same track." and use this information to add the missing links. I believe this is what Spotify does. The catalogue looks more complete, because there are less missing tracks e.g. in best-of-collections. In most cases this works well, but occasionally the matching is too fuzzy and you get served a wrong track, as in Paul's Hope Of Deliverance case.
Again, this is speculation. I have not seen the code of any of the backends. I am just inferring from the symptoms. Nevertheless, for me, a streaming service user with a mild OCD, the implications are sad:
• The availability of a track on an album does not guarantee that it's really the track it is supposed to be. One has to listen to them to figure it out. Otherwise, one might get disappointed by an offline playlist on a train.
• The unavailability of a track on an album does not tell me that the track is not part of the catalogue. Again, I have to search and listen to multiple candidates and maybe find it.
• Playlist and Library management is physically painful. That applies to all services I tested. I have discussed this with several friends of mine who repeatedly told me that this is a point almost nobody cares about, except those OCD people like me. Presumably I am a dinosaur on the verge of extinction, so it might not be a problem for you. Judge for yourself:
  The clients do not deal gracefully with the removal of tracks from the catalogue. With Google Play Music, when an album gets removed from the catalogue, either one of the following can happen:
• The album disappears from "My Library", and all tracks that were part of some playlist disappear from said playlist. No notification, no way of knowing other than either
• checking your stuff manually every day, notice that a playlist that yesterday contained 87 tracks now only contains 86 tracks. With Google Music, the only "official" way to find out what happened is to use a web browser and a mobile device and exploit a bug in the mobile app: The mobile app updates its view of the world via incremental syncs. It seems Google has not implemented a command for "this track was in the catalogue yesterday, but now it's gone". Thus, the mobile apps do not notice when an album is removed, and it would still show the 87 tracks playlist. Then, you can compare the track listing on the mobile device with the web browser and identify the missing tracks. Of course, you then have to manually remove that track from the mobile device's playlist to get your data consistent over the various devices you own. If you have 10 mobile devices, have fun doing that every fu**ing week. Oh, and don't use iOS. Google Music for iOS does things to your playlists, especially the "Thumbs Up!" list that not even your worst enemy will do to you.
• writing a python script that detects these changes for you using Simon Weber's excellent google music API on github. That's what I did after the n-th time the problem occurred.
• The album disappears from "My Library" but no tracks are removed from playlists. In this case, you find out by more advanced scripting and Simon Weber's google music API. Or, of course, you play your favourite playlist and instead of playing one of the tracks in question, it just tells you "Can't reach Google Play Music at the moment." Great stuff Google, that's how library management is done!
• Nothing disappears, neither from playlists nor from My Library. Scripting does not help here, manual checking also does not help. The only thing that helps is playing an album and noticing that every track is skipped until an error message pops up.
While I have scrutinised Google Play Music the most, I observed similar annoyances in Spotify and TIDAL (and its predecessor in Europe, WiMP). And while there seems to be no problem changing the UI all the time, no action has been taking in fixing the defective playlist / library management functionality for years.
The bottom line is this: If you give a rat's ass about your playlists and don't have an eidetic memory, you better write scripts. Otherwise it is just a question of when, but eventually all your playlists will be almost empty.
• TIDAL seems to be interested in maximising the number of clicks on their client UI. Supposedly simple tasks, like rearranging the play queue, or adding multiple tracks to a playlist, require a ridiculous number of clicks. It is almost like they don't want the users to use those features. And thanks to the quite useful article by James Murphy (http://thestute.com/2014/11/07/tidal-changes-the-music-game-with-high-fidelity-streaming/) we know this UI is going to stay sucking for a while. Bad news for TIDAL subscribers here.
• Offline playback (not referring to local files here, only to files that have been 'made available for offline use' via the respective apps) on mobile devices is painful. It seems to come as a surprise to streaming software services, but mobile internet is nowhere near where it should be. We are in desperate need for a great offline mode, which is nowhere to be seen. For instance:
• Google Play Music always attempts to fetch artists's latest pictures and cover art whenever it identifies something that resembles an internet connection. In Germany, this means whenever I am in a car outside a major city, and I have only 2G connection, I have to wait for a minute network timeout each time I click on an artist or album, before I can press the Play button, even if that album is already available offline. This is an app for the patient friends among us.
• TIDAL (when put into its obscure "offline mode") is significantly reduced in functionality. "My Music" is unavailable, most interesting functions are missing. The app feels a bit too minimalist. But this has one huge advantage: Access to stored offline playlists and albums is quick and painless. The disadvantage: It won't play your local files and you have to switch to a different app for that.
• Caching of streamed files is not handled well in all services. This is something where Google's app shines. It caches, reducing network load and (maybe more importantly) is easier on your mobile quota. Also it allows to manually clear the cached content in case you need some space for something else. For the latter feature there has been a petition at Spotify's customer base for years now with no chance of ever being implemented. TIDAL's app seems to do no useful caching whatsoever, hitting your mobile plan hard.
• Local files handling is insufferable. This may not be a problem for the overwhelming majority of iOS users, as non-jailbreaked iOS tries hard to take the burden of "files" away from the user by pretending they don't exist. But on Android, files do exist and users have file managers, so there are ways to keep your device (at least partially) organised as you wish. So when I choose to put my own music files onto my mobile device, I bloody damn expect the music player apps to play them. Android comes with a bunch of codecs for that. Of course, most streaming services apps don't. Why would they? It would be great for the users, but they wouldn't see additional money for it. Why implement a feature that does not get you any additional money? The notable exception here is Google Play Music which happily plays local files. Playlists consisting of both local content and streaming content works seamlessly as well. Big plus points if you ask me.
• Mixed business models don't take place. From product management 101 some may remember that exploiting opportunities via obvious low hanging fruits is usually worthwhile. Let's get back to the 'incomplete album' example from above. Remember that Gabrielle's Rise album is available on all major streaming services except for the title track, Rise itself. Now Google has not only a streaming service but also a store where you can purchase tracks and albums. In said music store, you can actually buy Rise. The whole thing. Now when I am playing the album through my streaming subscription, the title track Rise is simply not listed. Wouldn't it make sense to have it listed (but striked through or otherwise indicated that it can't be played via subscription) alongside a one-click-buy-button to sell additional peace of mind to the customer? "Hey, dear customer. Click here, and you will have all the tracks from this album! Also, when we remove it from the streaming catalogue, you will still be able to listen to it! It's going to happen soon!" Additional sources of revenue, isn't that what the music industry wanted? Why is this not happening?

Part 4: The sound quality sucks - says who?

This is actually a part I am siding with the streaming providers. I have been a sound quality nerd all my life, but I never made the transition to the hifi-voodoo camp: I don't hear a difference between loudspeaker cables just lying on the floor compared to them being clamped into tiny iridium-platin bridge poles preventing them from touching the floor, and I find it hard to believe somebody else does.
Anyway, my point is: Despite interesting discussions on how bad digital watermarking techniques impair sound quality (e.g. http://www.mattmontag.com/music/universals-audible-watermark), the sound quality on Google Play Music and TIDAL leaves no reason to complain in my opinion. I did not test Spotify at high quality in the last couple of months, so I can't be sure about them, but especially TIDAL's lossless subscription sounds great, at least with the content I am usually listening to. (Most complaints against watermarking artefacts I found on the web are from customers who predominantly listen to classical music.)
But the fire of criticism gets additional fuel from a famous musician: Neil Young (http://www.rollingstone.com/music/news/neil-young-pulling-catalog-from-streaming-services-20150715) pulls his songs from streaming platforms because streaming is "[...] the worst quality in the history of broadcasting or any other form of distribution", Young says. I am not sure whether that's really what he means. This claim states that the standard 256kbit MP3 from Amazon's store sounds better than TIDAL's or qobuz's16bit/44.1kHz lossless FLAC-based streaming. Without suggesting that this was a marketing stunt geared towards his Pono player and the Pono Music ecosystem, that calls for a series of double-blind listening tests.
By the way, it is totally conclusive that professional musicians find it easier to distinguish lossy compressed music from the lossless source; but not because they have an inherently better hearing ability than other people. They were subjected to loud music in their lives, lots and lots of hours of it. A double blind test by the German c't magazine conducted in 2000 already demonstrated that impaired hearing sometimes helps in these tests because the impairment changes how sounds are perceived. Hence, the psychoacoustic model of lossy codecs do not match the perception of the hearing impaired candidate. (German only, sorry: http://www.heise.de/ct/artikel/Kreuzverhoertest-287592.html)

Conclusion

After almost two years of paying for monthly subscription to music streaming, it's time for me to pull the plug and go back to buying CDs. The painful truth is that none of the fundamental flaws of music streaming has changed. The client software is still horrible without any indication of somebody fixing the bugs. The volatility of the streaming catalogue and the strategic removals of content to squeeze more money out of already paying customers causes the streaming subscription model to taste bitter.
I'll probably keep on testing services every now and then, hoping for improvement. As soon as one service gets it right, I am all yours and happy to pay the monthly fee again.

New home!

As a new home for the hashtag #SoftwareÜberfordertDieMenschheit (German, lit. software is over-challenging mankind) which I have been using for quite a while on Google+ to tag stories about software-related issues that are just not supposed to happen (but obviously do), I wanted to quickly find a new suitable story. Given the terrifying state of some of today's building blocks of our information age, it took about 20 seconds to extract more than one suitable example:

• We haven't heard of a zero-day vulnerability in Adobe's flash player for *drumroll* multiple days, but our reliable friend Java came to the rescue: http://thehackernews.com/2015/07/java-zero-day-vulnerability.html
• After countless recalls in the past two to three years, one could argue that Toyota should have learned its lesson by now. But it's quite possible that this lovely bug here,
  http://it.slashdot.org/story/15/07/15/1230225/toyota-recalls-625000-hybrid-vehicles-over-software-glitch
  is way older than that. Anyway, in case you own a Prius, have fun with it. The jury is still out on the question whether it is really as green as you believe: http://science.howstuffworks.com/science-vs-myth/everyday-myths/does-hybrid-car-production-waste-offset-hybrid-benefits.htm