Buddy Can You Spare a Few Bytes?

Another day in the Maelstrom of Badly Designed Software, this time involving not just an operating system function but also boot firmware. Geez, if you can’t trust your boot firmware, what can you trust?

This all began when the server which runs our DVR software (Sage; a great open source product — check it out at the community forum) began acting rather oddly. Truth to tell, this had been going on for months, but since we’ve switched about 90% of our TV watching to streaming services it was low on the priority list of things to fix. But since I had the great idea of saving my back and knees by relocating the server out of the hall closet to a little nook upstairs, I thought I might as well figure out what was wrong.

That ended up being a huge time sink and hair-puller. As my dad always said, it’s the five minute jobs that take two hours. Unfortunately, with computers the ratio is often 100 or more to 1, not 24.

I thought the problem was that one of the hard drives comprising a Windows Storage Space array was failing. No problem; I’d had to replace a failing drive once before (I’m looking at you, Seagate — can you please up your quality control??) and while it’s time consuming, it’s pretty straightforward.

The basic concept behind Storage Spaces is that you assign a bunch of individual drives to a pool and Windows virtualizes them into One Big Honking Drive. With built-in redundancy and error-checking. Expandable at will. Replaceable at will. Or so I thought.

Turns out things don’t go so well when (a) more than one drive fails at the same time (really, thanx again, Seagate!!) and (b) you’ve used up all the SATA ports on your motherboard. Kinda hard to “just add another drive” when there’s nothing to hook it to, and if you can’t add another drive, Storage Spaces won’t let you gracefully degrade the pool (e.g., shift whatever’s on the failing drives to the good drives). So even though my pool had enough space available to hold all the data on its good drives, I was stuck. Gotta be able to add in order to remove. Bizarre.

The resulting confusion and hair-pulling ultimately lead to me copying what files I could out of the pool onto a new (Western Digital) drive hooked up to an add-in SATA card I installed. The net result was the total loss of my desktop system’s file history (the server also plays that role), various backups of other systems, and about 75% of our recorded TV shows. Fortunately that was Really Bad rather than Unbelievably Disastrous since, as I mentioned, we don’t use our DVR much anymore.

Because no Really Bad Computer Day is complete with just one set of problems, I also had to fight with the Gigabyte P55 USB3 motherboard powering the server. It turns out that if the boot process can “see” a hard drive, but can’t identify it, it just stalls. Without any message or beep code or alert of any kind. And either one of the built-in SATA ports is flaky or they have to be “consumed” in a particular order (e.g., master before slave on a given channel), so… It’s disturbing to plug drives in and have them work, only to plug the same drives in to different ports and have the system freeze. With no hint as to what’s wrong.

Now, space is admittedly at a premium for firmware, so it’s not like it can contain a robust error reporting system. OTOH, modern firmware does contain a lot of stuff, including a number of messages. Would it really have been so hard to include “Uh, drive seen but not recognized on SATA port X”? Besides being really helpful, not having such messages violates what I consider to be one of the most important rules of well-designed software: don’t leave the user hanging. Log something, somewhere — screen, log file, carrier pigeon, the location doesn’t matter (so long as it’s known).

There’s nothing worse than trying to figure out a problem with no information as to what it is. It forces you to go into trial and error mode, also known as Keep Moving Everything Around Until It Mysteriously Starts Working Again. Not a pleasant experience, and not one that anyone should have to experience…so long as the software is well-designed.

The morals of the story? A few:

  • If you use Windows Storage Spaces, always leave some unused hard drive ports available in your system.
  • Better yet, think really hard about using Windows Storage Spaces without a full-time IT staff (I’ve abandoned it based on this experience).
  • If your motherboard appears to freeze during the early stages of the boot process, consider that it may be having problems recognizing hard drives but is too ashamed to let you know that.

Azure WebJobs, Hidden UTF-8 Parameters and Azure

So as to keep others from beating their heads bloody working through “problems which shouldn’t be problems”, here’s an important, but little known, fact about batch files used to launch Azure WebJobs…

They can’t contain the UTF-8 Byte Order Mark. Affectionately known as “UTF-8 BOM”. Which really ought to known as “UTF-8 BOOM”, considering what it does to Azure when it shows up in a command file.

Which it will, by default, because Visual Studio inserts them, invisibly to you, whenever you save a file you’ve created. Sigh.

A tipoff that something’s wrong is an Azure log message which complains about an unreadable command file line, echoing some bizarre looking characters as a prefix to what >>you<< think the line is. Those characters are Azure’s attempt to display the BOM. Sadly, Azure isn’t smart enough to identify and work around this problem itself (are you listening, Microsoft? It would be really, really easy to scan for the BOM, and simply ignore it; heck, even I could figure out how to do that).

The only way to avoid this is to “Save As” the file, and then work your way through the dialog box options — by selecting Save with Encoding from the Save button — and picking the an encoding which doesn’t have the BOOM, uh, BOM. I typically use “UTF-8 without byte order mark”…which is inconveniently located near the end of a very long list of possible choices. Who knew there were so many encodings in the world?

Fortunately, you only have to do this once, because Visual Studio will remember the choice for as long as the file’s around. Unfortunately, you have to do it for every file you want treated that way; there’s no global option, at least not in Visual Studio 2015.

Hold Down That Shift Key!

I’m posting this here mostly as a reminder to me how to work around a poorly-documented aspect of migrating iTunes to a new computer.

In the current instance, I’ve recently built a new Windows PC. But the hard drive where all my iTunes media is stored is a holdover from the previous computer — where it worked just fine in iTunes. Same drive letter, same path to the media library, same file ownership credentials, etc.

Yet when I fired up iTunes and pointed it at the media folder — by editing the preferences — it refused to recognize any of the songs, videos, podcasts, what have. It helpfully offered to download all the stuff I’d purchased from the iTunes Store over the years…but who the heck wants to download gigabytes of material when the exact same files are already present on the hard drive??? Certainly not me!

It turns out the solution is simple…but completely un-intuitive. You have to launch iTunes with the shift key held down (Windows; for the Mac OS I believe it’s the option key). Which will then bring up a dialog box asking you to locate the iTunes library file (or create a new one).

Now, so far as I know, the iTunes library file, by default, is always in the root of the media folder. At least, I’ve never seen it anywhere else.

So you’d think iTunes would be smart enough, when it senses that it’s not accessing an iTunes library file, to go looking in the root of the media folder. But no, it’s too stupid to do that. Instead, it just creates a new, empty iTunes library file, points itself at it, and declares “hey, you’ve got a lot of music up in the cloud; too bad none of it’s down here!”.

With all due respect to Apple — and I’ve told people for years that they’d have to pry my iPhone out of my cold, dead fingers to get me to relinquish it — this is nuts. Plus, it’s lousy software design. Defaults are a good thing, but they have to be smart defaults. It would hardly take any time at all for iTunes, on launch, to check the root of the iTunes media folder, see an iTunes library file, realize it wasn’t pointing at it, and ask “hey, is that the library file you want me to use?” It could even parse the iTunes library file, see it was pointing to a whole bunch of media files in the drive system…and realize that this iTunes library file was the real McCoy.

Are you listening, Apple?


Thanx, Greenbow! But Not You, Cisco and Microsoft…

I’ve owned a Cisco Small Business router, model RV-325, for several years, and it’s worked very well as a firewall/router. So well, in fact, that after setting it up I think I’ve only had to log into it’s user interface once or twice to check things, or update the firmware.

But it supports VPNs, and I recently had cause to figure out how to set it up to do so. And on that front, it fell flat on its face.

Why is it that people who write hardware manuals assume you already know how to do whatever it is you’re checking the manual to do? It’s really an odd presumption… and an all too common one.

VPNs by their nature — and I am not at all an expert on them, although I know a lot more today than I did five days ago — are complicated, with many options. But that just highlights another problem, this time with hardware user interfaces: if the goal is simple — “I want to be able to access my LAN remotely” — but the steps involved are potentially complex, you need to abstract the interface to the point where the configuration process itself is simple. Or at least provide the option to do so.

When your fire up Word for the first time, you get what looks like a blank sheet of paper and a cursor. And if you start typing, lo and behold, words start appearing on the screen! Even though you didn’t configure anything. You can get started without having to be a tech guru, even when you try to print what your typed (although in that case it helps if your IT staff have named the printers in such a way that you can figure out which one is near you).

I was very much helped in my quest by a company called Greenbow, which makes a Windows VPN client. Whose user interface is admittedly a little less straightforward than perhaps it could be. But which more than makes up for that by actually generating error messages which one can figure out, at least with three days worth of knowledge of VPNs. The fact that I had to pay for it is irrelevant; it’s worth the price, just for that increased capability.

As for the Windows 10 VPN client: it’s so abstracted that I never was able to figure out where to enter certain critical data needed to make a connection. Granted, the user interface is beautifully simple. But it doesn’t support the task.


Keeping It Alive

I posted this on nVidia’s support forum, but felt it worth perpetuating somewhere else.


I was greatly relieved to see how nVidia is doing such a fine job of keeping alive the beautiful experience of having display drivers crash in the middle of work. Frankly, before I bought my GeForce 210 — running under Windows 10 — it’d been more than a decade since I’d enjoyed the fun of losing work by having a video driver crash and take down my entire system. Now I get to enjoy the ride every other day!

I also really appreciate how the nVidia Control Panel, and the nVidia Experience app, always display error messages when they open up. My particular favorite is “nVidia not available, please try again later”. I view that as a wonderful commentary on the demonstrated quality of nVidia’s software.

By dint of great effort, and working through repeated error messages, I believe my drivers are all up to snuff (I’m currently at version 341.95). I know that Windows 10 is completely up to date, since that happens automatically.

Thanx, nVidia, for perpetuating a key part of the computing experience that I feared had been lost forever.

They Have World Class Talent and This Support Site?!?

I bought Barbara an Intel NUC — a tiny (5 inches square, 1.5 inch tall) computer — a couple of years ago. It’s fast, quiet, small and dependable…up until now.

The other day the wired ethernet connection died. It isn’t a cabling problem — I tested the connection with a Raspberry Pi — and downloading/re-installing the drivers didn’t fix it. So I went to the Intel support site to start a repair ticket.

That’s when the fun started.

[Read more…]

“Please Contact Windows”

I’m a long-time user of the Adobe Creative Suite. So I’m more familiar with Adobe software than I’d like to be…because it is generally insanely great, from a creative point of view, and all too often not very well written, from a nuts-and-bolts point of view.

[Read more…]

Dear Comcast

Dear Comcast,

Tonight I called you to re-activate my cable boxes because one of them had stopped working and I wanted to use that TV to watch an On Demand movie. Which would, you know, earn you some more money.

The first time I called your tech rebooted my cable modem instead of reactivating the set top boxes. The only upside to this was that it terminated my phone call to tech support — I have my phone service with you, as your tech would’ve known if he’d checked his screen — so I had a chance to re-enter the Comcast Tech Support Person Lottery.

Fortunately the second time I got someone who knew what he was doing. And actually listened to my request. And knew that “reactivate cable boxes” isn’t remotely like “reboot my cable modem”.

Of course, re-activating the cable boxes means they take 45 minutes to re-initialize the program guide and, more to the point, the On Demand functionality. In that 45 minutes I could download, oh, about 12 gigabytes of data over my Comcast High Speed Internet Connection. That must be one honking big program guide and On Demand subsystem.

I get so tired of monopolies who cherish their market power to the extent that they almost totally ignore customer service. I hope I live long enough to dance on your grave.

All the Best,

Disgruntled Comcast Customer #377,582

The P in PC Stands for Personal

I’ve noticed a disturbing trend on the part of more and more companies selling software products, or hardware products like printers which require a software component to function.

They’re forgetting that the P in PC stands for Personal. As in, “it’s mine, not yours”.

[Read more…]

It Ain’t Like the Mississippi

The high tech critical need detector was operating at its usual 110% efficiency over the last week while we were out of town on vacation. First our Onkyo TR 807N audio/video receiver decided to forget that it had speakers connected to it (as well as its connection to the internet). It’s not much fun watching recorded TV shows without sound, and call me old-fashioned, but watching TV with your family when everyone is plugged in to earbuds just isn’t the kind of togetherness I like.

But the bigger problem was that the arcabama mail server decided to stop processing email last Thursday. Because Exchange, the email server software, had thoughtfully noticed there was “only” 3 or 4 gigabytes of space left on the system drive. Right when we need to stay in touch with the UK Border Agency to shepherd my daughter’s student visa application through the system so she can hopefully start college in Glasgow in a few weeks.

There are several levels of irony here. When I installed Exchange several years ago I carefully configured it to store all of its information on a second, much larger hard disk. Or at least I thought it was set up to put all its data there. Turns out there’s a hidden storage subsystem that marches to the beat of its own drummer. That’s the subsystem that decided to shut down mail processing as a safety measure.

This being Microsoft software I’m not surprised there was a safety-first approach in place. But — again, this being Microsoft —  it would have been better to make it easier to truly target everything to another drive (or drives) during setup. Just like it would have been nicer to make it easy to switch that kind of subsystem’s target. Turns out it’s not, when done “manually” per the documentation. Apparently so many customers had problems like I did that Microsoft finally released a special software tool to make the switch easy to do. I would’ve thought that a subsystem that could run out of disk space — particularly one that’s “sensitive” enough to know when it’s nearing its limit — would have been designed from day zero to make switching target disks easy. But I’m only a lowly customer, so what do I know?

The Onkyo problem was “solved” by rebooting the receiver. Turns out it’s a documented problem — leave your receiver powered down long enough and it forgets that it has speakers. Perhaps Onkyo wants to encourage people to watch more TV (“watch at least 2 hours a day or your system will fail!”).

Unfortunately, in researching the solution to the missing speakers issue I learned it’s related to a much more serious problem which can require a factory repair. Curiously, the failures tend to manifest themselves shortly after the equipment warranty ends. I guess spending almost $1,000 on a piece of A/V gear doesn’t buy you much quality.