The Unexpected Impacts Of A “Better Technology”
An example of when better isn't better.
![[picture of 3 iPhones]](https://otb.cachefly.net/wp-content/uploads/2016/02/iPhones.jpg)
As many regulars know, I work for Code for America, a non-profit that works on improving the delivery of government services to the people who need them most and strengthening our social safety net. In a team meeting last week one of our engineers shared a great example of the types of problems that can happen when a company makes a decision to promote a “better” technology.
A number of years ago, Apple made the decision to default their iOS devices to the High-Efficiency Image File Format (HEIF) in 2017. Compared to JPEG it’s a technically superior format. It’s able to compress images more effectively without compromising on image quality. So long as you stay in the iOS/Apple ecosystem .heic great!
However, if you decide to venture outside of that system, then problems start to happen. For example, if you get an email from an iPhone user and you’re not on an Apple device chances are it will be a .heic and you most likely won’t be able to open it without installing new software. For a home user, that can be frustrating. But for a government office, most of which are locked into PC platforms, that can create lots of problems. Here’s a civic tech friend who works in the criminal legal system calling out this problem on Twitter:
Mobile phone photography has revolutionized document submission, making it much easier for people to scan and submit documents–provided that we can actually easily open the images. And that adds a lot of back and forth to people’s days (slowing down the delivery of government services).
This gets us to the problem that my Code for America colleague caught. He noticed that we were experiencing a higher than usual rate of image upload failures for one of our food assistance application tools. After some digging, he realized that in a recent iOS update, Apple tweaked something about the .heic format and that “broke” our code. As a result, we stopped being able to accept documents from iPhone users.
We’re not the first organization that had this issue. For example, this issue caused problems with Advanced Placement tests in 2020. In that case, students had to pay to take the tests again. In our case, people who can’t upload documents risk not getting or losing their food assistance. And while some might say “why don’t they just learn to change their default settings,” the reality is that is creating yet another administrative burden on people who are already under a lot of stress (and may not necessarily even read English particularly well).
The good news is that our code is updated and there was an immediate drop in image upload failures. The not-so-good news is that we got a reminder that we have to proactive watch for these sorts of “little tweaks.” And, unless Apple publicly announces the changes, that really means watching for the traces of a change–the impacts they have in order to identify when something has changed. And, unfortunately, in our field, that means someone not being able to upload a document to help get benefits (and the frustration and insecurity that causes for individuals and their families). More often than not, when things break in government service delivery, people also get broken too.
Social scientists have argued for years that devices have politics (intentionally and unintentionally) embedded in them by their creators. This came up about a year ago in a discussion about the degree to which infrastructure can be racist. This is another example of this concept in action. Apple has prided itself on doing things differently for years. However, as this example shows, doing things differently often has unexpected impacts and sometimes harms the most vulnerable of their customers.
Kudos to your colleague for noticing the error rate. I expect it’s really kudos to another colleague for setting up an alarm that poked someone.
I’m surprised that I never knew about this file format before now. Do I know so few Android and Windows folks? Has my phone been carrying around an old setting? No clue.
@Gustopher:
Part of the reason you might know is that in certain interactions–say sending a photo via SMS–there’s an automatic conversation to JPEG.
Folks run into problems when it comes time to “attach” images to things. Sometimes Apple will treat a .heic file as an image and automatically convert it. And in other cases, it treats the file and a file and just uploads the “raw” file as-is. It’s the latter where folks run into problems.
We’re a small and mighty team. So I’m pretty sure this was all the result of him doing due dilligence and checking out the error logs.
As a younger engineer, standards would often chafe me. Why can’t we do this better thing?! I would think, and moan.
But standards carry a lot of value, and now I’m really quite impatient with corporations that avoid using them and instead push their own “better” stuff. Apple is one of the worst, but Microsoft isn’t that good, either. (C# I’m looking at you)
@Jay L Gischer:
We had some minor troubles when Microsoft switched Word and Excel files to DOCX and XLSX on a new version of Office. These were resolved with a conversion utility that took all of three minutes to find online.
I’m more peeved when people send photographs I need to print or copy at work over WhatsApp. It seems a lot of people don’t know they can take photos in their phones with the camera app, and then email them.
@Jay L Gischer:
I’m not sure that is fair. A lot of Apple “breakages” are due them being the first to move to a new technology, and they actively drive the standards bodies to include whatever is necessary. A good example of this is the way they connect to external hard drives. A significant portion of their user base (video editors) values the fastest possible transfer speeds. Apple has consistently pushed the envelope on this, but has always tried to make their technology as part of a standard. Confusingly, they usually have their own name for the technology so you might not know it is a standard. An example of this is Firewire (IEEE 1394 standard). It was never successful as a general standard but used by companies concerned with video. For example, Sony implemented it as i.Link. Firewire is now way too slow for modern devices, and high resolution, high refresh rate monitors have created a demand for superfast transfer speeds even outside of the videoworld. Apple worked with the USB-C standards committee to include their Thunderbolt technology.
Other examples of Apple getting slammed for introducing something new just so they could “break” things, but that actually stayed around for a long time: PDFs, 3.5″ floppies, Lightning connector (despite “common knowledge”, Apple has only replaced their iPhone charging connector once, with the lightning connector replacing the 30 pin plug 10 years ago).
I’m no software engineer, but there are many examples of when better isn’t better.
Just for one example; Ford could have stopped re-designing trucks after the 5th Generation (1967-72) of the F-series as far as I’m concerned.
But maybe that’s just me…
I run into this problem often with my iPhone / Mac students sending me screenshots / pictures / movies asking me for help that I have no way of opening without spending my own money to download a converter. They are engaging in behavior I actively try to encourage (asking for help after getting stuck for a short while, rather than spinning wheels indefinitely), but I’m unable to provide them with the positive reinforcement for that behavior. It is incredibly frustrating.
A couple of weeks ago, I spent an entire evening sitting in Zoom waiting for students to pop in once they realized they couldn’t get their online proctoring program up and running. The issue only affected students using Macs. Once I figured out how to resolve the issue, it took me two minutes to help each one after they shared their screen with me (due to FERPA, I had to help each once individually since confidential information was on their screen). It was an exhausting way to spend a Thursday night after a full week of teaching.
(to whomever it applies):
Are you bemoaning that the world doesn’t have a Technology Dictator?
Preach it! This file issue is a pain in my (thing that is not being tanned). I’ve created work-arounds, but it’s still a hassle. To me.
It hasn’t occurred to me how this is more than a hassle to others. Why hasn’t it occurred to me? Because I’m self-centered of course.
My inconvenient hassles are substantive frictions for others. And those frictions create and perpetuate disadvantages to folks who have fewer social degrees of freedom than I do.
This is a great reminder of that ugly reality. Thanks Matt.
@MarkedMan: There’s probably at least something in what you say. I don’t know much about video production or the technology support for it.
And, I’ve owned a Mac laptop of one kind or another for maybe 20 years now, because Unix. The same story keeps playing out: I need some sort of way to hook something up, and this forces me to buy Apple-specific hardware: For instance, external hard drives. Or adapters.
Ye gods, they even got rid of the magnetic power cable connector, which I thought was a magnificent idea. I guess its patent was about to expire, and they couldn’t have third-parties start selling cheaper power cables, could they now?
@Jay L Gischer:
The magnetic was great. That said, I’m really excited that they moved to the USB-C standard as that definitely opened up the options for third party power (and other accessories). That said, I wish they still had a couple additional ports on the Powerbooks.
@Jay L Gischer: As I understand it, the magnetic connector was a reliability problem. I also suspect that it didn’t work as well as the laptops got lighter. And, as Matt said above, they replaced it with an open standards connector, not a proprietary one.
Matt, you’re pushing a hot button with me. I agree, asking people to fiddle around with settings and logging out and logging in and refreshing your browser and updating the OS and refreshing your cache and deleting unused files and checking your iCloud configuration and and and…Is all predicated on a technology adoption model centered on a group of people who are both (1) technically savvy enough to guess what sorts of changes might fix a problem, and (2) don’t might spending a frustrating 10, 15, 30 minutes until they discover the fix. I’ve been in the technology business for decades, and I’ve always thought it was unfair to expect the average consumer of the technology to be like the average producer of the technology.
Lets face it, most software is written for the benefit of other IT geeks, not the rest of us. I hope there is a special place in hell where the software engineers go and they are forced to use software designed by people who are not good at IT.
Steve
My last real job was managing software updates for a user base of ~ 105,000 people in HQ and several thousand remote offices.
Push out software updates. Sounds easy. Was fiendishly difficult. The possible interactions explode exponentially really fast.
Everybody loves an update until it breaks an existing process. It is a very fraught thing to update software.
We had a system to break down application interactions and test in a lab. We had extremely smart folks chasing down processes and noting inputs and outputs. The how, where and the what of it. Note touch points.
Until I had the job I never realized how much a huge-ass company runs on jerry-rigged, unsupported data driven processes. All inter-related.
People do amazingly creative things in Excel or Access that is, or feeds into, a core process. Some of it was amazing. It usually wasn’t elegant, but it did the job. People are really good problem solvers.
I reported to a person who reported to the CFO. I laid it out to my boss that one day we are inadvertently going to break a core process. It is inevitable. It will happen. We can revert to the previous version overnight either enterprise wide or just to specific machines, but a nasty surprise will happen eventually guaranteed. We had wicked smart business and data analysts, but if you don’t talk to Jo in Accounts you are blind. We relied upon department contacts to steer us towards the right folks. If no one steered us at Jo we basically had no idea.
We identified so many processes that needed full IT support, development and backup it totally over-whelmed the application development staff and DBAs.
It is crazy what smart, clever people can do with Access. I hired some of them.
And all this happened before mobile phone apps.
I met a woman who figured out how to a waterfall with Access.
A Waterfall is series of datasets each reliant on the forbear.
1. I want people whose FICO is 710 or plus.
2. 710 plus and green eyes.
3 710 plus, green eyes, and from Missouri, Minnesota, Michigan or Massachusetts.
4. All above, and haven’t paid late in the last 18 months.
Each resulting set is a subset of the previous set. If the criteria are static and the order of the winnowing is static you can do it easily: do a query that writes the results into a temp db, rinse and repeat, but if the order of the winnowing questions changes it completely invalidates and you have start over from scratch.
They were looking to send out n refinance letters a week. They had the capacity to process n refinances a day. It was a complicated calculus. Bigger brains than mine calculated probable response rates. PHD statisticians, (Dr. Jeff was a pretty cool dude.)
Once they hit the number they could handle. It was stop the waterfall. Send refinance packages to those people. The winnowing factors and how they were ordered changed daily.
So H. did it all in Access until they hired me on. H. is a goddamn genius. Her structure was brilliant. She sucked down the likely set and stored it. Used fucking Access of all things. Did a query for every step. Made a macro for the waterfall. Brute forced it. A series of nested queries. Fucking brilliant. First day, it took me half a day to figure it out.
If the winnowing questions changed order she opened up her macro and clicked, dragged and dropped. With the toolset that I used then I could do the same in maybe a half day, maybe two full days to cross the i’s and dot the t’s properly. Documention. That’s with unfettered access to a dedicated DBA.
She did it in ten seconds.
Of course it was brutally inefficient, but it was pure genius.
She disliked me from word one, before word one, actually. Why did she need me?
Well a full run took 19 hours. It was brilliant, but brutally inefficient.
You can do crazy, crazy shit with Access and a Big Brain.
She wasn’t even trained. She went to school to be a CPA.