Owl House Build

Programming is difficult. Programming is more than just the "coding".
Establishing requirements, designing structure of code that is maintainable and easy to understand (and thus update), and testing with adequate use cases are the hard parts. These are the marks of good programming and good programmers.

If programming is difficult, so by extension is an “AI” app.
Where AI fails is at almost every point except coding.
Establishing and understanding requirements, being able to know how to structure code, and knowing how to test it are certainly not “machine friendly”.

I will share a couple of incidents that highlighted these concepts to me. After that, I will consider fallout, costs and liability.

Case #1:

My wife is a Type 1 Diabetic. Before she started using an insulin pump, she used “Kwikpens” for long acting insulin.
Suddenly, out of the clear blue sky, she started receiving texts and emails saying that her doctor’s prescription for the Kwikpens urgently needed approval.

This medication wasn't used for over 5 years.  There was absolutely no reason to think it was relevant in any way.

The messages continued after responses and the local employees did not seem to know why the messages were initiated or how to stop them. The same messages were also sent to my wife’s endocrinologist.

Obviously this was all done via an “automated” systems that the pharmacy uses.

These automated messages were totally inappropriate, totally unrequired, and totally unrequested.

What was particularly egregious was that no human being could seem to explain why they were generated or how to stop them.

This leads to a "no confidence" opinion in this pharmacy which seems to mistake paper receipts longer than any roll of toilet paper for good customer relationships.

Case #2:

Local grocery store chain has a rewards program. Something happened. Suddenly the checkouts were saying I didn’t have any rewards, but I could see on my receipt and on my account that I do.
Because I am somewhat neurodivergent, I called the store. Apparently their "computer" flags multiple purchases made with the same phone number in a short time.

However, my local Credit Union requires that I make 25 purchases in a month with my debit card. So when I tried to rack up debits for my Credit Union acct, it flagged me. Another automated system that missed the mark. I won’t say this is AI gone wild, because there's no "intelligence" to it.

What’s bad is that there was no indication at the checkout or in my “app” to inform me that my account was frozen and I needed to call in.

Costs:

In the case of the long receipt pharmacy, any goodwill or trust that the company had with me are gone. How many other customers will they lose?

I can’t believe they unleashed a system like this on people. Keep in mind that this went to the doctors as well. In retrospect I cannot find any logic in how these messages were generated.

What if these messages went to someone who was unable to spot a screw up? What if someone thought the doctor ordered the medication? This is insulin and not aspirin. They are playing with people’s lives not to mention their time and money.

I was tempted to call a “You may be entitled to significant compensation” lawyer. I shudder to think if someone got the insulin and took it. This is not as far fetched as it might sound.

The local grocery rewards program was not quite as disappointing. At least when I called them they were able to tell me why it was happening and assure me that it would be fixed on my account.

However, the loss of confidence, respect and trust is incalculable.

Don’t care about your customers? How many employees have to waste time listening to complaints mostly without the ability to help. Somebody will hopefully have to clean up the mess too.

 

Conclusion:
Programming is more than writing syntactically correct lines that produce an output.

Where is the requirements definition and validation?

Where is the testing? Where are the use cases?

Did anyone even review any of it . . . at all?

Pimpin ain’t eazy and programming is even harder.

Sterling