CDK Global, a company that provides software for thousands of auto dealers in the U.S. and Canada, was hit by back-to-back cyberattacks Wednesday. That led to an outage that has continued to impact operations.
For prospective car buyers, that’s meant delays at dealerships or vehicle orders written up by hand. There’s no immediate end in sight, but CDK says it expects the restoration process to take “several days” to complete.
This will probably be your standard someone’s exposed password compromised CDK’s network but bare in mind the staffing situation at a car dealership. For the most part you have the regular mix of personalities and capabilities that you see at any business. Except a car dealership has one dept that has a higher number than normal of clueless posers. The sales staff. Considering my experience doing IT work for a dealerships I can tell you the biggest security hole is the sales staff. What a bunch of techless fools. just take this one example.
I installed a mesh wireless system at one dealership in 2012. Did it right with a separate vlan for the guest ssid. I also at the owners request set the qos lower on the guest network to prevent a guest from hogging the bandwidth. I ended up with a loss on the job due to all the complaints from the sales weenies. Here is a sample list of their complaints.
- Its slow
- Its so slow
- Its not working
No specifics just nice generalizations and a complete lack of understanding that what they were doing at the time it was slow was important to being able to work the problem. None of them, not one would let me do some tests using their phone. They didn’t want their porn habits to be known I guess.
The situation was always just two things. Half of them were convinced that you never saved the wireless connection since one half wit there said one time the reason why your phone couldn’t connect to WiFi quickly sometimes was that you had too many saved connections. They just used the guest network which was heavily rate limited. They were told this many times but sales staff are not hired for their comprehension skills. Just the ability lie and manipulate in a pleasant way to the customers. Since I was not a customer they were as rude as any group of humans can be.
The other major thing at that time was they all were asshole buddies who had some of the stupidest status one ups. One of them was that they had to have the latest IOS update the day it came out. Of course millions of other people were grabbing the update that day. They just couldn’t comprehend that just having fast internet didn’t guarantee that apple would let them have the update at that speed. I had a iPhone at that time and they would ask how long it took me to get the update and I would just laugh and say I’ll get it in a few weeks after all of you have tested it for apple.
All of the above leads to the ol straw that broke the camels back. I had already lost money on the deal due to all the complaints I had to deal with. One IOS update broke the phones. They called angry that the WiFi broke their iphones. I told them it was the apple update that broke them not the wireless. Apple had already released a fix but these guys insisted I come out there and fix the wireless. I refused saying that it was clearly a issue with apple. The GM there called the police on me accusing me of sabotage/hacking. I get a call from the local PD who I also did work for asking me what was going on. After I got through explaining that they were idiots I called the owner and told him I would be out the next day to remove the equipment and give him a refund for it.
He asked why and I told him what the sales staff were saying and how the GM called the chief of police on me. All due to their stupidity. He told me that it works great for him every time he is there and I wouldn’t hear any more complaints. I later learned he drove from his other dealership two counties over and reamed the sales staff and GM out. Told them that if they had a problem with the WiFi to call him. I did note the next time I was there I only had to deal with the office staff who I had never had any trouble with. The sales staff and the GM made a production number our of ignoring me.
Maybe I’m being silly because I’m not in IT, but it would seem to me that one of the ways to avoid this sort of thing happening would be a diverse array of software to choose from rather than everyone using the same one. I don’t think compatibility should be an issue any more than it is for OpenOffice to be able to open Microsoft Word files. We’re not generally talking about complex interactions here, are we? It’s usually database info that can’t be access, isn’t it? But I don’t hear about diversification as a solution.
Please do explain to me what I’m missing because I feel like I’m missing something.
Ah we cross paths again…
diverse array of software
Nope the bosses want us to use one of the largest platforms because those are the best supported… usually
Also security; in many places, IT is a cost rather than being seen as an investment… car dealers want a nice building because that attracts people— fsck IT, it doesn’t attract people to buy cars…
There are a lot of industries that have niche software needs. It’s hard for a competitor to break in because the market is only so big and it’s better to have something standard and time tested.
Interoperability is often limited to a one-time database migration, and often requires a specialist to do a lot of the transfer manually.
I don’t know if that’s the case with this software because it’s not my industry, but I’ve dealt with similar issues. You’d be surprised how much of the world still runs on AS/400
All major car franchises have their own systems. I’ve been away from car dealerships for a while now but they all use similar systems and for the most part the cheapest service is always the choice. The dealerships all have differing but competent standards when dealing connections to the cooperate head quarters but everything else is a crap shoot of poorly considered decisions driven by cost and only cost. Not that the hole that the crooks used were probably through the a dealership but its possible since I know how obtuse certain groups are at dealerships.
I was about to comment similarly.
This is why I always advocate against cloud and “always connected” services for critical line-of-business software (and software for personal use, but that’s a slightly different but also similar argument).
I’m unclear if CDK is a cloud service that’s offline for customers, but it sure sounds like it. The other possibility is a supply-chain attack which affected local installs, such as what happened with SolarWinds a few years ago, but with that many dealerships being simultaneously affected by CDK shutting down their systems, it seems more like the former.
one of the ways to avoid this sort of thing happening would be a diverse array of software to choose from
In an ideal world, that would be the case. But as is often the case with niche business software, there’s usually only a few players (if that many), and any newcomers are either bought out or can’t compete.
Isn’t that monopolistic though? I realize this is a pipe dream, but wouldn’t it be theoretically possible to use the law to stop that?
I don’t know much about the market for car dealership software, but I work for a non-profit that deals with environmental remediation. Finding LOB software that meets our needs is an absolute nightmare because it’s so niche. What we can find is either crazy expensive, doesn’t do what we need it to do, is from some terrible fly-by-night vendor, or some combination of those. So when you do find something that mostly meets your needs, you pretty much have to take what you can get.
The government can incentivize or contract out companies to write software, but AFAIK, they can’t compel any company to do so. IANAL, but I would also assume they’d need to stop approving any M&As that may be contributing to market consolidation
You basically nailed it with “pipe dream”.
I guess the only other option would be for the companies to write the software themselves, which they don’t have the time or the money to hire people to do, I’m sure.
Right.
In reality, we’d end up with about a million Access “databases” (or Excel files) getting emailed around, lost, stolen, corrupted, etc (ask me how I know that lol).
It’s the same problem with every other monopoly. Everyone wants it, both shareholders and customers. It’s objectively more efficient to standardize on the same equipment or software, train workers on it. It’s better for workers too since their skills are transferrable. It’s only bad when the negatives show up, such as price gouging by the shareholders, or them cutting corners in quality or security. But my point is that not going with a single vendor isn’t free on all sides of the equation, it requires work, which is why on average we tend to prefer monopolies even as consumers.
To put it bluntly, I really don’t want to have to think about grocers profit margins and prices after having worked 9 hours. I just want to get fucking eggs and bread from the store nearby. I don’t want to drive or bus ride to another one. It won’t happen. And that’s why it doesn’t. The assumptions about the individual (constantly shopping around for the best price) in the mainstream microeconomic theory are just wrong. This translates into small businesses (not only) shopping for their dealer sales software system.
You make a lot of good points. I wasn’t really thinking about it from an economic perspective, just a security perspective.
Security doesn’t make money. They will have lost sales due to this event, but not nearly as much as they saved by skimping on security.
And they haven’t actually lost that many sales, either. If you’re going to buy a car, you’re going to buy a car. If the place is closed, you’re going to come back later. Few people are going to go to a competitor if they’ve already made their choice of brand. And even fewer are going to decide not to buy a car at all over this event.
