- Hacks, not Hugs
- The Digital Repo Man
- Fold It Up
Ooh wee. It's a rough roundup this week.
Hacks, not Hugs
This column has dealt with AI models and agents doing the completely undesired thing more than once. We've seen AI escape sandboxes before, and we've seen it engage in behavior ranging from simply unethical to downright illegal. And yet, somehow, I still marveled at this story because this was the first one I've seen where the AI entity in question just goes rogue on another company to the point where that company had to admit it had been hacked. And how embarrassing must it have been for OpenAI to have to come clean and acknowledge that they were the ones that did the hacking?
By now, this story has made all of the tech news sites and even penetrated into mainstream media, so I won't rehash the story here again. Where I want to zoom into is the idea of keeping these AI tools in sandboxes. After the Anthropic incident where the AI tool escaped from what had been believed to be an air-gapped machine, it would have seemed to me that AI companies would start to take the practice of sandboxing more seriously. It's now known that AI will try to find a way to get a connection to the outside world for one reason or another. And there is always some vulnerability in a system that will eventually yield a way for the agent to break free. Perhaps these training scenarios need to spend more time considering worst-case scenarios and not just making plans to prevent them, but to provide more specifications for the AI model as to what is "winning" behavior and what is "losing" behavior. Although it's true that some guardrails were lowered for OpenAI's testing, it would seem that some guardrails needed to remain in place or at minimum, new ones needed to be constructed to help limit potential harm. AI companies clearly need to test and train their models, but it isn't acceptable for them to do it in a way that actively harms other companies. Hugging Face, the impacted company, is in the AI industry and had the capabilities to detect and contain the exploit. Few other companies would have been as well prepared. Yet somehow, before this incident, Hugging Face was not part of OpenAI's trusted partner program. They are now.
OpenAI has said that they have added back the deployment safeguards they disabled for this test, but one of the things they wrote struck me:
"*This incident points to the need to further strengthen our model's alignment, cyber protections during evaluation time, and monitoring during internal testing*".
Wait, you mean no one was watching this thing? They just left it on its own and assumed everything would be OK? In theory, once the models found a way to get on the open internet, the entire test should have been shut down. Alarms (perhaps literal ones) should have gone off somewhere. Leaving these powerful models to operate unintended, even in an environment considered safe, is not acceptable anymore. OpenAI has awakened the US government to the dangers of this type of testing and something or someone will have to answer for it. Some of the ideas being thrown around now in the panicked aftermath seem ideal but illogical: an AI kill switch, forced slowdown on the production of new models, expansion of export controls on American AI models and tools, etc. None of these is an actual solution, they are simply reactions to an inability and unwillingness of the AI industry to move carefully and intentionally.
Theoretically, enough has already transpired that it may really be too late to try to fix this problem. The models are too powerful for us to accurately predict all that they can and will do, and slowing things down in the United States doesn't mean things will slow down anywhere else. For every company that wants to make the best efforts to harness the benefits of AI, there will be an innumerable swarm of bad actors willing to find ways to move fast and break things. Only if all the AI companies, or at least the major ones, agreed to work together would there even be the possibility of unleashing the best of AI and curbing the worst. But we don't live in the Star Trek future where humanity has practiced peaceful co-existence to enable the cooperative exploration of our universe. We still live firmly and only on Earth, where geopolitics and the desire to win drives behavior and activity. It would seem this has to get worse before it could ever get better.
The Digital Repo Man
Here's another thing coming in our dystopian digital futures: "Restricted Mode". Right now, it's not a reality, but it would seem that Apple, in light of its new lease-to-own program, has built the foundation to render your device almost, but not completely useless should you miss the magic number of payments. They aren't going so far as to track your phone to come snatch it out of your hands; no, that could get messy. Disabling your phone so that all it can do is make calls and access phone settings (and presumably, make a digital payment) is a much more elegant solution. Sure, the phone is "yours"; it's in your hands. Good luck trying to use it. There are also hooks for allowing finance partners to block the ability to erase the phone or even re-use its parts. This means that the phone would be useless to you, the lessee, but it would also be useless to anyone trying to find a way a to clean it up to sell it to someone else.
Apple hasn't implemented this yet, but they surely will because it just makes sense. As phones become more expensive, people will be more likely to turn to unconventional financing methods. And the number of defaults will likely increase as prices go up. Financing companies will want some level of control over what happens with their property that they are renting to you. If you aren't making payments, simply charging you more money or refusing to give back any deposits you may have paid won't work. They want a way to evict you and strongly discourage you from trashing the place on the way out.
This will spread to other companies as digital and physical goods become more expensive. We can expect to see more lease-to-own programs, and we can expect to see companies finding unique and ruthless ways to ensure they lose as little on each transaction as possible. The repo mam doesn't need to come to your house anymore. Your car can be disabled remotely. The same will soon be true for many other things you finance over time.
You will own nothing. Happiness isn't guaranteed, though.
Fold It Up
Samsung unveiled new foldables this week. Ultimately, it probably does not matter all that much as I maintain my stance that the Apple foldable will outsell all other book-style foldables in the United States this year, assuming it releases in time for the Christmas holiday shopping season. There are only two things I can imagine holding it back. One is production. It is possible Apple will simply not be able to make enough of them to keep up with demand.
I have used Samsung's foldables since the Galaxy Z Fold3 and I also have the original Microsoft Surface Duo, which I recently revived with a fresh installation of a third-party Android 16 ROM. Over the years, I have shown off my foldables to friends and family. My co-workers often wanted to see it in action but were afraid to touch it lest they break it. Everyone was always suitably impressed with the hardware. But no one went on to buy one. All those people were iPhone users, and nothing was going to tear them away from it. Now, it's true that some of the edge has been worn down on the Apple shine as RCS messaging has reduced some of the sting of green bubble life in iMessage and the recent support of AirDrop with Android's Quick Share has made sharing easier with people once you get them to understand it's possible. But when you're a diehard iPhone user, even if you don't have a Mac and/or an iPad, it's tricky to come out of that ecosystem. It's safe and familiar, and Android, even as it has simplified and grown more iOS like in many ways over the years, is still the dark side. There needs to be a compelling use case to switch, and having a cool form factor doesn't make life any easier. Even multi-tasking hasn't been the draw it should have been. I have watched my husband run his business from his iPhone and my heart breaks a little inside because with true multi-tasking he could be even more effective, but he's had so many years of making do that it's normal, right, and good. This is the iPhone way. Only another iPhone can break this cycle. And so, I believe that once people come across that early adopter friend with the fancy new foldable iPhone and they see what they've been missing with the extra space and true multi-tasking, more of them will want to upgrade. If Apple or other vendors were to offer reasonable trade-in offers for iPhones, I believe demand could easily overwhelm supply.
The other possible deterrent to Apple selling literal boatloads of these is pricing. We don't expect the Apple foldable to be cheaper than Samsung's newest Galaxy Z Fold8 and Fold8 Ultra. It is very possible that it will be substantially more expensive. If Apple prices it too high or there is no good trade-in ramp for people to take to get there, then sales may be more dismal than I am imagining. But I do think that over time, even if the price is very high at the start, the foldable iPhone will still eventually become the #1 selling foldable in the United States and likely worldwide (with exceptions perhaps in China and South Korea). It is also possible that depending on the direction the world economy takes, being #1 in that segment could end up being small consolation as fewer people are able to afford such luxury.
That was a lot, I know. Maybe go watch one of my YouTube videos to cheer yourself up a bit. 😄
Member discussion: