Blog - David Helkowski
index

Controlling iPhones

I've spent over 5 years of my life researching and making software and systems to control iOS devices.

I sell a product that does this well: ControlFloor

Almost no one has bought a license for my product in the last 8 months. I can't survive from it despite it consuming the last 5 years of my life.

It wouldn't be much of an exaggeration to say that outside of Apple employees I am probably one of the top 10 people on planet in regard to knowing how it all works.

I say much of an exaggeration, because it's entirely possible there are a few hundred people hiding around the planet secretly know and are hoarding their knowledge. I'm mostly not. I go around blabbering about this crap all the time. I'm pretty sure people in the iOS automation scene are sick of seeing David comments appear on every single related thread about it.

I'd like to list out some of the things I have and things I have done:

Essentially, when it comes to internal Apple bullshit, I've studied the crap out of it and spent years using that research to build reliable business software that can be used by others to build iOS Device Farms that function smoothly while powerfully automating iOS devices. It was no small feat to do this and it continues to require maintenance and further research to keep it working well.

At this point you may be thinking "Yeah yeah ok whatever you think you are a genius and mr all knowing expert on Apple shit. We get it already. Could you get on with the point of your stupid post? Is there one?"

Which brings me to the point of this post. I'm fucking sick and tired of well meaning clueless dipshits asking on open source projects "Hey I have an idea! Maybe we can automate clicks and swipes on phones by blah blah bs bs bs. Isn't this exciting! Get motivated! I want to do this because it would be cool. :D"

Omg every fucking time I encounter these people I want to make them lay down on a pile of broken iPhones because it just gets on my nerves. I do actually try to educate them without chopping there heads off. I just slap them in the face a few times while educating them. Nobody really appreciates it. I can't really help it at this point though.

You see, I know all of the various ways to do what they want. I tell them how. Over, and over, and over, and fucking over again everywhere ad nauseum. The response I get back "But David I have an idea you should look into it." No shithead, your idea is crap. AND I already fucking did look into that idea, years ago. AND I fucking persistently look into it again and fucking again and again and again and a few more fucking thousand times all the damned time in case something changed. Your crap-ass "I have an idea!" comment is ONLY and completely annoying and useless trash.

But no. I can't fucking lose it like I'm a lunatic. I have to try to keep it together. Because these people aren't actually assholes or dipshits. They appear to me to be clueless morons, but they are well meaning clueless morons in some sense. Maybe even they are actually smart. I don't know. I can't tell. Their posts don't indicate they are smart. Anyway I'm getting side tracked. It's irrelevant whether they are intelligent or not.

The people asking this shit basically mean well. And then they encounter David. And David's reaction is nearly equivalent to a giant "FUCK YOU". I don't really mean to come off this way. It's bad for my reputation. It doesn't win me friends. I am actually attempting to explain what there is to know about this shit every time.

But, well, it's tiring. If I was making good money off all of my hard work in this regard, I wouldn't be so bitter. I might respond with a friendly "Sure good ideas. Here are the related things, go use them. Thanks for your question." But, well, that's not the reality in which I exist.

In the reality I exist, the field of iOS automation is fucking cutthroat. My research? My ideas? Fucking stolen from me by companies with $100m+ in capital. I have... 0 millions. I struggle to have money to pay my taxes and food.

Why should you care? Because I'm the person most likely to continue pushing forward advancement of iOS automation. If everyone involved would stop fighting me and enable me to survive instead of trying to funnel everyone to expensive solutions by greedy companies who won't share their knowledge.

You see, I may be a bitter grouch who tells you to fuck off to your face, but in the middle of telling you to fuck off I'll still point you at my research and share extensive knowledge with you. I'm just not going to sit there and pat your head and tell you that you are a good boy for reiterating clueless shitty ideas that I had 7 years ago and I know won't work out.

Let us return to the topic of the post though, which is Controlling iOS Devices, not "get indoctrinated into the sympathize with David cult by listening to his sob story."

I talked about this in a presentation I gave for the Appium Conference. In 2021. 5 fucking years ago.

The information I give there hasn't really changed much in the last 5 years. I still wear the same shirt I'm wearing in that video regularly.

I go through a lot of information rapidly in that video. I'm not going to dive into ALL the details of everything in this post and instead will focus on one aspect, which is causing taps to occur on iOS devices.

There are a number of ways to do this:

My main ire is from people saying "I had the most brilliant idea! Synthetic bluetooth HID devices! OMG OMG this will be great." STFU you fucking morons. The number of times people have tried to convince me this is the best and optimal solution is infuriating. I just can't take it. Stop stop stop.

Why?

Because there are piles of software projects on GitHub that do this. It's well understood if you'd spend any reasonable amount of time looking into it. You can literally fucking ask AI to create software to do this for you and it will within minutes.

So what's the issue? If it's so well understood and easy to do, how is it not the best solution?

Because it doesn't provide absolute coordinate tapping. It only provides relative movement of an accessibility mouse cursor. This means that if you want to do automation such as automated testing with it making synthetic taps, you basically can't.

I say basically because you -can- actually calibrate relative movement and combine it with mousekeys for precise small movements to do it, but it is very hacky. Still a valid solution. But hacky and fragile and painful.

I've not seen any open source programs that do this optimal way. There are companies that do this with private code. They haven't shared it. It's not that hard to do though. You could have AI create you the code to do it these days easily.

So what's my beef? If it can work, why do I still shit on this idea?

Because there's almost no fucking point to it. I say almost because there are situations where it is the best method. I may get into that...

The reason there is almost no point to doing it this way is because there is an official Apple supported mechanism for making taps and swipes. And it supports absolute positioning. XCTest. Yes the Apple supported way, XCTest scripts and the official calls, sucks badly. But you don't have to use that. You can use the underlying private API to quickly and efficiently make the calls. WebDriverAgent does this for example. MANY MANY open source software systems for controlling iOS devices use these private API calls. I explained how they work at a high level 5 years ago in my Appium conferencce presentation.

Just go use those private API calls of XCTest. Done.

So why are people refusing to do this? Because to use XCTest you need to have a developer account with Apple and sign the code doing it. People just don't want to have to do that.

This is where I think people are dumbasses. Yes I know it's annoying and I dislike Apple for forcing you to do all that to call the API. But, well, tough shit. That's how it is. If you don't like that, just fucking jailbreak your phone then.

But NOOOOO, fucktards persistently show up on my projects and every single damn iOS automation project and say "But I don't want to need a developer account. I think I've figured out a way." No. They haven't. STFU you fucking annoying ass shitheads. STOP.

I'm cool with people investigating Apple shit and hacking it. Awesome. Go do it. I'm all in favor. Find a way to do what you want without Apple Developer accounts. Cool beans. But for the love of anything that is holy fucking stop acting like you've come up with a good idea while having done jack shit to try it.

Go hack the damned devices. Go decompile the code. Go ask AI to figure it out. Go do literally fucking anything to move the needle forward besides fucking wasting my and other developers time with your crapass GitHub comments on "Hey what if we make virtual bluetooth devices! Best idea ever!" AAAAARRRGH.

You've made it this far. Your next objection to my ranting crazy man talk? "Uhh David relax these people are just trying to learn. They don't know any better. Why are you so angry at them?"

Well I've explained why I'm pissed off pretty extensively already. But what you are asking is not really "why are you pissed off" it is "why are you pissed off at them specifically."

A bunch of reasons:

So you see, I'm stuck inside a flaming volcano. The inside of my head has reached 4000 degree and is melting. I'd like it if someone handed me some water, or rescued me from inside the volcano, but, well, that's unlikely.

Time for you to reiterate your "David. Life is rough. Cry me a river I don't care. What exactly do you want from us?"

I want you to understand. It would be nice if you sympathized with my pain, but I don't expect that. I'd like some support for my work. Send me $5. I have a donation page on my company site. Go to it. Send me $5 so that I can survive slightly longer before I burn alive inside the volcano.

Oh. Wait. You aren't going to send me $5. You won't send me $1. You aren't here for any of that. You are here to be entertained by the comical raving ranting David character. That's okay. I'm intentionally here for you buddy. I write this shit because it -is- actually entertaining to both laugh at and with David. That's okay. I don't hate you.

I just hate those fucking dipshits on GitHub proclaiming they have an idea on how to discover valuables. If only we could invent a metal detector! Think of it! A metal detector! Brilliant!

So I've reached a relatively cool spot in the volcano. I can see the interior from here.

There is room to improve even iOS tap automation. I know how. I'd like to work on it. I need someone to support my efforts though. I can't keep giving the world valuable software while I starve. That doesn't work for me.

No matter though. I'll keep losing weight. I'm still a little pudgy. Starving isn't killing me just yet.

So what I'll request from you is this: Please just stop treating me like I'm an insane lunatic. I'm not. I'm just a guy with high technical skills who works hard to make a living from my abilities and I tire of being treated like some troll. While I may appear to exhibit troll like qualities in my comments and ranting, I have cohesive well reasoned thinking and points to share.

I would like to improve the state of software, both for iOS automation and MANY other software arenas. I simply would appreciate a bit of understanding of how ridiculous the stuff is that I've had to deal with over many years of working on this stuff.

I hope you've been entertained. Now please go find a sturdy fireproof rope and throw it down to me so I can climb out of this fucking volcano.