How we got Linux on the iPhone, iPad and other iDevices

Introduction

The story starts back in 2020 with Corellium releasing Project Sandcastle. I was amazed at the sheer thought of running Linux on an iDevice, especially considering their state-of-the-art SoCs, which keep consistently outperforming the competitors’ offerings (or.. plainly trashing them, as was the case with M1!). Some time around that I got an old, used iPhone 5S for very cheap to play around with, so the only natural consequence of this would be that I try and get Linux running on it.

Stage 1: how where when what

I looked at their Linux code drop, but many things didn’t make sense at first.. Why are the addresses so high up (the MMIO peripherals start at 0x200000000, so they go above 32bit)? Why did they introduce so many changes in asm files and what is this magic sequence!?!?

I tried a couple of things, such as trying to load the kernel as-is with a quick-and-dirty device tree, but got practically nothing out of it. So I put the phone in the drawer and forgot about it for some time.

Stage 2: M1 and Asahi

Some time after @marcan announced in late 2020 that he would lead the efforts to get Linux running on the newly-announced M1 machines, I thought about the 5S again. As M1 support got closer, more and more things in the Corellium port made sense. The register space on Apple SoCs is just a design decision and it’s fully okay for it to be so high up. Actually, when you read into the ADT (Apple Device Trees) of iDevices, you can notice that it is even described there… The changes to IRQ code were a result of Apple using FIQs, or “fast interrupts”, but after some of the values were altered in a couple of commits, they behaved just like normal interrupts, as far as Linux is concerned. M1 also mandated some more changes, such as making Linux handle some not-exactly-standard EL2-related weirdness gracefully, but these were of no concern to the 5S, as the A7 SoC does not even implement it (in full compliance with the spec, as it is not mandatory).

Now with some more reference, I tried mimicking what marcan did on his streams, such as implementing a write-to-framebuffer macro in the Linux .S code, that allows for very early debugging (once we’re in pongoOS, the framebuffer is already initialized and it is not shut down when booting something with pongo). Turns out, I was actually in Linux! Success! Or so I thought, because no matter what I tried, I could not get any further than enable_mmu. I pinpointed the exact line, which made the phone die, which turned out to be the one that actually tells the MMU to enable itself. I tried a couple of monkey-brain ideas, such as dumping the SCTLR_EL1 and TTBR0_EL1/TTBR1_EL1 values from both iBoot and pongoOS, but that changed precisely nothing (as I later found out, it couldn’t have..).

So, after a couple more tries, I got tired and benched the iPhone again.

Stage 3: Cyclone is oh-so-broken

The creation of Apple Cyclone, the armv8 core in the A7 SoC, started long before the armv8-a spec was finalized (armv8-a was announced in Oct 2011, Cortex-A53 was announced in Oct 2012, iPhone 5S with A7 was announced in Sept 2013, but core design, especially with a brand new architecture takes a LOT longer than 2 years, or so I’ve heard..) (Apple supposedly co-created the armv8 spec, but I don’t have a citation for that) and it was the first SoC featuring armv8 cores on the consumer market. This of course could not have ended smoothly. I got word that I need to look into “tunables” that would make the chip behave sanely. I then connected the dots, marcan also needed them on the M1 (he refered to them as Chicken bits, but that’s the same thing). Diving into the XNU source, I quickly noticed that there was a define for each core generation supported, so a quick grep for APPLECYCLONE told me all I needed to know.

I tried replicating Corellium’s way of doing it, by interjecting the kernel boot process, setting the tunables and then continuing like nothing ever happened, but that give me.. results that were no different than before! Stuck on enable_mmu again, arghh…

I could not really think of what was wrong back then, so - it’s a common theme now.. - I put the 5S back into the drawer yet again.

NOTE: I compiled a list of A7’s (and consequently A8/A8X’s) quirks here

NOTE2: I made a tool (an arguably dirty one at that) to decode supported features from standard armv8 architectural registers, the example values tell you what features are supported on A7 (and show you how it deviates from expected values in some cases): konradybcio/disarm

Stage 4: Let’s hack on pongoOS!

One day I picked up this piece of ewas… amazing engineering again and decided that it’s time to try some more random things. First thing I did, I moved the tunable setting into pongoOS, so as to make sure they are actually set before Linux starts execution. That did not change anything, but in meantime I decided that it’d be nice to keep it in there, as adding random SoC-specific vendor-register writes to the main common init code of Linux/arm64 was not exactly the prettiest solution.

I tried doing a couple more things around enabling MMU that I don’t even quite remember now, but to no avail.

While I only have blurry recollections of what I did in this stage, I recall that I often came back to it and put the device back into the drawer again and again.

Stage 5: Let’s get quackin'

Turns out I was not the only one trying to get pretty penguins on an iDevice. A fellow from a meme-posting channel, who owned an iPad Air 2 (based on the A8X SoC - as far as the bugs and most drivers are concerned - A8/A8X and A7 are identical) wanted to join in on the ‘fun’.

I shared what I learned and to no surprise, the things I tried did not work for him either.

What he did have though, is the lack of 1-year-or-so of discouragement, so he started with a fresh mind. His device was also getting stuck at enable_mmu. After many - also not exactly interesting or successful attempts - he was back in square one. One day he even went as far as hacking on the EL3 secure monitor and getting an OS-independent write-to-framebuffer function with a simple oneliner (smc #0 - aka “hey secure monitor, knock knock”). It was certainly fun to watch the modern solutions to modern problems that were created as part of this. Quite recently, he got so tired of not having any humane way of debugging, that he made a barcode print function - he could read the value of a register and print in to the display, in a way such that any barcode-scanning application would decode it - very nice (but ultimately not very useful, but still really nice)!

Stage 6: WAIT, WHAT IF

Not so long after, I decided to have a more educated and less monkeybrain look at the pongoOS code.. it had to be ok, as the iPhone 7 booted Project Sandcastle’s Linux no problem.. I took a deep dive into the land of eye-spaghetti and dead code (pongoOS is great, checkra1n team, but you know, clang-format exists) and then noticed…

..bootr (boots a raw image) and bootl (boots a Linux image) actually DO behave differently!

Moreover, the Linux boot path sets a different entry point (aka the address where Linux is loaded, and consequently where the processor will look for instructions to execute)!

I looked at this precise line of code:

1

gEntryPoint = (void *)(0x800080000);

and was like… waaaaait, isn’t that a bit too low?

..of course it must have been.. I didn’t have the iPhone with me, so I told quack about this and he came up with 0x803000000:

1
2

-    gEntryPoint = (void *)(0x800080000);
+    gEntryPoint = (void *)(0x803000000);

and wouldn’t you know it..

WE GOT LINUX!

Now it was just a matter of what I like the most, hacking on Linux

Stage 7: Linux time!

I hacked on the AIC driver to add support for A7-A8X, as well as A9-A11 SoCs and created device trees for A7-A8X devices, including quite a sizable number of peripherals already supported (with many more to come soon, hopefully). The tree is available on konradybcio/linux-apple and is supposed to be used with my fork of pongoOS.

The patches will be sent upstream for review soon, and hopefully merged into the mainline Linux.

If you want to boot Linux on your A7-A8X device, you should follow the guide on linux-apple-resources.

Stage 8: Can I help?

Yes, you can! If you have an iDevice, it would be really appreciated if you could dump the ADT from it (there are instructions in the README file), as Apple fills in some important fields (such as reserved memory) dynamically, so we can’t get that info from the un-filled ones present in IPSWs. You can find the repo here: SoMainline/adt_collection.

If you want to buy me a cup of tea, I have a page where you can find ways to support me (mostly crypto addresses, sorry!).

If you want to support quack or other Linux projects, contact them directly for ways to support them.