Exploring Your Apple M1 devices with Open Source Tools

Koan-Sin Tan,
freedom@computer.org
COSCUP, Taiwan
July 31th, 2021
Exploring Your Apple M1
Devices with Open Source Tools
1

• Learnt to use open source software before
the term “open source” was coined
• A software guy, learned to use Unix and
open source software on VAX-11/780
running 4.3BSD

• Learnt some Mach in early 1990s

• Became a mac user when Apple switched to
UNIX/Mach with Mac OS X in early 2000s
• Recently, working on NN performance on
edge devices related stu
f
• Contributed from time to time to
TensorFlow, esp. TFLite
who i am
https://gunkies.org/w/images/c/c1/DEC-VAX-11-780.jpg
2

• In case you don’t know it, Mac OS X’s
kernel is Mach-based

• its source code is open source

• usually released with BSD license
couple months after a speci
fi
c version
of macOS released

• And the main I/O Kit kernel space
framework

• macOS is UNIX

• https://www.opengroup.org/
openbrand/register/
some parts of Mac OS X are open
sourced
3
https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/Architecture/Architecture.html

Why this topic
• I am interested in knowing more about devices I am using

• I talked about iPhone thermal related software and hardware in COSCUP Taigi before, https://
www.slideshare.net/kstan2/exploring-thermal-related-stu
ff
-in-idevices-using-opensource-tool

• M1+macOS is useful

• Software for M1 is closer to iPhoneOS + A14 than Intel platform: which means understand M1+macOS helps
understand A14 devices

• macOS is fundamentally more hackable than iOS

• no need to jailbreak

• more useful tools, e.g., DTrace (http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scripts-for-mac-
os-x/)

• More symbols in kernel and kernel extensions. And there is Kernel Debug Kit (DKD), which contains more
verbose debug symbols of XNU
4

some iPhone sensor data
6
Model thermal current voltage
iPhone 6 32 21 29
iPhone 6s 48 27 23
iPhone 7 47 32 35
iPhone 8+ 68 3 7
iPhone Xs Max 67 4 8
iPhone 11 Pro 76 2 6
iPhone 12 110 2 6
M1 MBP 2020 57 36 38
With my little program: https://github.com/freedom/sensors

How to get sensor-related information
• No jailbreak required, but “undocumented” API is used. So don’t submit it to
App Store (mostly it will be rejected).

• IOKit: public and documented on macOS, but not on iOS.

• IOKit: Apple “hidclass”

• Code
• Objective-C: Get sensor information using the IOKit framework
• Swift: wrapper. because I wanna learn a bit Swift.
•
7

• IOKit: public and documented on
macOS, but not on iOS

• driver/kernel extension framework for
kernel and user space

• derived from NeXTSTEP’s DriverKit,
which uses Objective-C. As you
might know, in WWDC 2019, the
name DriverKit is back in macOS

• However, not all information we need is
documented

• “Use the Source, Luke”
IOKit
8
from “Mac OS X Internals: A Systems Approach “
http://www.nextcomputers.org/NeXT
fi
les/Docs/Developer/DriverKit/DriverKit.pdf

IOKit HIDClass
• IOKit/IOKit Family/HID class: originally it’s for USB, but it’s far beyond that now. So there is Usage Page.
• a command line tool that can be used to enumerate IOKit devices is ioreg(8)
 
• as we can see below, there are "PrimaryUsage" = 5, "PrimaryUsagePage" = 65280, and
"DeviceUsagePairs" = (“DeviceUsagePage”=65280,"DeviceUsage"=5)

•
9
https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/Families_Ref/Families_Ref.html
..
.

+-o AppleEmbeddedNVMeTemperatureSensor <class AppleEmbeddedNVMeTemperatureSensor, id 0x1000003f8, registered, matched, active, busy 0 (1 ms), retain 8
>

| | | | |
{

| | | | | "IOCFPlugInTypes" = {"7DDEECA8-A7B4-11DA-8A0E-0014519758EF"="IOHIDFamily.kext/PlugIns/IOHIDLib.plugin","FA12FA38-6F1A-11D4-BA0C-0005028F18D5"="IOHIDFamily.kext/PlugIns/
IOHIDLib.plugin"
}

| | | | | "VendorID" =
0

| | | | | "CountryCode" =
0

| | | | | "IOUserClientClass" = "IOHIDEventServiceUserClient
"

| | | | | "Product" = "NAND CH0 temp
"

| | | | | "VersionNumber" =
0

| | | | | "IOGeneralInterest" = "IOCommand is not serializable
"

| | | | | "PrimaryUsage" =
5

| | | | | "LocationID" = 141441035
0

| | | | | "HIDEventServiceProperties" = {"DeviceOpenedByEventSystem"=Yes,"PreserveTimestamp"=Yes,"BatchInterval"=1,"LogLevel"=6
}

| | | | | "ProductID" =
0

| | | | | "DeviceUsagePairs" = ({"DeviceUsagePage"=65280,"DeviceUsage"=5}
)

| | | | | "Built-In" = Ye
s

| | | | | "ReportInterval" =
0

| | | | | "HIDServiceSupport" = Ye
s

| | | | | "PrimaryUsagePage" = 6528
0

| | | | | "VendorIDSource" =
0

| | | | | "QueueSize" =
0

| | | | |
}

..
.

Build the App
• if you git clone the source code and try to build it for iOS devices, you will
get error message saying IOKit related header can’t be found (of course, you
know you have to change signing stu
ff
)

• you have to borrow them from macOS SDK,

$ pushd .

$ cd /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/
Developer/SDKs/iPhoneOS.sdk/System/Library/Frameworks/IOKit.framework/

$ sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/
MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/
IOKit.framework/Headers .

$ popd
10

The devil is in the details
• There are non-public data types, e.g.,

• AppleHIDUsageTable
s

• Some functions don’t have prototype, e.g.,

• IOHIDEventSystemClientRef
IOHIDEventSystemClientCreate(CFAllocatorRef allocator)
;

• On iOS devices, .plist
fi
le(s) in /System/Library/ThermalMonitor/ is/are
interesting
11

• Ghidra is NSA’s reverse
engineering tool, open sourced
in 2019 with Apache license

• written in Java and C++

• plugin could be in Java or
Python

• https://github.com/
NationalSecurityAgency/ghidra
ghidra
12
https://ghidra-sre.org/

IOHIDEventSystemClientCreate()
• I learned to use IOHIDEventSystemClientCreate() from the iphonedev wiki

• Currently, there is no function prototype for IOHIDEventSystemClient() in the <hidsystem/
IOHIDEventSystemClient.h> of standard IOKit headers, but there is
IOHIDEventSystemClientCreateSimpleClient(), which is not what we want
13
https://iphonedev.wiki/index.php/IOHIDFamily

https://iphonedev.wiki/index.php/AppleProxShim

• Sensors: names (in 4CC)
di
ff
erent from what we got

• Control loops

• Component Controls

• Supervised Control: in
hotspots

• Others

• Decision tree tables
iPhone 12 ThermalMonitor plist
14

More about the little program
• more details in my previous talk

• https://www.slideshare.net/kstan2/exploring-thermal-related-stu
ff
-in-
idevices-using-opensource-tool

• Bonus, there is a command line version, https://github.com/freedomtan/
sensors_cmdline

• Converted my old code to use Foundation class instead of Core
Foundation type and used command line interface, which is supposed to
be more UNIX friendly
15

Is there any other way to access thermal sensors
• The short answer is YES

• At least I know we can talk to AppleSMCSensorDispatcher to get some information

• The good thing is that we don’t need undocumented functions to talk to AppleSMCSensorDispatcher. All we need
are “standard” documented IOKit functions

• IOMasterPort(), IOServiceMatching(), IOServiceGetMatchingService(), IOServiceOpen(),
IOConnectCallScalarMethod(), and IOConnectCallStructMethod(
)

• However, we still need to
fi
gure out (magically, well, mostly be reverse-engineering) what are the parameters to use

• e.g., kern_return_t IOConnectCallStructMethod(mach_port_t connection, uint32_t
selector, const void *inputStruct, size_t inputStructCnt, void *outputStruct, size_t
*outputStructCnt);

• only the mach_port_t connection is trivial

• And special entitlement is needed to access AppleSMCSensorDispatcher
16

permission needed
• Error

% ./smc_sensors

2021-07-24 20:05:59.261 smc_sensors[25161:158890] matched
{

IOProviderClass = AppleSMCSensorDispatcher
;

}

2021-07-24 20:05:59.262 smc_sensors[25161:158890] matched 563
5

Error: IOServiceOpen() = 0xe00002c
2

• in IOKit/IOReturn.h, https://github.com/apple/darwin-xnu/blob/main/iokit/
IOKit/IOReturn.h#L146

#define kIOReturnBadArgument iokit_common_err(0x2c2) // invalid argument

17

entitlement checked
• in AppleSMCSensorDispatcherUserClient::initWithTask(), we can see that “com.apple.private.smcsensor.user-access"
needed
18

• When
IOConnectCallScalarMethod(), or
IOConnectCallStructMethod() is
called, a kernel dispatcher is called. It’s
similar to old UNIX ioctl().

• For this one, it’s AppleSMC.kext’s

• AppleSMCSensorDispatcherUserClient::externalMet
hod (unsigned int, IOExternalMethodArguments*,
IOExternalMethodDispatch*, OSObject*, void*
)

• it relies on a table at
AppleSMCSensorDispatchUserC
lient::_methods to choose the
method to invoke.
the dispatcher method
19
https://developer.apple.com/documentation/driverkit/iouserclient/3325619-externalmethod

0 sGetSMCSensorCoun
t

1 sGetSMCSensor4C
C

2 sGetSMCSensorDat
a

3 sSetSMCSensorDat
a

4 sGetMagnetKe
y

5 sGetPowerPauseKe
y

6 sGetNfcStateKey
methods
20
kern_return_t IOConnectCallStructMethod(mach_port_t connection,

uint32_t selector, const void *inputStruct, size_t inputStructCnt,

void *outputStruct, size_t *outputStructCnt);

results
% ./smc_sensors

2021-07-27 08:57:21.339 smc_sensors[3161:167859] matched {

IOProviderClass = AppleSMCSensorDispatcher;

}

2021-07-27 08:57:21.339 smc_sensors[3161:167859] matched 10243

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor = 0xc

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor_count = 1

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor_count = 60

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor 4CC[ 0] = mTPL

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor 4CC[ 1] = Te3z

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor 4CC[ 2] = Ts5z

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor 4CC[ 3] = Ta1z

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor 4CC[ 4] = Tp2z






2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor 4CC[10] = Tp9z

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor 4CC[11] = tGAM

2021-07-27 08:57:21.339 smc_sensors[3161:167859] sensor data [ 0] = 0.00










2021-07-27 08:57:21.340 smc_sensors[3161:167859] sensor data [10] = 51.00

2021-07-27 08:57:21.340 smc_sensors[3161:167859] sensor data [11] = 41.00
21
4CC longer name
LPTm ??
Te3z eACC MTR Temp Sensor 3
Ts5z ISP MTR Temp Sensor 5
Ta1z ANE MTR Temp Sensor 5
Tp2z pACC MTR Temp Sensor 2
tGAM ??

Cache Information of M1
• Apple didn’t publicly disclose cache con
fi
gurations of A14 and before

• yes, M1’s are available

• for iPhone and M1 before macOS 12.x and iOS 15.x

• information not in boot dmesg

• sysctl doesn’t work as expected:

• info of the
fi
rst booting core was used

• there are P and E clusters in modern Apple SoCs

• device tree

• https://github.com/freedomtan/iOS-device-tree-dump

• there are new sysctl entries in iOS 15.x and macOS 12.x
23

• Modern mobile SoCs consist of
cores using the same instruction
set architecture (ISA), but
di
ff
erent micro-architectures.

• Apple does’t use the term
“big.LITTLE”. On A{12,13,14}
and M1, there big and LITTLE
cores were called P
(performance) and E (e
ffi
ciency)
cores respectively
big.LITTLE
24
https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/where-does-big-little-
fi
t-in-the-world-of-dynamiq

M1 + macOS 11.x and iPhone 11 Pro iOS + 15.0 beta
hw.cacheconfig: 8 1 1 0 0 0 0 0 0
0

hw.cachelinesize: 12
8

hw.cachesize: 3616980992 65536 4194304 0 0 0 0 0 0
0

hw.cpu64bit_capable:
1

hw.cpufamily: 45878776
3

hw.cpusubfamily:
2

hw.cpusubtype:
2

hw.cputype: 1677722
8

hw.ephemeral_storage:
0

hw.l1dcachesize: 6553
6

hw.l1icachesize: 13107
2

hw.l2cachesize: 419430
4

hw.logicalcpu:
8

hw.logicalcpu_max: 8
cache info from sysctl
25

• Old ‘nm’ could
fi
nd those new
symbols

/usr/bin/nm /Library/Developer/KDKs/
KDK_12.0_21A5268h.kdk/System/Library/Kernels/
kernel.development.t810
1

• sysctl() family system calls,
which was introduced in 4.4BSD,
were used in Mac OS X when
FreeBSD/NetBSD kernel code
was used in its BSD “kernel”

perflevel symbols
26
fffffe00072663a8 s _sysctl__hw_nperflevel
s

fffffe0007266300 s _sysctl__hw_perflevel
0

fffffe00072662f8 s _sysctl__hw_perflevel0_childre
n

fffffe0007266628 s _sysctl__hw_perflevel0_cpusperl
2

fffffe00072666c8 s _sysctl__hw_perflevel0_cpusperl
3

fffffe0007266588 s _sysctl__hw_perflevel0_l1dcachesiz
e

fffffe0007266538 s _sysctl__hw_perflevel0_l1icachesiz
e

fffffe00072665d8 s _sysctl__hw_perflevel0_l2cachesiz
e

fffffe0007266678 s _sysctl__hw_perflevel0_l3cachesiz
e

fffffe0007266498 s _sysctl__hw_perflevel0_logicalcp
u

fffffe00072664e8 s _sysctl__hw_perflevel0_logicalcpu_ma
x

fffffe00072663f8 s _sysctl__hw_perflevel0_physicalcp
u

fffffe0007266448 s _sysctl__hw_perflevel0_physicalcpu_ma
x

fffffe0007266358 s _sysctl__hw_perflevel
1

fffffe0007266350 s _sysctl__hw_perflevel1_childre
n

fffffe0007266948 s _sysctl__hw_perflevel1_cpusperl
2

fffffe00072669e8 s _sysctl__hw_perflevel1_cpusperl
3

fffffe00072668a8 s _sysctl__hw_perflevel1_l1dcachesiz
e

fffffe0007266858 s _sysctl__hw_perflevel1_l1icachesiz
e

fffffe00072668f8 s _sysctl__hw_perflevel1_l2cachesiz
e

fffffe0007266998 s _sysctl__hw_perflevel1_l3cachesiz
e

fffffe00072667b8 s _sysctl__hw_perflevel1_logicalcp
u

fffffe0007266808 s _sysctl__hw_perflevel1_logicalcpu_ma
x

fffffe0007266718 s _sysctl__hw_perflevel1_physicalcp
u

fffffe0007266768 s _sysctl__hw_perflevel1_physicalcpu_max
https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/sysctl.3.html

• sysctl_mib_init() in
xnu-7194.121.3

• no “per
fl
evel” related code

• no such symbols or binaries in
iOS 14.x and macOS 11.x either
sysctl_mib_init()
27
#if defined (__i386__) || defined (__x86_64__
)

/* hw.cacheconfig */
cacheconfig[0] = ml_cpu_cache_sharing(0)
;

;

;

;

cacheconfig[4] = 0
;

/* hw.cachesize */
cachesize[0] = ml_cpu_cache_size(0)
;

;

;

;

cachesize[4] = 0
;

/* hw.packages */
packages = (int)(roundup(ml_cpu_cache_sharing(0), cpuid_info
)

/ cpuid_info()->thread_count)
;

#elif defined(__arm__) || defined(__arm64__) /* end __i386 */
watchpoint_flag = arm_debug_info()->num_watchpoint_pairs
;

breakpoint_flag = arm_debug_info()->num_breakpoint_pairs
;

arm_mvfp_info_t *mvfp_info = arm_mvfp_info()
;

gNeon = mvfp_info->neon
;

gNeonHpfp = mvfp_info->neon_hpfp
;

gNeonFp16 = mvfp_info->neon_fp16
;

cacheconfig[0] = ml_wait_max_cpus()
;

cacheconfig[1] = 1
;

cacheconfig[2] = cache_info()->c_l2size ? 1:0
;

cacheconfig[3] = 0
;

cacheconfig[4] = 0
;

cacheconfig[5] = 0
;

cacheconfig[6] = 0
;

;

https://opensource.apple.com/source/xnu/xnu-7195.121.3/bsd/kern/kern_mib.c.auto.html

sysctl_mib_init() dissembled and decompiled
28

more perflevel related code
• e.g., ml_cpu_get_info_type() and ml_get_cpu_number_type()
29

sysctl_hw_generic()
• sysctl_hw_generic() is the function you should check if you are interested in
learning more, but it’s too long to be listed and discussed here.
30

Using IOReport functions to retrieve some system information
• I need to read some of the information reported by powermetrics(1), but don't want to invoke powermetrics(1).

• It turns out I have to use IOReport functions to retrieve data from kernel side IOKit modules.

• Unfortunately, I could not
fi
nd documentation on using IOReport in user-level programs.

• Fortunately, there are kernel-side code comments, such as [1], and some user-space source code open-sourced by Apple,
e.g. [2]. With them and some previous work such as [3], I put together an example that could dump all the IOReport
information it could access.

• code: https://github.com/freedomtan/test-ioreport

[1] https://opensource.apple.com/source/xnu/xnu-4570.41.2/iokit/IOKit/IOKernelReporters.h.auto.html

[2] https://opensource.apple.com/source/PowerManagement/PowerManagement-1132.81.1/pmset/pmset.c.auto.html

[3] https://github.com/samdmarshall/OSXPrivateSDK/blob/master/PrivateSDK10.10.sparse.sdk/usr/local/include/IOReport.h
31

CPU related information
What we might need from powertrics(1)
• CPU related control on iOS devices and macOS M1 devices are quite di
ff
erent Intel ones

• There is no ACPI. Apple doesn’t use Power State Coordination Interface (PSCI) and System Control and Management
Interface (SCMI)

• If you know a bit CPU about Linux kernel

• CPU hot plug: Mach processor_exit(2) used to work, but not on M1

• http://web.mit.edu/darwin/src/modules/xnu/osfmk/man/processor_exit.html

• cpufreq: no way to retrieve current running frequency of a CPU

• no public doc on how its DVFS works?

• peak frequency is relatively easy to detect (with a loop of assembly code)

• scheduling: yes, there is some scheduling in open source XNU code, but it seems far from complete

• powermetrics(1) provides some data (you don’t know how accurate they are, though)
32

Partial results from powermetrics(1)
33

Getting IOReport info
• We can see some IOReport information from output of ioreg -
l

• “Use the Source, Luke” again
34

IOReporting
• As suggested by the name, it’s to report something

• ‘IOReporting is a mechanism for I/O Kit drivers to gather statistics (or other information) and make it available to various "observers,"
which are generally in user space.’

• more descriptions

• Available information is organized into "channels." A channel is uniquely identi
fi
ed by both driver (registry) ID and a 64-bit channel ID.

• One way drivers can advertise their channels is by publishing "legends" in the I/O Kit registry.

• In addition to collecting information and responding to queries, IOReporter objects can produce legend entries describing their
channels. The IOReportLegend class helps manage legend entries from multiple reporter objects as well as with grouping channels
logically for observers.

• With these, we know how to interpret ioreg -l outputs, see entry from spi0@35100000 on M1 MBP:

[1] https://opensource.apple.com/source/xnu/xnu-7195.81.3/iokit/IOKit/IOKernelReporters.h.auto.html
35

Sample output from my little program
36

How the little program works
• select IOReporting channels with IOReportCopyChannelsInGroup() or
IOReportCopyAllChannels(
)

• subscribe to selected channels with IOReportCreateSubscription(
)

• get samples with IOReportCreateSamples(), IOReportIterate(), and
other utility functions
37

• Apple machine learning accelerators

• CPU: AMX/AMX2

• GPU

• Apple Neural Engine (ANE)

• How can we use ANE?

• Since the public doc only says MLComputeUnitsAll

• Amazing reverse-engineering work

• AMX/AMX2: Dougall Johnson’s work, https://gist.github.com/
dougallj/7a75a3be1ec69ca550e7c36dc75e0d6f

• ANE: geohot’s work, https://github.com/geohot/tinygrad/tree/
master/accel/ane

• lldb

• ghidra reverse-engineering

• trial-and-error with system knowledge
Am I using ANE?
38
from Apple’s Nov 2020 event
https://developer.apple.com/documentation/coreml/mlcomputeunits

• The Apple Neural Engine is a fancy DMA Engine
that is based around convolutions. At its core,
it runs through 0x300 ops in an hwx
fi
le.

• It operates out of RAM or its 4MB L2 cache. The
L2 "cache" appears to be manually managed,
and only applies to the input and output, not the
weights. The weights are usually included in the
program, and it's unclear where they are copied
to.

• The 16 cores likely refer to the 16 wide Kernel
DMA engine. They claim 11 TOPS total, which
would be 687.5 GOPS/core. Perhaps it's a 32x32
MAC running at 335 MHz. That clock speed
matches the cycle count time ratio from the
debug perf stats.
ANE according to geohot
39
• It works with 5D Tensors, you specify the stride for the latter 4. All strides must be a
multiple of 0x40 bytes

• Column (width) -- aneRegs.Common.InDim.Win /
aneRegs.Common.OutDim.Wout

• Row (height) -- aneRegs.Common.InDim.Hin / aneRegs.Common.OutDim.Hout

• Plane (channels) -- aneRegs.Common.Cin.Cin / aneRegs.Common.Cout.Cout

• Depth

• Group (batch) -- aneRegs.Common.GroupConvCfg.NumGroups

• It works with 3 data types

• UInt8, Int8, Float16

• The ops have several parts

• Header -- The base addresses for the DMA engines

• KernelDMASrc -- 16x wide DMA engine for the weights/bias/nonlinearity

• Common -- Speci
fi
es the parameters for the convolution

• TileDMASrc -- Input DMA engine

• L2 -- Use the L2 cache for Source/Result instead of RAM

• NE -- Con
fi
gure Kernel/MAC/Post

• TileDMADst -- Output DMA engine

• It can work with 8 base addresses for the DMA streams per OP

• 2x Read, both used for things like sum

• 1x Write

• 1x T?

• based on geohot’s work

• drawn with latex
hwx format
40
https://github.com/freedomtan/coreml_to_ane_hwx/blob/main/ane_hwx.tex

CoreML stack
41
ANE App
aned
ANECompilerService
AppleH11ANEInterface
XPC
XPC
ANE
fi
rmware
/System/Volumes/Preboot/6137CD19-CD63-4ECE-998A-C22AA459B018/restore/Firmware/ane/h13_ane_fw_styx_j5x.im4p
I/O Kit
I/O Kit
CoreML.framework
CoreML.framework/Versions/A/CoreML
:

/System/Library/Frameworks/CoreML.framework/Versions/A/CoreML (compatibility version 1.0.0, current version 1.0.0
)

/System/Library/Frameworks/MetalPerformanceShaders.framework/Versions/A/MetalPerformanceShaders (compatibility version 1.0.0, current version 124.2.1
)

/System/Library/PrivateFrameworks/DuetActivityScheduler.framework/Versions/A/DuetActivityScheduler (compatibility version 1.0.0, current version 1.0.0, weak
)

/System/Library/PrivateFrameworks/CoreAnalytics.framework/Versions/A/CoreAnalytics (compatibility version 1.0.0, current version 1.0.0
)

/usr/lib/libMobileGestalt.dylib (compatibility version 1.0.0, current version 1.0.0
)

/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0
)

/System/Library/Frameworks/CloudKit.framework/Versions/A/CloudKit (compatibility version 1.0.0, current version 962.0.0
)

/System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo (compatibility version 1.2.0, current version 1.5.0
)

/System/Library/PrivateFrameworks/Espresso.framework/Versions/A/Espresso (compatibility version 1.0.0, current version 0.0.0
)

/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 59754.60.13
)

/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1770.255.0
)

/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0
)

/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 904.4.0
)

/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1
)

/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0
)

/System/Library/PrivateFrameworks/SoftLinking.framework/Versions/A/SoftLinking (compatibility version 1.0.0, current version 37.0.0)
Espresso
:

Contains ANECompilerEngin
e

AppleNeuralEngine: Objective-C interface called by Espress
o

ANEServices: communication with the devic
e

ANECompiler: compile plist into hwx fil
e

com.apple.ANECompilerService.allow in AppleNeuralEngine
?

Called from ANECompilerService.xpc in AppleNeuralEngine.framewor
k

Frameworks
:

/System/Library/Frameworks/CoreML.framewor
k

/System/Library/PrivateFrameworks/Espresso.framewor
k

/System/Library/PrivateFrameworks/AppleNeuralEngine.framewor
k

/System/Library/PrivateFrameworks/ANEServices.framewor
k

/System/Library/PrivateFrameworks/ANECompiler.framewor
k

Called from ANECompilerService.xpc in AppleNeuralEngine.framework
com.apple.driver.AppleH11ANEInterface

With ghidra we can find
• the dispatcher for user level clients is
H11ANEInUserClient::externalMethod(
)

• methods used by the dispatcher are in
H11ANEInUserClient::sMethod
s

• _ANE_DeviceOpen(), _ANE_DeviceClose(), _ANE_GetStatus
42

• device

• open/close

• status check

• register read/write

• power

• IsPowerOn, PowerOn, PowerO
ff
, PowerManagement

• Program

• Create, CreateInstance, Prepare, UnPrepare,
SendRequest, Destroy

• Firmware

• Load, Forget, SetFirmwareBootArg, etc.

• ….
selsector name
0 H11ANEInUserClient::_ANE_DeviceOpen
1 H11ANEInUserClient::_ANE_DeviceClose
2 H11ANEInUserClient::_ANE_GetStatus
3 H11ANEInUserClient::_ANE_ReadANERegister
4 H11ANEInUserClient::_ANE_WriteANERegister
5 H11ANEInUserClient::_ANE_IsPowered
6 H11ANEInUserClient::_ANE_LoadFirmware
7 H11ANEInUserClient::_ANE_ForgetFirmware
8 H11ANEInUserClient::_ANE_SendCommand
9 H11ANEInUserClient::_ANE_PowerOn
10 H11ANEInUserClient::_ANE_PowerOff
11 H11ANEInUserClient::_ANE_SetPowerManagement
12 H11ANEInUserClient::_ANE_GetTime
13 H11ANEInUserClient::_ANE_SetDriverLoggingFlags
14 H11ANEInUserClient::_ANE_ShowSharedMemoryAllocations
15 H11ANEInUserClient::_ANE_SetDARTCacheTTL
16 H11ANEInUserClient::_ANE_SetFirmwareBootArg
17 H11ANEInUserClient::_ANE_SetThrottlingPercentage
18 H11ANEInUserClient::_ANE_CreateClientLoggingSession
19 H11ANEInUserClient::_ANE_TerminateClientLoggingSession
20 H11ANEInUserClient::_ANE_GetDriverLoggingFlags
21 H11ANEInUserClient::_ANE_FlushInactiveDARTMapping
22 H11ANEInUserClient::_ANE_GetVersion
23 H11ANEInUserClient::_ANE_RegisterFirmwareWorkProcessor
24 H11ANEInUserClient::_ANE_UnregisterFirmwareWorkProcessor
25 H11ANEInUserClient::_ANE_GetFirmwareWorkProcessorItem
26 H11ANEInUserClient::_ANE_CompleteFirmwareWorkProcessorItem
27 H11ANEInUserClient::_ANE_ReleaseFirmwareWorkProcessorBuffers
28 H11ANEInUserClient::_ANE_ProgramCreate
29 H11ANEInUserClient::_ANE_ProgramCreateInstance
30 H11ANEInUserClient::_ANE_ProgramPrepare
31 H11ANEInUserClient::_ANE_ProgramUnprepare
32 H11ANEInUserClient::_ANE_ProgramSendRequest
33 H11ANEInUserClient::_ANE_ProgramDestroy
methods in the table
43

How to trace and check out details
• dtrace has probes for most of the methods/functions we listed

• not all of them. there is no probe for H11ANEInUserClient::_ANE_PowerOn()
44

DTrace probes in AppleH11ANEInterface kext
• no dtrace probe for H11ANEInUserClient::_ANE_PowerOn(), if we
check it, it’s clear that we can trace
H11ANEInUserClient::ANE_PowerOn(), which has a probe, instead
45

quick example of using DeviceOpen and PowerOn probes
46

• if you followed geohot’s work, it seems something like a tool to convert
coreml model to ANE’s hwx
fi
le isn’t there

• I wrote a little program that could convert a .mlmodel to a .hwx
fi
le (and
generate related
fi
les, see https://github.com/freedomtan/
coreml_to_ane_hwx/)

• .mlmodel -> .mlmodelc with some meta-information in /tmp/
espresso_ir_dump/

• espresso_dump_ir() to dump the them

• mlmodelc -> .hwx

• Using MobileNetV2.mlmodel hosted by Apple as an example

• generate
fi
les:

• model.hwx: the main mach-o binary

• model.hwx.h13_perf_all.csv: per layer pro
fi
ling (simulated)

• *.dot: graph dumped in Graphviz .dot format

• intermediate
fi
les:

• net_aux.json: list of unsupported layers/ops
coreml -> hwx
47
{

"unsupported" : [

{

"name" : "MobilenetV2/Predictions/Softmax:0",

"type" : "softmax",

"ilayer" : 100

}

]

}
net_aux.json
generated
fi
les

Recap
• Why a M1 device running macOS is a useful platform

• Review of my little sensor program

• including a “new” way

• more examples of exploring a M1 device with open source tools

• Getting cache information with sysctlbyname()

• Retrieving IOReporting information

• Getting ANE hwx related information, including examining which layers could be
run on ANE
48

Bonus exercise
Getting more CPU information from kext
• Not all of the CPU related information is available in XNU source code we mentioned

• There are some kernel extensions called “Apple*CLPCv*”

• For M1, what we want to check is com.apple.driver.AppleT8103CLPCv3

• CLPC may stands for “closed loop performance controller” (by googling it)

• There are some CPU (and GPU, and others) related functions in AppleT8103CLPCv3.kext, which is a kext used by M1 MacBook Pro

• Goal:
fi
nd the dispatcher in the .kext and write a user-level program to retrieve information
50

Exploring Your Apple M1 devices with Open Source Tools

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Exploring Your Apple M1 devices with Open Source Tools

Similar to Exploring Your Apple M1 devices with Open Source Tools (20)

More from Koan-Sin Tan

More from Koan-Sin Tan (16)

Recently uploaded

Recently uploaded (20)

Exploring Your Apple M1 devices with Open Source Tools