Since 2020, Apple computers come in two flavours: those powered by Intel CPUs, and those powered by “Apple Silicon” M1 (soon M2) CPUs. Because those CPUs use different instruction sets (x86_64 for Intel CPUs; arm64 or aarch64 for Apple CPUs), programs compiled for one kind of CPU cannot, in principle, run on the other kind of CPU.
I wrote “in principle” because in reality, Apple uses a system called Rosetta that does allow programs compiled for the x86_64 architecture to run on arm64 CPUs – it’s a neat trick they had also used several years before, during the PowerPC-to-Intel transition, when it was possible to run programs compiled for PowerPC on (then new) Intel-based Macs.
Two things however should be noted about the Rosetta emulation thing. First, it only works in one direction, to run x86_64 binaries on arm64 machines – as far as I know, it does not allow to run arm64 binaries on x86_64 machines. Second, despite Apple’s claims that the technology is performant enough that users should not notice any difference when a program is run under emulation compared to when it is run on its proper target architecture, the truth is (unsurprisingly) that emulation does incur some performance penalties. Such penalties can be very much noticeable, and developers are advised to provide arm64-targeting binaries.
It is of course possible, for each application, to provide two variants: one for Intel-based Macs and one for Apple-based Macs, and to let users pick the appropriate variant for their machines. From a user perspective however, a better solution is to use another Apple trick: universal binaries.
Universal binaries, sometimes called “fat binaries”, are standard Mach-O binaries that happen to contain executable code for different architectures, for example for both x86_64 and arm64. When the executable is run on a Intel-based machine, the x86_64 code is executed; on a Apple-based machine, the arm64 code is executed. This allows to distribute a single binary that will run natively no matter the type of machine it is run on. (This is another trick that was also used during the PowerPC-to-Intel transition.)
Applications written in Java are typically compiled into “byte code” that targets a Java Virtual Machine (JVM) rather than a real CPU. Thus, such applications are normally oblivious of the architecture of the real computer they run on, as long as a Java Virtual Machine is available for that architecture.
Things do no work so well, however, when you combine Rosetta with a Java Virtual Machine – that is, when all you have to run your Java application on a M1-powered computer is a Java virtual machine intended for the x86_64 architecture. It does work, but the performance penalty for the double emulation layer (with Rosetta translating the x86_64 code of the virtual machine into arm64 code, which then has to execute the Java byte code1) can hit really hard, which is not really surprising. So, Java applications should really be run with a Java virtual machine intended for the real architecture of the computer, and not rely at all on Rosetta.
Java virtual machines, or more formally Java Runtime Environments (JREs), are available for the arm64 architecture on macOS, so this shouldn’t be a problem. But since one cannot assume that such a JRE will be present on the user’s system, and users on macOS (and Windows for that matter) typically don’t expect to have to install a JRE themselves, it is more or less customary to distribute a Java application bundled with its own JRE so that the application can run “out of the box” once the user has downloaded it.2 This is then the subject of this note: How to package a self-sufficient Java application that could run natively on both x86_64 and arm64 Apple computers?
The first step is to obtain a Java Runtime Environment that is itself made of universal binaries, targeting both x86_64 and arm64. As far as I know, no such JREs are already available for download. All providers of JREs, such as Adoptium or Azul, only provide JREs for a single architecture (either x86_64 or arm64). We thus need to build a “universal” variant of the JRE.
One method would be to build the entire JRE from the
OpenJDK source code,
somehow tweaking the build system to ask the compiler to generate universal
binaries (-arch x86_64 -arch arm64). I have not investigated
this, though it should most likely be possible.
Instead, a much easier and faster method is to simply combine a pre-built x86_64 JRE and a pre-built arm64 JRE into a single “universal” JRE. It’s actually as simple as it sounds and only requires the lipo tool that is provided with the XCode Command Line Tools. Among other tasks, lipo can take two single-architecture binaries and produce one dual-architecture binary:
$ lipo -create -output universal_binary x86_binary arm_binary
All we need to do then is to download both the x86_64 and arm64 variants of the same JRE (same supplier, same build), extract their contents side-by-side, and iterate over the files from one of them (say, the arm64 variant):
That’s it! At the end, we get a “universal” JRE that can run Java applications natively regardless of whether we are on an Intel-powered or M1-powered computer.
The following script contains the entire procedure:
#!/bin/bash
die() {
echo "${0##*/}: $@" >&2
exit 1
}
version=$(ls OpenJDK*-jre_x64_mac_hotspot_*.tar.gz | sed -E "s,OpenJDK.+-jre_x64_mac_hotspot_(.+).tar.gz,\1,")
[ -n "$version" ] || die "Cannot identify JRE version"
[ -f OpenJDK*-jre_aarch64_mac_hotspot_$version.tar.gz ] || die "Missing corresponding arm64 JRE"
echo "Extracting native JREs..."
rm -rf x86_64 arm64
mkdir x86_64 arm64
(cd x86_64
tar xf ../OpenJDK*-jre_x64_mac_hotspot_$version.tar.gz)
(cd arm64
tar xf ../OpenJDK*-jre_aarch64_mac_hotspot_$version.tar.gz)
echo "Creating universal JRE..."
rm -rf universal
mkdir universal
find arm64 -type f | while read arm_file ; do
noarch_file=${arm_file#arm64/}
mkdir -p universal/${noarch_file%/*}
if file $arm_file | grep "Mach-O.\+arm64" ; then
# Create universal binary from both x86_64 and arm64
lipo -create -output universal/$noarch_file x86_64/$noarch_file $arm_file
if file $arm_file | grep executable ; then
chmod 755 universal/$noarch_file
fi
else
# Not a file with binary code, copy it as it is
cp $arm_file universal/$noarch_file
fi
done
echo "Packaging the JRE..."
(cd universal/jdk*/Contents
rm -rf Info.plist MacOS _CodeSignature
mv Home jre)
jar --create --file jre.os-x-$version.jar -C universal/jdk*/Contents .
It has been tested with the JRE 11 from Adoptium.
Now that we have a universal JRE that we can bundle with our Java application, what’s left to do is to setup a small “launcher” that will load the Java virtual machine and execute our Java program.
We’ll assume our application is packaged as follows:
$ find MyApplication.app MyApplication.app MyApplication.app/Contents MyApplication.app/Contents/Info.plist MyApplication.app/Contents/jre MyApplication.app/Contents/jre/bin MyApplication.app/Contents/jre/bin/java [ … Other files from the JRE omitted for brievity … ] MyApplication.app/Contents/MacOS MyApplication.app/Contents/MacOS/MyApplication MyApplication.app/Contents/jars MyApplication.app/Contents/jars/my-application.jar MyApplication.app/Contents/jars/some-dependency.jar MyApplication.app/Contents/jars/some-other-dependency.jar
That is, the bundle contains a jre/ directory that itself
contains the universal JRE we have prepared above, a MacOS
directory that by convention contains the executable file that is the “entry
point” of our application (that’s the launcher we need to setup; that file is
referenced in the Info.plist file under the key named
CFBundleExecutable), and a jars directory that
contains the archives with the Java byte code. The bundle could of course also
contain any kind of additional files that would be needed by our applications,
such as data or configuration files, icons, help files, etc.
Let’s also assume the main class of our Java program is called
org.incenp.myapplication.Launcher and is located in the
my-application.jar archive.
A very naive (too naive, as we’ll see in a minute) launcher in
MacOS/Application would be a shell script as follows:
#!/bin/zsh cd "$(dirname $0)/.." ./jre/bin/java \ -cp jars/my-application.jar:jars/some-dependency.jar:jars/some-dependency.jar \ org.incenp.myapplication.Launcher
This script is very simple as it basically does two things: 1) moving to
the Contents directory (one directory above the
MacOS directory containing the launcher script), and 2) running
the bundled java executable with the appropriate class path and
main class name.
Unfortunately, while it superficially seems to work, it actually does not ensure our Java application will run natively. For some reasons, the launcher script and all processes started from it may very well end up running under Rosetta emulation. It’s unclear to me why this can happen (and even more unclear why it doesn’t happen systematically; sometimes it works as expected, sometimes it doesn’t), but according to an Apple developer this is actually not surprising as it is apparently well known that using a shell script as the executable entry point for an application bundle is asking for unnecessary troubles. Apple developers strongly recommend that the entry point should always be a compiled binary rather than an interpreted script. So much for our simple launcher.
Let’s thus replace the above script by a very simple C program:
#include <stdlib.h>
#include <unistd.h>
#include <limits.h>
#include <string.h>
#include <err.h>
#include <mach-o/dyld.h>
/* Remove the last n components of a pathname. The pathname
* is modified in place. Returns 0 if the requested number
* of components have been removed, -1 otherwise. */
static int
remove_last_components(char buffer *, unsigned n)
{
char *last_slash = NULL;
while ( n-- > 0 ) {
if ( (last_slash = strrchr(buffer, '/')) )
*last_slash = '\0';
}
return last_slash ? 0 : -1;
}
int
main(int argc, char **argv)
{
char app_path[PATH_MAX]
uint32_t path_size = PATH_MAX;
int ret;
(void) argc;
(void) argv;
/* Get the path to the "Contents" directory. */
if ( _NSGetExecutablePath(app_path, &path_size) == -1 )
err(EXIT_FAILURE, "Cannot get application directory");
if ( remove_last_component(app_path, 2) == -1 )
errx(EXIT_FAILURE, "Cannot get application directory");
/* Move to that directory. */
if ( chdir(app_path) == -1 )
err(EXIT_FAILURE, "Cannot change current directory");
/* Run the java program. */
ret = system("./jre/bin/java "
"-cp jars/my-application.jar:"
"jars/some-dependency.jar:"
"jars/some-dependency.jar "
"org.incenp.myapplication.Launcher");
return ret;
}
A more refined version would probably use a function of the exec() family rather than system(), so as to avoid spawning a new process, but for the purpose of this note system() will do – this keeps the C code very simple as a more-or-less direct translation of the shell script above.
Let’s compile this file as a universal binary:
$ clang -o MyApplication -arch x86_64 -arch arm64 launcher.c
Put the resulting MyApplication binary under the
MacOS directory in the application bundle (replacing our too
naive launcher script).
The last thing to do is to add the following key in the
Info.plist dictionary, to be really sure that on M1, macOS will
execute the arm64 code of our launcher:
<key>LSRequiresNativeExecution<key> <true />
That’s it! Now, trying to run MyApplication.app on a M1 machine should always result on the arm64 JVM being used, removing any performance penalty due to Rosetta emulation.
The simple C launcher above works well, save for a little (or not so little, depending on what your application does) detail: the Java application started with that launcher will be unable to receive Apple events, a problem that will manifests itself by the fact that some event handlers set up with the java.awt.Desktop API will not work.
If your Java application doesn’t make use of java.awt.Desktop,
you can still stick with that launcher and simply ignore the problem. If you
do need your application to respond to Apple events however, you need a more
elaborate launcher.
The key point for java.awt.Desktop to work as expected is that
the Java virtual machine must run in a dedicated thread, while the first
thread of the launcher runs an infinite loop in which it is available to
receive the events sent by the operating system.
This raises the complexity of the launcher significantly, since we can no
longer rely on executing the bundled java program: doing that
with system() would mean that the Java virtual machine would run in a
different process than the one to which the operating system sends the events;
using exec() would keep the virtual machine in the same process but
would automatically terminate the events-receiving thread. Instead, we need
to “manually” load the Java library and start the virtual machine directly
from the C code.
Here’s a rudimentary example of such a launcher:
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <err.h>
#include <dlfcn.h>
#include <pthread.h>
#include <mach-o/dyld.h>
#include <CoreFoundation/CoreFoundation.h>
#include <jni.h>
typedef jint (JNICALL CreateJavaVM_t)(JavaVM **, void **, void *);
static char app_dir[PATH_MAX];
/* Dummy callback for the main thread loop. */
static void
dummy_callback(void *info) { }
static char *
get_application_directory(char *buffer, uint32_t len)
{
char *last_slash = NULL;
int n = 2;
if ( ! _NSGetExecutablePath(buffer, &len) ) {
while ( n-- > 2 ) {
if ( (last_slash = strrchr(buffer, '/')) )
*last_slash = '\0';
}
}
return last_slash ? buffer : NULL;
}
/* Execute the main method of our application. */
static int
start_java_main(JNIEnv *env)
{
jclass main_class;
jmethodID main_method;
jobjectArray main_args;
if ( ! (main_class = (*env)->FindClass(env, "org.incenp.myapplication.Launcher")) )
return -1;
if ( ! (main_method = (*env)->GetStaticMethodID(env, main_class, "main",
"([Ljava/lang/String;)V")) )
return -1;
main_args = (*env)->NewObjectArray(env, 0,
(*env)->FindClass(env, "java/lang/String"),
(*env)->NewStringUTF(env, ""));
(*env)->CallStaticVoidMethod(env, main_class, main_method, main_args);
return 0;
}
/* Load and start the Java virtual machine. */
static void *
start_jvm(void *arg)
{
char lib_path[PATH_MAX];
void *lib;
JavaVMInitArgs jvm_args;
JavaVMOption jvm_opts[1];
JavaVM *jvm;
JNIEnv *env;
CreateJavaVM_t *create_java_vm;
(void) arg;
/* Load the Java library in the bundled JRE. */
snprintf(lib_path, PATH_MAX, "%s/jre/lib/jli/libjli.dylib", app_dir);
if ( ! (lib = dlopen(lib_path, RTLD_LAZY)) )
errx(EXIT_FAILURE, "Cannot load Java library: %s", dlerror());
if ( ! (create_java_vm = (CreateJavaVM_t *)dlsym(lib, "JNI_CreateJavaVM")) )
errx(EXIT_FAILURE, "Cannot find JNI_CreateJavaVM: %s", dlerror());
/* Prepare options for the JVM. */
jvm_opts[0].optionString = "-Dclass.path=jars/my-application.jar"
":jars/some-dependency.jar"
":jars/some-other-dependency.jar";
jvm_args.version = JNI_VERSION_1_2;
jvm_args.ignoreUnrecognized = JNI_TRUE;
jvm_args.options = jvm_opts;
jvm_args.nOptions = 1;
if ( create_java_vm(&jvm, (void **)&env, &jvm_args) == JNI_ERR )
errx(EXIT_FAILURE, "Cannot create Java virtual machine");
if ( start_java_main(env) != 0 ) {
(*jvm)->DestroyJavaVM(jvm);
errx(EXIT_FAILURE, "Cannot start Java main method");
}
if ( (*env)->ExceptionCheck(env) ) {
(*env)->ExceptionDescribe(env);
(*env)->ExceptionClear(env);
}
(*jvm)->DetachCurrentThread(jvm);
(*jvm)->DestroyJavaVM(jvm);
/* Calling exit() here will terminate both this JVM thread and the
* infinite loop in the main thread. */
exit(EXIT_SUCCESS);
}
int
main(int argc, char **argv)
{
pthread_t jvm_thread;
pthread_attr_t jvm_thread_attr;
CFRunLoopSourceContext loop_context;
CFRunLoopSourceRef loop_ref;
(void) argc;
(void) argv;
if ( ! get_application_directory(app_dir, PATH_MAX) )
errx(EXIT_FAILURE, "Cannot get application directory");
if ( chdir(app_dir) == -1 )
err(EXIT_FAILURE, "Cannot change current directory");
/* Start the thread where the JVM will run. */
pthread_attr_init(&jvm_thread_attr);
pthread_attr_setscope(&jvm_thread_attr, PTHREAD_SCOPE_SYSTEM);
pthread_attr_setdetachstate(&jvm_thread_attr, PTHREAD_CREATE_DETACHED);
if ( pthread_create(&jvm_thread, &jvm_thread_attr, start_jvm, NULL) != 0 )
err(EXIT_FAILURE, "Cannot start JVM thread");
pthread_attr_destroy(&jvm_thread_attr);
/* Run a dummy loop in the main thread. */
memset(&loop_context, 0, sizeof(loop_context));
loop_context.perform = &dummy_callback;
loop_ref = CFRunLoopSourceCreate(NULL, 0, &loop_context);
CFRunLoopAddSource(CFRunLoopGetCurrent(), loop_ref, kCFRunLoopCommonModes);
CFRunLoopRun();
return EXIT_SUCCESS;
}
Compile the launcher for both x86_64 and arm64 architecture. You’ll need the include files from the Java Development Kit.
$ clang -arch x86_64 -arch arm64 \ -I<path_to_the_jdk>/include -I<path_to_the_jdk>/include/darwin \ -framework Cocoa -o MyApplication launcher.c
Place the resulting MyApplication binary in the
MacOS directory of the application bundle.
And this time, this is really it. We have a self-sufficient Java application that will run natively on either Intel-powered or M1-powered Apple computers and that will be able to react correctly to desktop events.
You can add a comment by replying to this message on the Fediverse.