Micah Lee

What goes into making an OnionShare release: Part 3

Posted October 20, 2023 in code onionshare

About a month ago I started working on an OnionShare release, documenting the entire arduous process. It's always a painful process, but it's absolutely bonkers how much work has gone into this release. This is the third, and not quite final, part of this epic release engineering saga.

In part 1 of this series, I describe how I started making the release. I merged in translations from Weblate and made sure the correct translations were enabled for the desktop app and the documentation. After some struggling I got the Snapcraft release working--this involved upgrading the snap base from core20 to core22 so that I could upgrade the from PySide2 to PySide6. I then ran into a wall trying to get Flatpak working.

In part 2, I finally finished Flatpak packaging, albeit with a bit of work still left for future Micah to solve. I successfully finished the Windows build, and I started making the macOS build. Where we last left off, OnionShare was running fine from the source tree on my Mac, but but after I used cx_Freeze to convert the Python source code into a runnable Mac app bundle, it crashed with a segmentation fault when I ran it.

⚠️ WARNING: This blog post, and this whole cursed series of posts, may make you want to smash your face against your keyboard due to the sheer volume of technical issues, error messages, and general injury to morale. Because of all of the problems I consistently have been running into, and also how complex making OnionShare releases has become, this blog post is stupidly long.

In this post I describe how I make a universal2 macOS app bundle that runs fine on both older Intel Macs and newer Apple Silicon Macs. This journey includes fixing a bug, and submitting an upstream PR, in the cx_Freeze project. I also discovered that if only OnionShare's GitHub organization were using a paid plan, I could automate a lot more of making the universal2 app bundle with GitHub Actions--but alas, we're not paying for it yet. (Hey GitHub, can we have a free Team plan?)

I decided that since so much work has gone into this release so far, and so much new code has been added, that instead of making a full release I will just make a development release, OnionShare 2.6.1.dev1. This way it can be more thoroughly tested before making a stable release. So, I made the dev release, complete with code signing binaries in Windows using a HARICA-trusted secret key on a USB smart card that I passed through to a Windows VM, and also code signing and notarizing the macOS app bundle.

Here's a table of contents:

I'll start where I left off, with the macOS build.

Fixing the OnionShare Mac app bundle

As I described in part 2, I have an old early 2015 MacBook Pro that I use as my dedicated macOS software release laptop. I made sure I had an up-to-date OnionShare development environment, and confirmed that I could run OnionShare locally from the source tree by changing to my onionshare folder and running:

cd desktop
poetry run onionshare -v

This runs the OnionShare desktop Python script, and everything works great. To actually build the Mac app bundle, I run these commands:

poetry run python ./setup-freeze.py build
poetry run python ./setup-freeze.py bdist_mac
poetry run python ./scripts/build-macos.py cleanup-build

The first one, ./setup-freeze.py build, freezes the Python code and creates a folder that has macOS Mach-O binary files in it (onionshare and onionshare-cli) that you can natively run on a Mac. This folder contains a copy of Python and all of the libraries and extra resources that OnionShare needs. The second command, ./setup-freeze.py bdist_mac turns these binaries into an app bundle, and the third command, ./scripts/build-macos.py cleanup-build, deletes unused files in the app bundle to save disk space.

When I left off, I got a segfault when trying to run the onionshare binary in the final app bundle. This time, I'm going to back up and see if I can get the binaries from ./setup-freeze.py build to work, before I even make an app bundle.

Freezing the Python code

Since I'm using Python 3.11, that command puts the binaries in build/exe.macosx-10.9-universal2-3.11, so I'm starting by deleting that folder, just to clean up from last time:

rm -rf build/exe.macosx-10.9-universal2-3.11

Then I rebuilt the binaries:

% poetry run python ./setup-freeze.py build
running build
running build_py
running build_exe
--snip--
WARNING: In file [/Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/PySide6/QtXml.abi3.so] guessing that @rpath/libshiboken6.abi3.6.5.dylib resolved to /Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/shiboken6/libshiboken6.abi3.6.5.dylib.
WARNING: In file [/Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/PySide6/QtWebEngineCore.abi3.so] guessing that @rpath/libshiboken6.abi3.6.5.dylib resolved to /Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/shiboken6/libshiboken6.abi3.6.5.dylib.

There were a bunch of other similar warnings at the end of the command, but for now I'm going to ignore them and see if the binaries run. I'll start with onionshare-cli, since that takes far fewer dependencies than the desktop version and is more likely to work:

cd build/exe.macosx-10.9-universal2-3.11
./onionshare-cli

It worked! Okay, let's try the GUI...

./onionshare -v

It worked too!

OnionShare binary successfully running

Okay, this is great. This means that the problem is turning it into an app bundle, not the binary itself.

Debugging the app bundle

Next, I'll try delete the app bundle from the previous build and then run the script to build the app bundle and make sure it still crashes, just to be sure:

% rm -rf build/OnionShare.app
% poetry run python ./setup-freeze.py bdist_mac
running bdist_mac
running build_exe
creating directory /Users/user/code/onionshare/desktop/build/exe.macosx-10.9-universal2-3.11/lib
copying /Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/cx_Freeze/bases/lib/Python -> /Users/user/code/onionshare/desktop/build/exe.macosx-10.9-universal2-3.11/lib/Python
copying /Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/cx_Freeze/bases/console-cpython-311-darwin -> /Users/user/code/onionshare/desktop/build/exe.macosx-10.9-universal2-3.11/onionshare
--snip--

It finished. I'll try running the CLI binary from the app bundle first:

% ./build/OnionShare.app/Contents/MacOS/onionshare-cli 
╭───────────────────────────────────────────╮
│    *            ▄▄█████▄▄            *    │
│               ▄████▀▀▀████▄     *         │
│              ▀▀█▀       ▀██▄              │
│      *      ▄█▄          ▀██▄             │
│           ▄█████▄         ███        -+-  │
│             ███         ▀█████▀           │
│             ▀██▄          ▀█▀             │
│         *    ▀██▄       ▄█▄▄     *        │
│ *             ▀████▄▄▄████▀               │
│                 ▀▀█████▀▀                 │
│             -+-                     *     │
│   ▄▀▄               ▄▀▀ █                 │
│   █ █     ▀         ▀▄  █                 │
│   █ █ █▀▄ █ ▄▀▄ █▀▄  ▀▄ █▀▄ ▄▀▄ █▄▀ ▄█▄   │
│   ▀▄▀ █ █ █ ▀▄▀ █ █ ▄▄▀ █ █ ▀▄█ █   ▀▄▄   │
│                                           │
│                  v2.6.1                   │
│                                           │
│          https://onionshare.org/          │
╰───────────────────────────────────────────╯

usage: onionshare-cli [-h] [--receive] [--website] [--chat] [--local-only] [--connect-timeout SECONDS] [--config FILENAME] [--persistent FILENAME]
                      [--title TITLE] [--public] [--auto-start-timer SECONDS] [--auto-stop-timer SECONDS] [--no-autostop-sharing] [--data-dir data_dir]
                      [--webhook-url webhook_url] [--disable-text] [--disable-files] [--disable_csp] [--custom_csp custom_csp] [-v]
                      [filename ...]

positional arguments:
  filename                  List of files or folders to share

options:
  -h, --help                show this help message and exit
  --receive                 Receive files
--snip--

This worked. Now, the desktop version:

% ./build/OnionShare.app/Contents/MacOS/onionshare    
zsh: segmentation fault  ./build/OnionShare.app/Contents/MacOS/onionshare

This crashed. So most likely this has something to do with how PySide2 (which is what the desktop uses to display the GUI) is being packaged in the app bundle.

Since I know that the frozen app works when I built it, but not when I packaged it in an app bundle, I should compare the two and see what's different. Here are the files in the working build:

% ls -l build/exe.macosx-10.9-universal2-3.11 
total 304
-rw-r--r--    1 user  staff  35285 Oct  2  2022 LICENSE
-rw-r--r--    1 user  staff   3211 Sep 28 14:54 frozen_application_license.txt
drwxr-xr-x  249 user  staff   7968 Oct  9 20:54 lib
-rwxr-xr-x    1 user  staff  54736 Sep 28 14:54 onionshare
-rwxr-xr-x    1 user  staff  54736 Sep 28 14:54 onionshare-cli

And here are the files in the broken build (the app bundle):

% ls -l build/OnionShare.app/Contents/MacOS 
total 224
lrwxr-xr-x  1 user  staff     16 Oct  9 20:54 lib -> ../Resources/lib
-rwxr-xr-x  1 user  staff  54736 Oct  9 20:54 onionshare
-rwxr-xr-x  1 user  staff  54736 Oct  9 20:54 onionshare-cli

In the working build, the lib folder is in the same folder as the onionshare binary, but in the broken build, lib is a symbolic link to ../Resources/lib. Maybe it's choking on the symlink? Let's see what happens if I remove the symlink:

% cd build/OnionShare.app/Contents/MacOS 
% rm lib 
% mv ../Resources/lib .
% ./onionshare
zsh: segmentation fault  ./onionshare

It still crashes, so that's not it. After a cursory glance, the lib folders in the working and broken builds appear to be the same. But just to be safe, let me try copying the lib folder from the working build into the broken build and see if that fixes things.

% rm -rf lib
% cp -r ../../../exe.macosx-10.9-universal2-3.11/lib .
% ./onionshare
╭───────────────────────────────────────────╮
│    *            ▄▄█████▄▄            *    │
│               ▄████▀▀▀████▄     *         │
│              ▀▀█▀       ▀██▄              │
│      *      ▄█▄          ▀██▄             │
│           ▄█████▄         ███        -+-  │
--snip--

It worked! Let me try adding the symlink back and see if it still works...

% mv lib ../Resources 
% ln -s ../Resources/lib lib
% ./onionshare
╭───────────────────────────────────────────╮
│    *            ▄▄█████▄▄            *    │
│               ▄████▀▀▀████▄     *         │
│              ▀▀█▀       ▀██▄              │
│      *      ▄█▄          ▀██▄             │
│           ▄█████▄         ███        -+-  │
--snip--

Yup, still works!

So I think I have one potential way forward: When building the app bundle, I can just copy the lib folder from exe.macosx-10.9-universal2-3.11 into OnionShare.app. That feels hacky, but I do see any other obvious solutions, so I'm going for it.

Putting it all together

I'm going to start over from scratch and make sure this all works as intended. First, I'm going to delete the build folder, and rebuild the app bundle:

rm -rf build
poetry run python ./setup-freeze.py bdist_mac

I noticed that after this command finished, the new build folder had both exe.macos-10.9-universal2-3.11 and OnionShare.app in it, which means that I don't need to manually run poetry run python ./setup-freeze.py build first.

Next, I will delete the broken lib folder from the app bundle, and movethe working lib folder into its place:

rm -rf build/OnionShare.app/Contents/Resources/lib
mv build/exe.macosx-10.9-universal2-3.11/lib build/OnionShare.app/Contents/Resources/

Next, I'll run the last step, the cleanup script:

% poetry run python ./scripts/build-macos.py cleanup-build
> Delete unused Qt Frameworks
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/Qt/lib/QtMultimediaQuick.framework
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/Qt/lib/QtQuickControls2.framework
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/QtQuickControls2.abi3.so
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/QtQuickControls2.pyi
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/Qt/lib/QtQuickParticles.framework
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/Qt/lib/QtRemoteObjects.framework
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/QtRemoteObjects.abi3.so
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/QtRemoteObjects.pyi
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/lib/PySide6/Qt/lib/Qt3DInput.framework
--snip--
> Delete more unused PySide6 stuff to save space
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/Resources/lib/PySide6/Designer.app
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/Resources/lib/PySide6/glue
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/Resources/lib/PySide6/include
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/Resources/lib/PySide6/lupdate
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/Resources/lib/PySide6/Qt/qml
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/Resources/lib/PySide6/Assistant.app
Deleted: /Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/Resources/lib/PySide6/Linguist.app
--snip--
> Freed 1233 mb

Nice, it deleted an extra 1.2GB of data that we hopefully don't need to distribute. I'll try running it to make sure everything works:

./build/OnionShare.app/Contents/MacOS/onionshare

Yup! It works great.

Fixing the GitHub Actions build

As I described in part 2, whenever a commit gets pushed into a PR branch in the OnionShare project, there are GitHub Actions workflows that try to automatically build Windows and Mac binaries. When making a release, I'd like to download this Mac binary, make sure it works, and codesign it.

Here's the "Build OnionShare" step in the GitHub Actions workflow, in the build-mac-intel job:

- name: Build OnionShare
  run: |
    cd desktop
    /Library/Frameworks/Python.framework/Versions/3.10/bin/poetry run python ./setup-freeze.py build
    /Library/Frameworks/Python.framework/Versions/3.10/bin/poetry run python ./setup-freeze.py bdist_mac
    /Library/Frameworks/Python.framework/Versions/3.10/bin/poetry run python ./scripts/build-macos.py cleanup-build

I've updated it to this instead, basically to follow the steps I just manually performed:

- name: Build OnionShare
  run: |
    cd desktop
    /Library/Frameworks/Python.framework/Versions/3.10/bin/poetry run python ./setup-freeze.py bdist_mac
    rm -rf build/OnionShare.app/Contents/Resources/lib
    mv build/exe.macosx-10.9-universal2-3.10/lib build/OnionShare.app/Contents/Resources/
    /Library/Frameworks/Python.framework/Versions/3.10/bin/poetry run python ./scripts/build-macos.py cleanup-build

Notice that this time, the working lib folder is in a folder called exe.macos-11.0-universal2-3.10 instead of exe.macos-11.0-universal2-3.11 (3.10 instead of 3.11), because at the moment the GitHub Actions workflow is still using Python 3.10 -- I'll update that to 3.11 soon, but first I want to see if this works.

I committed my change to the GitHub actions workflow and pushed it to a PR branch on GitHub, and waited for GitHub Actions to kick in and build the Intel Mac binary.

Testing the Intel Mac binary from GitHub Actions

The GitHub Actions workflow generated artifacts, including mac-build one:

Artifacts generated by GitHub Actions

I downloaded all mac-build.zip and copied it to my Intel Mac build machine. From there, I extracted it and tried running the OnionShare app, and it worked!

OnionShare app bundle from GitHub Actions running in macOS

Updating dependencies again

I'm making great progress! I think now is a good time to pause and update all of the dependencies one final time. I'll start with the Python deps:

cd cli
poetry update
cd ../desktop
poetry update
cd ../docs
poetry update

Next, I need to make sure Snapcraft and Flatpak use these latest dependencies too. I won't have to make any changes for Snapcraft since it just pip installs the latest versions, but I do need to update the Flatpak manifest to use these new versions.

I did this using the patched flatpak-poetry-generator.py script from flatpak-builder-tools, just as I had described in part 2.

Next, I want to update Python across the board to 3.11.6. I updated the instructions in desktop/README.md to tell people to install the latest version 3.11, and I also updated the version in the GitHub Actions workflow for both build-win64 and build-mac-intel.

Adventures in ARM64 and GitHub Actions

Good thing this release is taking so long to finish, because at the beginning of October, GitHub announced support for M1 macOS runners, so now I can make the GitHub Actions workflow build Apple Silicon binaries as well!

Or, well, that's what I thought. I spent about an hour and a half of working on this. I made a new build-mac-arm64 job in my GitHub Actions workflow and made it run on it the macos-13 runner, got it working and creating app bundles, and even started testing it. That's when I realized that the macos-13 runner is still using an Intel processor, and the GitHub blog post I read about M1 runners was talking about "large runners" which are different than "standard GitHub-hosted runners", and are only available with paid GitHub plans. The OnionShare organization is using a free plan, so it's not actually available unless we pay for it.

We could maybe pay for a paid GitHub organization plan, but that's a separate topic. (Or if you're from GitHub and can donate Team plans to open source projects, we'd happily accept.)

So I git reset my branch back to before I started making the build-mac-arm64 job and force pushed, reverting all my work. Maybe some day. But in the mean time, it looks like I'll have to manually build the Apple Silicon version of OnionShare, and also manually merge it with the Intel version to make a universal2 app bundle.

Building OnionShare for Apple Silicon

I have a MacBook Pro with an M1 processor which I can use to make the ARM64 (a.k.a. Apple Silicon) build. But I use this computer on a regular basis and I have all sorts of development environment relics installed, and I'd prefer a clean environment for creating a release build. Plus, I have Rosetta installed on this computer, which allows me to run Intel binaries even though I have an ARM64 processor--I specifically don't want Rosetta installed in my ARM64 development environment (or at least test environment) so I can make sure the final build actually works on ARM64 and isn't actually using Intel in the background.

So, I opened the open source virtualization software UTM and installed a new macOS Sonoma VM. UTM is great, and is the only free VM software that works with Apple Silicon right now, there for macOS isn't support for things shared clipboard, changing the resolution, and not very good support for shared folders.

Setting up the development environment

Once I finished installing my new macOS VM, I set up an OnionShare development environment by following the instructions in desktop/README.md:

  • When I try running git clone to clone the source code the first time, it prompts me to install the Xcode command line developer tools.
  • After cloning the OnionShare git repo, I checkout the release-2.6.1 branch.
  • I downloaded and installed Python 3.11.6 from python.org.
  • I installed Poetry with pip3 install poetry. I also edited ~/.zshrc and this line, so that Python 3.11's bin dir is in the path: export PATH=$PATH:/Library/Frameworks/Python.framework/Versions/3.11/bin.
  • Installed Poetry dependencies by running: cd desktop; poetry install.
  • Downloaded Tor binaries from Tor Browser by running: poetry run python ./scripts/get-tor.py macos.

When I did this last step I hit an error. The get-tor.py script was trying to subprocess out to gpg, but I didn't have it installed. So, I installed Homebrew and then install GnuPG with brew install gnupg, and then tried it again. This time it worked. Then I moved on to the rest of the steps:

  • I installed Go from https://go.dev/dl/, making sure to use the ARM64 version. I downloaded go1.21.3.darwin-arm64.pkg.
  • I compiled the pluggable transports (for ARM64) by running:
    ./scripts/build-pt-obfs4proxy.sh
./scripts/build-pt-snowflake.sh
./scripts/build-pt-meek.sh
  • Finally, I ran OnionShare from the source tree by running poetry run onionshare -v, just to make sure it's all working. Nice, the OnionShare window popped up.

With this, my VM is set up for OnionShare development. Now I need to create an app bundle. To remind myself how to do that I looked at the GitHub Actions workflow for build-mac-intel and remembered that there's this step too:

- name: Install cx_Freeze/PySide6 build dependencies
  run: |
    brew install libiodbc
    cd ~/Downloads
    curl -O -L https://github.com/PostgresApp/PostgresApp/releases/download/v2.6.5/Postgres-2.6.5-14.dmg
    hdiutil attach Postgres-2.6.5-14.dmg
    cp -r /Volumes/Postgres-2.6.5-14/Postgres.app /Applications/
    hdiutil detach /Volumes/Postgres-2.6.5-14

So before building the app bundle:

  • Installed libiodbc by running: brew install libiodbc.
  • Downloaded Postgres-2.6.5-14.dmg, opened it, and dragged it to Applications.

Building the app bundle

Here's the "Build OnionShare step" from the GitHub Actions workflow:

- name: Build OnionShare
  run: |
    cd desktop
    /Library/Frameworks/Python.framework/Versions/3.11/bin/poetry run python ./setup-freeze.py bdist_mac
    rm -rf build/OnionShare.app/Contents/Resources/lib
    mv build/exe.macosx-10.9-universal2-3.11/lib build/OnionShare.app/Contents/Resources/
    /Library/Frameworks/Python.framework/Versions/3.11/bin/poetry run python ./scripts/build-macos.py cleanup-build

So I simply have to run these manually:

% poetry run python ./setup-freeze.py bdist_mac
running bdist_mac
running build_exe
--snip--
copying /Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/PySide6/Qt/plugins/sqldrivers/libqsqlodbc.dylib -> /Users/user/code/onionshare/desktop/build/exe.macosx-10.9-universal2-3.11/lib/PySide6/Qt/plugins/sqldrivers/libqsqlodbc.dylib
copying /usr/local/opt/libiodbc/lib/libiodbc.2.dylib -> /Users/user/code/onionshare/desktop/build/exe.macosx-10.9-universal2-3.11/lib/libiodbc.2.dylib
error: [Errno 2] No such file or directory: '/usr/local/opt/libiodbc/lib/libiodbc.2.dylib'

Hmm, it can't find libiodbc.2.dylib, but I did install the libiodbc package.

Ahh, it looks like Homebrew put it in /opt/homebrew/opt/libiodbc/, but my build script is looking for it in /usr/local/opt/libiodbc/. I can fix that by creating a symlink and trying again:

% sudo mkdir /usr/local/opt
% sudo ln -s /opt/homebrew/opt/libiodbc /usr/local/opt/libiodbc
% poetry run python ./setup-freeze.py bdist_mac
running bdist_mac
running build_exe
--snip--
Setting relative_reference_path for: Mach-O File: /Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/cx_Freeze/initscripts/frozen_application_license.txt
Resolved rpath:
Loaded libraries:
Applying AdHocSignature
/Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/frozen_application_license.txt: No such file or directory
error: [Errno 2] No such file or directory: '/Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/frozen_application_license.txt'

Nice, that problem is fixed, but now I'm hitting another.

frozen_application_license.txt is a license file that cx_Freeze adds to binaries it creates, but for some reason it didn't copy it into the app bundle. But even if it did, the output says it's trying to sign that file, and in macOS you can't sign text files, only Mach-O binaries, so it shouldn't be doing that. What's going on?

Fixing cx_Freeze

I spent some time searching the internet and reading issues in the cx_Freeze GitHub repo, but still didn't come to a firm solution. So I decided to dig into the source code and see what's going on. In cx_Freeze code's cx_Freeze/command/bdist_mac.py, I see a few relevant functions a function called set_relative_reference_paths. Here's the code for that function:

def set_relative_reference_paths(self, build_dir: str, bin_dir: str):
    """Make all the references from included Mach-O files to other included
    Mach-O files relative.
    """
    darwin_file: DarwinFile

    for darwin_file in self.darwin_tracker:
        # get the relative path to darwin_file in build directory
        print(f"Setting relative_reference_path for: {darwin_file}")
        relative_copy_dest = os.path.relpath(
            darwin_file.getBuildPath(), build_dir
        )
        # figure out directory where it will go in binary directory for
        # .app bundle, this would be the Content/MacOS subdirectory in
        # bundle.  This is the file that needs to have its dynamic load
        # references updated.
        file_path_in_bin_dir = os.path.join(bin_dir, relative_copy_dest)
        # for each file that this darwin_file references, update the
        # reference as necessary; if the file is copied into the binary
        # package, change the reference to be relative to @executable_path
        # (so an .app bundle will work wherever it is moved)
        for reference in darwin_file.getMachOReferenceList():
            if not reference.is_copied:
                # referenced file not copied -- assume this is a system
                # file that will also be present on the user's machine,
                # and do not change reference
                continue
            # this is the reference in the machO file that needs to be
            # updated
            raw_path = reference.raw_path
            ref_target_file: DarwinFile = reference.target_file
            # this is where file copied in build dir
            abs_build_dest = ref_target_file.getBuildPath()
            rel_build_dest = os.path.relpath(abs_build_dest, build_dir)
            exe_path = f"@executable_path/{rel_build_dest}"
            changeLoadReference(
                file_path_in_bin_dir,
                oldReference=raw_path,
                newReference=exe_path,
                VERBOSE=False,
            )

        applyAdHocSignature(file_path_in_bin_dir)

This is where the problem is -- remember, this is part of the error message:

Setting relative_reference_path for: Mach-O File: /Users/user/Library/Caches/pypoetry/virtualenvs/onionshare-aqknF-N0-py3.11/lib/python3.11/site-packages/cx_Freeze/initscripts/frozen_application_license.txt

This same file also includes a function called set_absolute_reference_paths, and that function has some logic to specifically skip txt and zip files, and I think some similar logic here might fix my problem. So I forked cx_Freeze and patched the code. Here's my patch:

diff --git a/cx_Freeze/command/bdist_mac.py b/cx_Freeze/command/bdist_mac.py
index 44cb03e..0cee824 100644
--- a/cx_Freeze/command/bdist_mac.py
+++ b/cx_Freeze/command/bdist_mac.py
@@ -332,6 +332,10 @@ class BdistMac(Command):
         darwin_file: DarwinFile

         for darwin_file in self.darwin_tracker:
+            # Skip text files
+            if str(darwin_file.path).endswith(".txt"):
+                continue
+
             # get the relative path to darwin_file in build directory
             print(f"Setting relative_reference_path for: {darwin_file}")
             relative_copy_dest = os.path.relpath(

I ran the build again, and this time it worked! At least, until it hit this error:

% poetry run python ./setup-freeze.py bdist_mac
running bdist_mac
running build_exe
--snip--
Setting relative_reference_path for: Mach-O File: /Users/user/code/onionshare/LICENSE
Resolved rpath:
Loaded libraries:
Applying AdHocSignature
/Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/LICENSE: No such file or directory
error: [Errno 2] No such file or directory: '/Users/user/code/onionshare/desktop/build/OnionShare.app/Contents/MacOS/LICENSE'

The LICENSE file is OnionShare's license, and it gets included in the app bundle. It's not skipping it because it's not a .txt file. So, I'm just going to go ahead and rename it to LICENSE.txt, and also update desktop/setup-freeze.py to include LICENSE.txt instead of LICENSE. Let's see if that does the trick.

It finished without errors! Almost there. Before moving on, I opened a PR in the cx_Freeze project with my patch.

Finishing the build

Again, here's the full "Build OnionShare" step in the GitHub Actions workflow:

- name: Build OnionShare
  run: |
    cd desktop
    /Library/Frameworks/Python.framework/Versions/3.11/bin/poetry run python ./setup-freeze.py bdist_mac
    rm -rf build/OnionShare.app/Contents/Resources/lib
    mv build/exe.macosx-10.9-universal2-3.11/lib build/OnionShare.app/Contents/Resources/
    /Library/Frameworks/Python.framework/Versions/3.11/bin/poetry run python ./scripts/build-macos.py cleanup-build

I successfully ran poetry run python ./setup-freeze.py bdist_mac, and now I need to run the rest of the commands:

% rm -rf build/OnionShare.app/Contents/Resources/lib
% mv build/exe.macosx-10.9-universal2-3.11/lib build/OnionShare.app/Contents/Resources/
% poetry run python ./scripts/build-macos.py cleanup-build
--snip--
> Freed 1233 mb

No problems yet. Can I run the binary?

Running the OnionShare app bundle in the Mac VM

I can! I've created a working Apple Silicon binary in build/OnionShare.app.

Merging app bundles into a universal2 Mac app bundle

My original plan for this release was to make two version for Mac, Intel and Apple Silicon. However, I'm sooo close I might as well just go the final yard and merge these two app bundles into a single universal2 app bundle.

In macOS, Mach-O binaries are compiled for a specific architecture, like arm64 or x86_64. Using a program called lipo you can merge binaries of different architectures into the same file so that it will run on either architecture--this is called universal2 (the original universal format for Mach-O binaries that had both PowerPC and Intel architectures, from back in the day).

According to Apple's documentation, if you have an Intel binary called x86_app and an ARM64 binary called arm_app, you can create a universal2 binary called universal_app with this command:

lipo -create -output universal_app x86_app arm_app

At this point in the release I have an Intel app bundle and an ARM64 app bundle. To turn this into a single universal2 app bundle, I just need a simple script that looks through all of the files in both app bundles for binaries to merge, and use lipo to merge them. Hang on while I do that.

🎵 Programming sounds... 🎵

Alright, I've created a new script, desktop/scripts/macos-merge-universal.py, which does just this:

#!/usr/bin/env python3
import os
import shutil
import click
import subprocess


def get_binary_arches(app_dir):
    universal = []
    silicon = []
    intel = []
    for dirpath, dirnames, filenames in os.walk(app_dir):
        for basename in filenames:
            filename = os.path.join(dirpath, basename)
            if os.path.isfile(filename):
                out = subprocess.check_output(["file", filename]).decode("utf-8")
                if (
                    "Mach-O 64-bit executable" in out
                    or "Mach-O 64-bit bundle" in out
                    or "Mach-O 64-bit dynamically linked shared library" in out
                ):
                    arm64, x86 = False, False
                    if "arm64" in out:
                        arm64 = True
                    if "x86_64" in out:
                        x86 = True

                    if arm64 and x86:
                        universal.append(filename)
                    elif arm64:
                        silicon.append(filename)
                    elif x86:
                        intel.append(filename)

    return universal, silicon, intel


@click.command()
@click.argument("intel_app", type=click.Path(exists=True))
@click.argument("silicon_app", type=click.Path(exists=True))
@click.argument("output_app", type=click.Path(exists=False))
def main(intel_app, silicon_app, output_app):
    # Get the list of binaries in each app
    print("Looking up binaries from Intel app:", intel_app)
    intel_universal, intel_silicon, intel_intel = get_binary_arches(intel_app)
    print("Looking up binaries from Silicon app:", silicon_app)
    silicon_universal, silicon_silicon, silicon_intel = get_binary_arches(silicon_app)

    # Find which binaries should be merged
    intel_intel_filenames = [i[len(intel_app) + 1 :] for i in intel_intel]
    silicon_silicon_filenames = [i[len(silicon_app) + 1 :] for i in silicon_silicon]
    intersection = set(intel_intel_filenames).intersection(
        set(silicon_silicon_filenames)
    )

    # Copy the Silicon app to the output app
    print("Copying the app bundle for the output app")
    shutil.copytree(silicon_app, output_app, symlinks=True)

    # Merge them
    for filename in intersection:
        print(f"Merging {filename}")
        intel_binary = os.path.join(intel_app, filename)
        silicon_binary = os.path.join(silicon_app, filename)
        output_binary = os.path.join(output_app, filename)
        subprocess.run(
            ["lipo", "-create", intel_binary, silicon_binary, "-output", output_binary]
        )

    print(f"Merge complete: {output_app}")


if __name__ == "__main__":
    main()

I put the ARM64 app bundle in a folder called ~/tmp/arm64 and the Intel app bundle in ~/tmp/intel, and I made a new empty folder called ~/tmp/universal2. Let's see this script in action:

% poetry run ./scripts/macos-merge-universal.py ~/tmp/intel/OnionShare.app ~/tmp/arm64/OnionShare.app ~/tmp/universal2/OnionShare.app
Looking up binaries from Intel app: /Users/user/tmp/intel/OnionShare.app
Looking up binaries from Silicon app: /Users/user/tmp/arm64/OnionShare.app
Copying the app bundle for the output app
Merging Contents/Resources/lib/psutil/_psutil_posix.abi3.so
Merging Contents/Resources/lib/zope/interface/_zope_interface_coptimizations.cpython-311-darwin.so
Merging Contents/Resources/lib/psutil/_psutil_osx.abi3.so
Merging Contents/Resources/lib/libiodbc.2.dylib
Merging Contents/Resources/lib/charset_normalizer/md.cpython-311-darwin.so
Merging Contents/MacOS/onionshare
Merging Contents/MacOS/onionshare-cli
Merging Contents/Resources/lib/charset_normalizer/md__mypyc.cpython-311-darwin.so
Merging Contents/Resources/lib/_cffi_backend.cpython-311-darwin.so
Merge complete: /Users/user/tmp/universal2/OnionShare.app

It found that onionshare-cli and onionshare needed to be merged, but it also found a bunch of other binaries that needed merging too which I didn't know about. I also expected that I'd have to merge all of the pluggable transports that I built with Go (obfs4proxy, snowflake-client, and meek-client) but was pleasantly surprised to find that I didn't have to. It turns out that if you're using the ARM64 Go compiler, it makes universal2 binaries by default.

As a final test, I copied my new universal2 binary to my two machines (my physical Intel Mac build machine, and my new ARM64 Mac VM) and tried running it in both places to make sure it would work. They work!

As a final step, I updated RELEASE.md with instructions on how to make the ARM64 app bundle, and how to merge them into a universal2 app bundle.

Making a development release

If this release weren't such an epic slog I would maybe just go ahead and make the final OnionShare 2.6.1 release. But as it is, I think it's much safer to make a development release, with development binaries and a development tag, and have the other OnionShare devs test it out and review my PR before making the actual release. So I decided that right now I'm going to release version 2.6.1.dev1.

I updated the version string from 2.6.1 to 2.6.1.dev1 in all of the places I've documented at the top of RELEASE.md, committed this, and then made a signed git tag:

git tag -s v2.6.1.dev1

I pushed my commits and my new tag, and waited for the GitHub Actions workflow to finish creating builds.

Windows release

I switched to my Ubuntu computer and booted my Windows 11 VM that I created in part 2 while troubleshooting the Windows build.

I'll start by running git fetch, verifying the PGP signature on the v2.6.1.dev1 tag I just created, and checking out that tag, so I'm sure I'm working from the correct branch.

Verifying the git tag in Windows

Last time I had set up a Windows development environment for OnionShare, but there's still a few more things I need to install to actually make a release. Also, I need to learn how to use my new HARICA smart card, which I haven't used yet! I'm going to start with that, setting up HARICA.

Setting up the HARICA smart card

I got this USB smart card from HARICA, but that's the extent of my experience with this CA.

HARICA smart card

I plugged it into my computer and then used VirtualBox to set up USB passthrough to pass it to the Windows VM. VirtualBox sees this device as "SafeNet Token JC".

I followed the instructions that HARICA sent when we ordered the code signing certificate: I installed the SafeNet Authentication Client software, logged into the HARICA control panel and downloaded the signed certificate, and then I used the SafeNet client software to import it into the smart card. I also changed the token password (PIN) and administrator password (PUK) on my smart card.

Viewing the smart card info

Note that the token name is Science and Design Inc. As I mentioned in part 1, the new nonprofit Science & Design, founded by Glenn Sorrentino (who designed the beautiful OnionShare UX!), has taken on the role of fiscal sponsor for OnionShare. They purchased the new code signing certificate from HARICA.

Now let's see if it works. In a Developer PowerShell for VS window:

cp C:\Windows\System32\calc.exe .
signtool.exe /v /d "Calc test" /n "Science and Design Inc." /fd sha256 /td sha256 /tr /tr http://timestamp.digicert.com calc.exe

I copied calc.exe into my home folder and then used signtool.exe to digitally sign it.

Signing calc.exe

The SafeNet Authentication Client popped up a window asking for my token password. I entered it, and then the secret key on my smart card digitally signed calc.exe. I inspected the digital signature, and it works!

Viewing digital signature info for calc.exe

Code signing and packaging

The instructions from RELEASE.md say to install the Windows SDK, to install the .NET Framework 3.5 SP1 Runtime, and to install WiX 3.11, which is the software I use to create the Windows MSI package.

It looks like since the last OnionShare release there's been a major update in WiX and now version 4 is out. Upgrading from WiX 3 to 4 looks like a huge project, so I'm going to skip that for this release and still use the older version of WiX, but it's a good thing to focus on in the future. You can still find information on using WiX 3.11 here -- I updated the RELEASE.md docs slightly to reflect this.

So, I went ahead and installed the Windows SDK, installed .NET 3.5, and installed WiX 3.11.

The next step in RELEASE.md is to download the Windows binaries from GitHub Actions, so I downloaded the win64-build artifact from GitHub and extracted it into my Downloads folder. The RELEASE.md instructions say to run these commands:

poetry run python .\scripts\build-windows.py codesign [path]
poetry run python .\scripts\build-windows.py package [path]

The first command uses signtool.exe to digitally sign all of the binaries (onionshare.exe, onionshare-cli.exe, obfs4proxy.exe, meek-client.exe, and snowflake-client.exe). The second command uses the WiX toolset to create the MSI.

Before running the codesign command though, I updated the sign() function in build-windows.py so that it will use the new Science & Design certificate, instead of the old one (which was from a CA called Certum, under the name "Open Source Developer, Micah Lee").

Then I ran:

poetry run python .\scripts\build-windows.py codesign C:\Users\dev\Downloads\onionshare-win64\

And it worked! It prompted me for the token password for each binary it signed, and one by one it signed them all. Then I ran the build-windows.py package command:

PS C:\Users\dev\code\onionshare\desktop> poetry run python .\scripts\build-windows.py package C:\Users\dev\Downloads\onionshare-win64\
> Build the WiX file
> Build the MSI
['C:\\Program Files (x86)\\WiX Toolset v3.11\\bin\\candle.exe', 'OnionShare.wxs']
Windows Installer XML Toolset Compiler version 3.11.2.4516
Copyright (c) .NET Foundation and contributors. All rights reserved.

OnionShare.wxs
['C:\\Program Files (x86)\\WiX Toolset v3.11\\bin\\light.exe', '-ext', 'WixUIExtension', 'OnionShare.wixobj']
Windows Installer XML Toolset Linker version 3.11.2.4516
Copyright (c) .NET Foundation and contributors. All rights reserved.

C:\Users\dev\Downloads\onionshare-win64\OnionShare.wxs(16) : warning LGHT1076 : ICE61: This product should remove only older versions of itself. The Maximum version is not less than the current product. (2.6.1 2.6.1)
> Prepare OnionShare.msi for signing
['C:\\Program Files (x86)\\WiX Toolset v3.11\\bin\\insignia.exe', '-im', 'C:\\Users\\dev\\Downloads\\onionshare-win64\\OnionShare.msi']
Windows Installer XML Toolset Inscriber version 3.11.2.4516
Copyright (c) .NET Foundation and contributors. All rights reserved.

> Signing C:\Users\dev\Downloads\onionshare-win64\OnionShare.msi
['C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.22621.0\\\\x64\\signtool.exe', 'sign', '/v', '/d', 'OnionShare', '/n', 'Science and Design Inc.', '/fd', 'sha256', '/td', 'sha256', '/tr', 'http://timestamp.digicert.com', 'C:\\Users\\dev\\Downloads\\onionshare-win64\\OnionShare.msi']
The following certificate was selected:
    Issued to: Science and Design Inc.
    Issued by: HARICA Code Signing RSA
    Expires:   Thu Jul 17 23:30:04 2025
    SHA1 hash: 0BD01CB561295C13EC72FF5ED7BB1AF65C70B723

Done Adding Additional Store
Successfully signed: C:\Users\dev\Downloads\onionshare-win64\OnionShare.msi

Number of files successfully Signed: 1
Number of warnings: 0
Number of errors: 0
> Final MSI: C:\Users\dev\code\onionshare\desktop\dist\OnionShare-win64-2.6.1.dev1.msi

This worked too. The final step code signed the MSI, and I was prompted for my token password. Now I have the final installer package, OnionShare-win64-2.6.1.dev1.msi. When I double-click it, it opens the installer.

OnionShare Windows installer

And User Access Control asks if I want to install OnionShare, listing Science and Design Inc. as the verified publisher.

OnionShare UAC

The install finishes successfully! Ahh, but there's a problem. It installed it in C:\Program Files (x86)\OnionShare, but that's where 32-bit Windows software is supposed to go. When we upgraded to PySide 6 we decided to stop releasing 32-bit Windows versions, since PySide 6 doesn't support it. The install should be putting OnionShare in C:\Program Files, not C:\Program Files (x86).

I uninstalled OnionShare and then went to work updating build-windows.py to support installing installing in Program Files instead. I won't bore you with the debugging details, but basically, this patch did it:

diff --git a/desktop/scripts/build-windows.py b/desktop/scripts/build-windows.py
index 224af59b..9579b81d 100644
--- a/desktop/scripts/build-windows.py
+++ b/desktop/scripts/build-windows.py
@@ -117,6 +117,7 @@ def wix_build_dir_xml(root, data):
             "Component",
             Id="ApplicationShortcuts",
             Guid="539e7de8-a124-4c09-aa55-0dd516aad7bc",
+            Win64="yes",
         )
         ET.SubElement(
             component_el,
@@ -152,6 +153,7 @@ def wix_build_components_xml(root, data):
                 "Component",
                 Id=subdata["component_id"],
                 Guid=subdata["component_guid"],
+                Win64="yes",
             )
             for filename in subdata["files"]:
                 file_el = ET.SubElement(
@@ -180,7 +182,7 @@ def msi_package(build_path, msi_path, product_update_code):
         "name": "SourceDir",
         "dirs": [
             {
-                "id": "ProgramFilesFolder",
+                "id": "ProgramFiles64Folder",
                 "dirs": [],
             },
             {
@@ -218,10 +220,11 @@ def msi_package(build_path, msi_path, product_update_code):
         Keywords="Installer",
         Description="OnionShare $(var.ProductVersion) Installer",
         Manufacturer="Micah Lee, et al.",
-        InstallerVersion="100",
+        InstallerVersion="200",
         Languages="1033",
         Compressed="yes",
         SummaryCodepage="1252",
+        Platform="x64",
     )
     ET.SubElement(product_el, "Media", Id="1", Cabinet="product.cab", EmbedCab="yes")
     ET.SubElement(

I re-ran the build-windows.py package command and it created a new signed MSI installer. I installed this one, and I confirmed that it successfully installed in C:\Program Files\OnionShare. I then ran it OnionShare from the Start menu:

Running the final signed and installed OnionShare in Windows

Nice! The Windows development release is done.

macOS release

Time to do the macOS release. First, I'll start with manually building the Apple Silicon app bundle.

Building the Apple Silicon app bundle

Back on my Apple Silicon Mac computer, I booted up my macOS development VM. I opened a terminal, changed to the onionshare folder, verified the PGP signature on v2.6.1.dev1 tag, and checked it out:

git fetch
git tag -v v2.6.1.dev1
git checkout v2.6.1.dev1

Following the instructions in RELEASE.md, I made sure all of the dependencies are installed and compiled:

cd desktop
python3 -m pip install poetry
poetry install
poetry run python ./scripts/get-tor.py macos
./scripts/build-pt-obfs4proxy.sh
./scripts/build-pt-snowflake.sh
./scripts/build-pt-meek.sh

After re-installing Poetry deps, I also patched cx_Freeze in the Poetry virtual environment to include the changes I made in my PR (which I just noticed has been merged!).

Then I followed the instructions to create the Apple Silicon app bundle:

poetry run python ./setup-freeze.py bdist_mac
rm -rf build/OnionShare.app/Contents/Resources/lib
mv build/exe.macosx-10.9-universal2-3.11/lib build/OnionShare.app/Contents/Resources/
poetry run python ./scripts/build-macos.py cleanup-build

And I end up with an Apple Silicon OnionShare.app app bundle. I opened Finder, found my newly created app bundle, and right-clicked on it to compress it. I then transferred OnionShare.app.zip out of my VM and onto my older Intel Mac that has my Apple Developer signing keys on it, which I'm using as a dedicated build machine.

Creating the universal2 app bundle

I manually made the Apple Silicon app bundle, but my GitHub Actions workflow made the Intel one for me. I downloaded the mac-intel-build artifact from GitHub and copied that to my dedicated build machine too.

On my build computer, I created a folder in my desktop called v2.6.1.dev1 with the following structure:

Desktop/
└── v2.6.1.dev1/
    ├── intel/
    │   └── OnionShare.app
    └── silicon/
        └── OnionShare.app

Then I merged the ARM64 and Intel app bundles into a single universal2 bundle:

poetry run ./scripts/macos-merge-universal.py ~/Desktop/v2.6.1.dev1/intel ~/Desktop/v2.6.1.dev1/silicon ~/Desktop/v2.6.1.dev1/universal2

I then code signed it using my Apple Developer signing keys:

% poetry run python ./scripts/build-macos.py codesign ~/Desktop/v2.6.1.dev1/universal2/OnionShare.app
['codesign', '--sign', 'Developer ID Application: Micah Lee (N9B95FDWH4)', '--entitlements', '/Users/user/code/onionshare/desktop/package/Entitlements.plist', '--timestamp', '--deep', '--force', '--options', 'runtime,library', '/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app/Contents/Resources/lib/_sqlite3.cpython-311-darwin.so'] # cwd=None
/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app/Contents/Resources/lib/_sqlite3.cpython-311-darwin.so: replacing existing signature
['codesign', '--sign', 'Developer ID Application: Micah Lee (N9B95FDWH4)', '--entitlements', '/Users/user/code/onionshare/desktop/package/Entitlements.plist', '--timestamp', '--deep', '--force', '--options', 'runtime,library', '/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app/Contents/Resources/lib/_scproxy.cpython-311-darwin.so'] # cwd=None
--snip--
/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app/Contents/Frameworks/QtCore.framework/Versions/A/QtCore: No such file or directory
Traceback (most recent call last):
  File "/Users/user/code/onionshare/desktop/./scripts/build-macos.py", line 303, in <module>
    main()
--snip--
subprocess.CalledProcessError: Command '['codesign', '--sign', 'Developer ID Application: Micah Lee (N9B95FDWH4)', '--entitlements', '/Users/user/code/onionshare/desktop/package/Entitlements.plist', '--timestamp', '--deep', '--force', '--options', 'runtime,library', '/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app/Contents/Frameworks/QtCore.framework/Versions/A/QtCore']' returned non-zero exit status 1.

Oops. It looks like it's trying to sign OnionShare.app/Contents/Frameworks/QtCore.framework/Versions/A/QtCore, but that file doesn't exist.

Fixing macOS code signing

Let's pull up the code...

@main.command()
@click.argument("app_path")
def codesign(app_path):
    """Sign macOS binaries before packaging"""
    for path in itertools.chain(
        glob.glob(f"{app_path}/Contents/Resources/lib/**/*.so", recursive=True),
        glob.glob(f"{app_path}/Contents/Resources/lib/**/*.dylib", recursive=True),
        [
            f"{app_path}/Contents/Frameworks/QtCore.framework/Versions/A/QtCore",
            f"{app_path}/Contents/Frameworks/QtDBus.framework/Versions/A/QtDBus",
            f"{app_path}/Contents/Frameworks/QtGui.framework/Versions/A/QtGui",
            f"{app_path}/Contents/Frameworks/QtWidgets.framework/Versions/A/QtWidgets",
            f"{app_path}/Contents/Resources/lib/Python",
            f"{app_path}/Contents/Resources/lib/onionshare/resources/tor/meek-client",
            f"{app_path}/Contents/Resources/lib/onionshare/resources/tor/obfs4proxy",
            f"{app_path}/Contents/Resources/lib/onionshare/resources/tor/snowflake-client",
            f"{app_path}/Contents/Resources/lib/onionshare/resources/tor/tor",
            f"{app_path}/Contents/Resources/lib/onionshare/resources/tor/libevent-2.1.7.dylib",
            f"{app_path}/Contents/MacOS/onionshare",
            f"{app_path}/Contents/MacOS/onionshare-cli",
            f"{app_path}",
        ],
    ):
        sign(path, entitlements_plist_path, identity_name_application)

    print(f"> Signed app bundle: {app_path}")

This code basically signs all of the .so files in the app bundle, signs all of the .dylib files, and then goes and signs a bunch of individual Mach-O binaries, and finally the app bundle itself. I forgot why this was necessary, other than just signing the app bundle itself would miss some of the binaries.

The problem is that with the new versions of PySide6 and cx_Freeze, these files have all moved around. Hmm, maybe I can solve this problem by writing code that just searches for all Mach-O binaries and signs them, so it will work in the future no matter the future file structure. In fact, I already wrote a function that finds these binaries in macos-merge-universal.py, the get_binary_arches() function.

In desktop/scripts, I made a new file called common.py and moved the get_binary_arches() function into it. I deleted the function from macos-merge-universal.py, and instead just imported it at the top:

from common import get_binary_arches

I also imported get_binary_arches in build-macos.py, and I updated the codesign() function to just be this:

@main.command()
@click.argument("app_path")
def codesign(app_path):
    """Sign macOS binaries before packaging"""
    bin_universal, bin_silicon, bin_intel = get_binary_arches(app_path)
    binaries = bin_universal + bin_silicon + bin_intel + [app_path]

    for filename in binaries:
        sign(filename, entitlements_plist_path, identity_name_application)

    print(f"> Signed app bundle: {app_path}")

This now individually signs every binary, and then signs the app bundle itself. Let's see if it works:

% poetry run python ./scripts/build-macos.py codesign ~/Desktop/v2.6.1.dev1/universal2/OnionShare.app
['codesign', '--sign', 'Developer ID Application: Micah Lee (N9B95FDWH4)', '--entitlements', '/Users/user/code/onionshare/desktop/package/Entitlements.plist', '--timestamp', '--deep', '--force', '--options', 'runtime,library', '/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app/Contents/MacOS/onionshare'] # cwd=None
/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app/Contents/MacOS/onionshare: replacing existing signature
--snip--
['codesign', '--sign', 'Developer ID Application: Micah Lee (N9B95FDWH4)', '--entitlements', '/Users/user/code/onionshare/desktop/package/Entitlements.plist', '--timestamp', '--deep', '--force', '--options', 'runtime,library', '/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app'] # cwd=None
/Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app: replacing existing signature
> Signed app bundle: /Users/user/Desktop/v2.6.1.dev1/universal2/OnionShare.app

I now have a code-signed app bundle.

Packaging the app

Now that I have a code-signed universal2 binary, I need to package it into a DMG for distribution--one of those things that you can open and then drag an app into Applications. OnionShare uses create-dmg to make this simple.

% poetry run python ./scripts/build-macos.py package ~/Desktop/v2.6.1.dev1/universal2/OnionShare.app 
> Create DMG
--snip--
Disk image done
> Finished building DMG: /Users/user/code/onionshare/desktop/dist/OnionShare-2.6.1.dev1.dmg

I have OnionShare-2.6.1.dev1.dmg. The final step is to notarize it.

Notarizing the app

For the last few years, Apple has been requiring all apps to get notarized. This means developers need to upload their app to Apple's notarization service which thoroughly scans it and makes sure it complies with Apple's security policies (for example, requiring that everything is code signed).

If it fails Apple's notarization, you get a report that explains what's wrong with it, and Mac users can't run your app. If it passes, you get a notarization "ticket". If someone tries running the app, macOS will query their notarization service to see if there's a valid ticket, and if there is it will let the app proceed. But to avoid having macOS querying Apple's service, you can "staple" the "ticket" to the app, so that macOS already knows it's passed notarization.

So, following the instructions in RELEASE.md, I uploaded the DMG to Apple's notarization service:

xcrun altool --notarize-app --primary-bundle-id "com.micahflee.onionshare" -u "[email protected]" -p "$APPLE_PASSWORD" --file dist/OnionShare-$VERSION.dmg

In this case, $APPLE_PASSWORD is an app-specific password for my Apple ID account, and $VERSION is v2.6.1.dev1. This command took a few minutes to run because it had to upload the 209MB DMG to Apple's service. Now that it's done uploading, we wait.

Email from Apple's notarization service

It passed notarization! Now I just need to staple the ticket.

% xcrun stapler staple dist/OnionShare-$VERSION.dmg
Processing: /Users/user/code/onionshare/desktop/dist/OnionShare-2.6.1.dev1.dmg
Processing: /Users/user/code/onionshare/desktop/dist/OnionShare-2.6.1.dev1.dmg
The staple and validate action worked!

Testing it out

On my old Intel Mac, I double-clicked on OnionShare-2.6.1.dev1.dmg and dragged OnionShare into Applications. I then went to Applications and double-clicked on OnionShare, and the OnionShare window opened. I clicked around, and everything seems to work!

I copied the DMG to my Apple Silicon Mac and did the same: I opened the DMG, dragged the app into Applications, and then ran it, and it worked there too!

The macOS development release is done.

Testing Flatpak

sh the Flatpak package to Flathub and I publish the Snapcraft package to snapcraft.io, the central repository. But I also distribute signed .flatpak and .snap files for people to download and install if they prefer. Both of these files are generated by the GitHub Actions workflow.

On my Ubuntu computer, I found the GitHub Actions workflow and downloaded flatpak-build.zip. I unzipped it found OnionShare.flatpak. I installed it by running:

flatpak install ./OnionShare.flatpak

To install it, you still need to download many dependencies from Flathub. Once it's installed, I ran it with:

flatpak run org.onionshare.OnionShare

The OnionShare window opened and everything seems to work.

Testing Snapcraft

I found the GitHub Actions workflow and downloaded snapcraft-build.zip. I unzipped it found onionshare_2.6.1.dev1_amd64.snap. I installed it by running:

sudo snap install --devmode ./onionshare_2.6.1.dev1_amd64.snap

Once it was installed, I ran it with:

/snap/bin/onionshare

The OnionShare window opened and everything seems to work.

OnionShare v2.6.1.dev1 released

And with that, I've made a development release! You can find the MSI Windows installer, the DMG macOS installer, and the Flatpak and Snapcraft packages, here: https://github.com/onionshare/onionshare/releases/tag/v2.6.1.dev1

If you're curious, go ahead and test them out, and submit any bugs you find by opening GitHub issues.

Whew, this has been a ridiculous process, but I'm glad it's nearly done! I expect part 4 will be the final part in this epic saga, and also hopefully it will be much shorter, because all I'll need to do in it is make the final release (unless typing those words already jinxed it).

Micah Lee is a security engineer, software developer, and a journalist. He's the author of Hacks, Leaks, and Revelations and is the person behind Semiphemeral.
Read more →

Buy Hacks, Leaks, and Revelations: The Art of Analyzing Hacked and Leaked Data

Semiphemeral

Creative Commons License

Unless otherwise specified, blog posts are licensed under Creative Commons Attribution-NonCommercial. You are free to share and adapt this content, but you must give me credit, you can't use it for commercial purposes, and if you make adaptations (like a translation), you must make it clear that it has been modified.

Subscribe to feed