Install python on Rosetta 2 on M1 Mac

2023-06-03 923 words 5 minutes

Contents

Azure Functions for Python language supports up to Python 3.9(not >= 3.10 as of Jun 3 2023). Also, Azure Functions local environment does not support Arm64 architecture on Mac, but supports i386 (Intel) architecture on Mac. I managed to install python 3.9 via pyenv on Rosetta2(x86_64) with configuring dependencies with separate environments from the default arm64 one.

Environment

MacOS Monterey 12.3, M1 Apple silicon

Rosetta 2

1
2


% uname -a
Darwin xxx 21.4.0 Darwin Kernel Version 21.4.0: Mon Feb 21 20:35:58 PST 2022; root:xnu-8020.101.4~2/RELEASE_ARM64_T6000 x86_64

pyenv 2.3.18
Python 3.9.16
Homebrew 4.0.20
azure-functions-core-tools@4/4.0.5198

Enable Rosetta2 on terminal

Ref: Follow steps at learn.microsoft.com | ARM64 での x86 エミュレーション
- Copy Terminal.app and name it as Terminal_rosetta.app.
- Right click the reference on terminal_rosetta.app > Get Info, and check the Open using Rosetta check box.
- Change shell to zsh by:
  1
  
  % chsh -s /bin/zsh
- Restart Terminal_rosetta.app
- Check the X86 emulator architecture
  1 2 3 4 5
  
  % arch i386 % uname -m x86_64

Install necessary packages

homebrew

At i386 architecture, the brew in installed on /usr/local/bin/brew

Add alias on .zshrc to avoid the conflict with arm64 architecture setting

1
2
3
4


# rosetta terminal setup
if [ $(arch) = "i386" ]; then
  alias brew86='/usr/local/bin/brew'
fi

After adding this to ~/.zshrc etc, reload the configuration by:

1

% source ~/.zshrc

Python

Though I tried to install it via pyenv, python installation to x86_64 architecture environment (/usr/local/bin) did not work well.

Install pyenv via x86_64 emulator
1

brew86 install pyenv

Add alias to ~/.zshrc and reload it.

1
2
3


if [ $(arch) = "i386" ]; then
  alias pyenv86="arch -x86_64 pyenv"
fi

1

% source ~/.zshrc

However, I could not install python on this environment. (as of 3 Jun 2023, I could not figure out the cause). What I did was:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


# Install dependency libraries
% brew86 install zlib

# Install python over pyenv86
% pyenv86 install 3.10
...
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 12.3 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/72/czdc_d1n5ds8rww6b_f_q2tw0000gn/T/python-build.20230603163417.77504
Results logged to /var/folders/72/czdc_d1n5ds8rww6b_f_q2tw0000gn/T/python-build.20230603163417.77504.log

Last 10 log lines:
DYLD_LIBRARY_PATH=/var/folders/72/czdc_d1n5ds8rww6b_f_q2tw0000gn/T/python-build.20230603163417.77504/Python-3.10.9 ./python.exe -E -S -m sysconfig --generate-posix-vars ;\
  if test $? -ne 0 ; then \
    echo "generate-posix-vars failed" ; \
    rm -f ./pybuilddir.txt ; \
    exit 1 ; \
  fi
dyld[87178]: symbol not found in flat namespace '_libintl_bindtextdomain'
/bin/sh: line 1: 87178 Abort trap: 6           DYLD_LIBRARY_PATH=/var/folders/72/czdc_d1n5ds8rww6b_f_q2tw0000gn/T/python-build.20230603163417.77504/Python-3.10.9 ./python.exe -E -S -m sysconfig --generate-posix-vars
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1   

Instead, I simply installed python by brew on x86_64 emulator.
1

brew86 install python3

Add alias to ~/.zshrc and reload it.

1
2
3


if [ $(arch) = "i386" ]; then
  alias python="/usr/local/bin/python3"
fi

1
2
3
4
5


% source ~/.zshrc
% python -V
Python 3.11.3
% which python
python: aliased to /usr/local/bin/python3

Add: 3 Jun 2023 - The above did not work because the latest python3.11 that was installed are not supported by azure Functions python feature. So, I retried to use pyenv. I tried sooo many stuffs, but the only one worked was from a github issues.

make for grpcio library Makefile:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


FUNC_VERSION = $(shell func --version)
FUNC_PATH = $(shell realpath $$(which func))
FUNC_OSX_WORKER_PATH = $(dir $(FUNC_PATH))workers/python/3.10/OSX
FUNC_WORKER_CONFIG_JSON = $(dir $(FUNC_PATH))workers/python/worker.config.json
FUNC_OSX_WORKER_X64 = $(FUNC_OSX_WORKER_PATH)/X64
FUNC_OSX_WORKER_ARM64 = $(FUNC_OSX_WORKER_PATH)/Arm64
.PHONY: $(FUNC_OSX_WORKER_ARM64)
$(FUNC_OSX_WORKER_ARM64):
  cp -r $(FUNC_OSX_WORKER_PATH)/X64 $(FUNC_OSX_WORKER_PATH)/Arm64
  pip install grpcio --upgrade --target $@

.PHONY: $(FUNC_WORKER_CONFIG_JSON)
$(FUNC_WORKER_CONFIG_JSON): 
  cp $(FUNC_WORKER_CONFIG_JSON) $(FUNC_WORKER_CONFIG_JSON).bak
  cat $(FUNC_WORKER_CONFIG_JSON) \
    | jq '.description.supportedArchitectures |= .+ ["Arm64"]' \
    > $(FUNC_WORKER_CONFIG_JSON).tmp
  mv $(FUNC_WORKER_CONFIG_JSON).tmp $(FUNC_WORKER_CONFIG_JSON)

.PHONY: install_func_arm64_worker
install_func_arm64_worker: $(FUNC_OSX_WORKER_ARM64) $(FUNC_WORKER_CONFIG_JSON)

This required jq on x86 environment
Ref: Github | Azure/azure-functions-python-worker

Installed pyenv for x86 on the different pyenv home. On ~/.zshrc,

1
2
3
4
5


if [ $(arch) = "i386" ]; then
  export PYENV_ROOT="$HOME/.pyenv-x86"
  eval "$(pyenv86 init --path)"
  eval "$(pyenv86 init -)"
fi

Also, add prefix for the libraries to CFLAGS environment variable. Added below to ~/.zshrc

1

export CFLAGS="-I$(brew86 --prefix openssl)/include -I$(brew86 --prefix bzip2)/include -I$(brew86 --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew86 --prefix openssl)/lib -L$(brew86 --prefix readline)/lib -L$(brew86 --prefix zlib)/lib -L$(brew86 --prefix bzip2)/lib"

1

pyenv86 install --patch 3.9.16 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

Ref: Github/pyenv | Python build fails on M1 Apple Silicon with arm64 homebrew

Then I could install python 3.9.16 (python 3.9 is the highest version supported by Functions python)

1
2
3
4
5


% pyenv86 install 3.9.16
% python -V
3.9.16
% which python
/Users/tato/.pyenv-x86/shims/python

Azure Functions Core Tools

Install by brew

1
2
3
4


% brew86 tap azure/functions
% brew86 install azure-functions-core-tools@4
# if upgrading on a machine that has 2.x or 3.x installed:
% brew86 link --overwrite azure-functions-core-tools@4

Add alias to ~/.zshrc and reload it. Note that you should check the detail release number as a directory name in azure-functions-core-tools@4.

1
2
3


if [ $(arch) = "i386" ]; then
  alias func="/usr/local/Cellar/azure-functions-core-tools@4/4.0.5198/func"
fi

1
2
3


% source ~/.zshrc
% which func
func: aliased to /usr/local/Cellar/azure-functions-core-tools@4/4.0.5198/func