|
@@ -1,37 +1,43 @@ |
|
|
# Extern-C-Opr with MACE |
|
|
# Extern-C-Opr with MACE |
|
|
|
|
|
|
|
|
### Build MegEngine `load_and_run` for arm64-v8a |
|
|
### Build MegEngine `load_and_run` for arm64-v8a |
|
|
|
|
|
NOTICE: build depends on [NDK](https://developer.android.com/ndk/downloads) |
|
|
|
|
|
after download, please config env by: |
|
|
|
|
|
```bash |
|
|
|
|
|
export NDK_ROOT=path/to/ndk |
|
|
|
|
|
export ANDROID_NDK_HOME=${NDK_ROOT} |
|
|
|
|
|
export PATH=${NDK_ROOT}/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH |
|
|
|
|
|
``` |
|
|
|
|
|
|
|
|
```bash |
|
|
```bash |
|
|
cd $MEGENGINE_HOME |
|
|
cd $MEGENGINE_HOME |
|
|
|
|
|
git checkout v1.0.0 (we only test v1.0.0 version) |
|
|
./scripts/cmake-build/cross_build_android_arm_inference.sh -a arm64-v8a -r |
|
|
./scripts/cmake-build/cross_build_android_arm_inference.sh -a arm64-v8a -r |
|
|
``` |
|
|
``` |
|
|
|
|
|
|
|
|
After successfully built, load_and_run should be in `$MEGENGINE_HOME/build_dir/android/arm64-v8a/Release/install/bin` |
|
|
|
|
|
|
|
|
After successfully built: |
|
|
|
|
|
* load_and_run should be in `$MEGENGINE_HOME/build_dir/android/arm64-v8a/Release/install/bin` |
|
|
|
|
|
* libmegengine.so should be in `$MEGENGINE_HOME/build_dir/android/arm64-v8a/Release/install/lib` |
|
|
|
|
|
|
|
|
### Build MACE libraries for arm64-v8a with GPU runtime |
|
|
### Build MACE libraries for arm64-v8a with GPU runtime |
|
|
|
|
|
|
|
|
```bash |
|
|
```bash |
|
|
cd $MACE_HOME |
|
|
cd $MACE_HOME |
|
|
RUNTIME=GPU bash tools/cmake/cmake-build-arm64-v8a.sh |
|
|
RUNTIME=GPU bash tools/cmake/cmake-build-arm64-v8a.sh |
|
|
|
|
|
|
|
|
cp -r $MACE_HOME/build/cmake-build/arm64-v8a/install $MEGENGINE_HOME/sdk/c-opr-loaders/mace/arm64-v8a |
|
|
|
|
|
|
|
|
export SDKPATH=${MACE_HOME}/build/cmake-build/arm64-v8a/install |
|
|
``` |
|
|
``` |
|
|
|
|
|
After successfully libmace.so should be in `$MACE_HOME/build/cmake-build/arm64-v8a/install/lib/libmace.so` |
|
|
|
|
|
|
|
|
### Build MACE loader for MegEngine |
|
|
### Build MACE loader for MegEngine |
|
|
|
|
|
|
|
|
``` |
|
|
|
|
|
SDK_PATH=/path/to/mace-sdk make |
|
|
|
|
|
``` |
|
|
|
|
|
|
|
|
|
|
|
If `SDK_PATH` is not set, by default it's `./arm64-v8a` |
|
|
|
|
|
|
|
|
If `SDKPATH` is not set, by default it's `./arm64-v8a` |
|
|
|
|
|
|
|
|
You can run with debug mode(by adding `DEBUG=1` to make command), which will show more running information |
|
|
You can run with debug mode(by adding `DEBUG=1` to make command), which will show more running information |
|
|
|
|
|
|
|
|
### Prepare a MACE model(for example: resnet_50), wrap it with MegEngine extern c opr |
|
|
### Prepare a MACE model(for example: resnet_50), wrap it with MegEngine extern c opr |
|
|
|
|
|
|
|
|
``` |
|
|
``` |
|
|
python3 dump_model.py path/to/resnet_50.pb path/to/resnet_50.data path/to/resnet_50.mdl path/to/resnet_50.yml |
|
|
|
|
|
|
|
|
python3 dump_model.py --input path/to/resnet_50.pb --param path/to/resnet_50.data --output resnet_50.mdl --config path/to/resnet_50.yml |
|
|
``` |
|
|
``` |
|
|
|
|
|
|
|
|
`*.pb` file denotes the model structure, `*.data` denotes the model parameters |
|
|
`*.pb` file denotes the model structure, `*.data` denotes the model parameters |
|
@@ -40,14 +46,22 @@ Check [here](https://github.com/XiaoMi/mace-models) to learn how to write yml fi |
|
|
|
|
|
|
|
|
### Run with load-and-run |
|
|
### Run with load-and-run |
|
|
|
|
|
|
|
|
First of all, send all files to the executed device: |
|
|
|
|
|
|
|
|
First of all, send all files to the executed device(for example: /data/local/tmp/test/): |
|
|
|
|
|
|
|
|
- load_and_run |
|
|
- load_and_run |
|
|
- resnet_50.mdl |
|
|
- resnet_50.mdl |
|
|
- libmace_loader.so |
|
|
- libmace_loader.so |
|
|
|
|
|
- libmegengine.so |
|
|
|
|
|
- libmace.so |
|
|
|
|
|
|
|
|
|
|
|
As mace build with `c++_shared` by default, but old AOSP device do not have `libc++_shared.so` by default, if you use this class devices |
|
|
|
|
|
also need send it to devices, which always can be found at `${NDK_ROOT}/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++_shared.so` |
|
|
|
|
|
|
|
|
``` |
|
|
``` |
|
|
MGB_MACE_RUNTIME=GPU MGB_MACE_OPENCL_CACHE_PATH=/path/to/opencl MGB_MACE_LOADER_FORMAT=NCHW /path/to/load_and_run /path/to/resnet_50.mdl --c-opr-lib /path/to/libmace_loader.so |
|
|
|
|
|
|
|
|
login to device |
|
|
|
|
|
cd /path/to/ (for example: /data/local/tmp/test/) |
|
|
|
|
|
|
|
|
|
|
|
MGB_MACE_RUNTIME=GPU MGB_MACE_OPENCL_CACHE_PATH=./ MGB_MACE_LOADER_FORMAT=NCHW LD_LIBRARY_PATH=. ./load_and_run resnet_50.mdl --c-opr-lib libmace_loader.so --input input-bs1.npy |
|
|
``` |
|
|
``` |
|
|
|
|
|
|
|
|
RUNTIME candidates: |
|
|
RUNTIME candidates: |
|
@@ -77,4 +91,4 @@ To generate the tunig param file, give `MACE_TUNING=1` env and set the `MACE_RUN |
|
|
|
|
|
|
|
|
# then run test using the param |
|
|
# then run test using the param |
|
|
MGB_MACE_TUNING_PARAM_PATH=opencl/vgg16.tune_param MGB_MACE_RUNTIME=GPU MGB_MACE_OPENCL_PATH=opencl MGB_MACE_LOADER_FORMAT=NCHW ./load_and_run mace/vgg16.mdl --c-opr-lib libmace_loader.so --input 4d.npy |
|
|
MGB_MACE_TUNING_PARAM_PATH=opencl/vgg16.tune_param MGB_MACE_RUNTIME=GPU MGB_MACE_OPENCL_PATH=opencl MGB_MACE_LOADER_FORMAT=NCHW ./load_and_run mace/vgg16.mdl --c-opr-lib libmace_loader.so --input 4d.npy |
|
|
``` |
|
|
|
|
|
|
|
|
``` |