hummingbird
/
mindarmour
Not watched
2
0
Fork 0
Code
Releases 17 Wiki Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain
Browse Source

add serving for real-world robustness test

tags/v1.8.0
ZhidanLiu 3 years ago
parent
2cbc26082e
commit
57903fd8f4
5 changed files with 435 additions and 0 deletions
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +192
    -0
      examples/natural_robustness/serving/README.md
  2. +41
    -0
      examples/natural_robustness/serving/client/perturb_config.py
  3. +61
    -0
      examples/natural_robustness/serving/client/serving_client.py
  4. +106
    -0
      examples/natural_robustness/serving/server/perturbation/servable_config.py
  5. +35
    -0
      examples/natural_robustness/serving/server/serving_server.py

+ 192
- 0
examples/natural_robustness/serving/README.md View File

@@ -0,0 +1,192 @@
# 自然扰动样本生成serving服务
提供自然扰动样本生成在线服务。客户端传入图片和扰动参数，服务端返回扰动后的图片数据。
## 环境准备
硬件环境：Ascend 910，GPU
操作系统：Linux-x86_64
软件环境：
1. python 3.7.5或python 3.9.0
2. 安装MindSpore 1.6.0可以参考[MindSpore安装页面](https://www.mindspore.cn/install)
3. 安装MindSpore Serving 1.6.0可以参考[MindSpore Serving 安装页面](https://www.mindspore.cn/serving/docs/zh-CN/r1.5/serving_install.html)
4. 安装serving分支的MindArmour:
- 从Gitee下载源码
`git clone https://gitee.com/mindspore/mindarmour.git`
- 编译并安装MindArmour
`python setup.py install`
### 文件结构说明
```bash
serving
├── server
│ ├── serving_server.py # 启动serving服务脚本
│ └── perturbation
│ └── serverable_config.py # 服务端接收客户端数据后的处理脚本
└── client
├── serving_client.py # 启动客户端脚本
└── perturb_config.py # 扰动方法配置文件
```
## 脚本说明及使用
### 部署Serving推理服务
1. #### `servable_config.py`说明。
```python
···
# 客户端可以请求的方法，包含4个返回值："results", "file_names", "file_length", "names_dict"
@register.register_method(output_names=["results", "file_names", "file_length", "names_dict"])
def natural_perturbation(img, perturb_config, methods_number, outputs_number):
"""method natural_perturbation data flow definition, only preprocessing and call model"""
res = register.add_stage(perturb, img, perturb_config, methods_number, outputs_number, outputs_count=4)
return res
```
方法`natural_perturbation`为对外提供服务的接口。
**输入：**
- img：输入为图片，格式为bytes。
- perturb_config：扰动配置项，具体配置参考`perturb_config.py`。
- methods_number：每次扰动随机从配置项中选择方法的个数。
- outputs_number：对于每张图片，生成的扰动图片数量。
**输出**res中包含4个参数：
- results：拼接后的图像bytes；
- file_names：图像名，格式为`xxx.png`，其中‘xxx’为A-Za-z中随机选择20个字符构成的字符串。
- file_length：每张图片的bytes长度。
- names_dict: 图片名和图片使用扰动方法构成的字典。格式为：
```bash
{
picture1.png: [[method1, parameters of method1], [method2, parameters of method2], ...]],
picture2.png: [[method3, parameters of method3], [method4, parameters of method4], ...]],
...
}
```
2. #### 启动server。
```python
···
def start():
servable_dir = os.path.dirname(os.path.realpath(sys.argv[0]))
# 服务配置
servable_config = server.ServableStartConfig(servable_directory=servable_dir, servable_name="perturbation", device_ids=(0, 1), num_parallel_workers=4)
# 启动服务
server.start_servables(servable_configs=servable_config)
# 启动启动gRPC服务，用于客户端和服务端之间通信
server.start_grpc_server(address="0.0.0.0:5500", max_msg_mb_size=200) # ip和最大的传输数据量，单位MB
# 启动启动Restful服务，用于客户端和服务端之间通信
server.start_restful_server(address="0.0.0.0:5500")
```
gRPC传输性能更好，Restful更适合用于web服务，根据需要选择。
执行命令`python serverong_server.py`启动服务。
当服务端打印日志`Serving RESTful server start success, listening on 0.0.0.0:5500`时，表示Serving RESTful服务启动成功，推理模型已成功加载。
### 客户端进行推理
1. 在`perturb_config.py`中设置扰动方法及参数。下面是个例子：
```python
PerturbConfig = [{"method": "Contrast", "params": {"alpha": 1.5, "beta": 0}},
{"method": "GaussianBlur", "params": {"ksize": 5}},
{"method": "SaltAndPepperNoise", "params": {"factor": 0.05}},
{"method": "Translate", "params": {"x_bias": 0.1, "y_bias": -0.2}},
{"method": "Scale", "params": {"factor_x": 0.7, "factor_y": 0.7}},
{"method": "Shear", "params": {"factor": 2, "director": "horizontal"}},
{"method": "Rotate", "params": {"angle": 40}},
{"method": "MotionBlur", "params": {"degree": 5, "angle": 45}},
{"method": "GradientBlur", "params": {"point": [50, 100], "kernel_num": 3, "center": True}},
{"method": "GradientLuminance",
"params": {"color_start": [255, 255, 255],
"color_end": [0, 0, 0],
"start_point": [100, 150], "scope": 0.3,
"bright_rate": 0.3, "pattern": "light",
"mode": "circle"}},
{"method": "Curve", "params": {"curves": 5, "depth": 10,
"mode": "vertical"}},
{"method": "Perspective",
"params": {"ori_pos": [[0, 0], [0, 800], [800, 0], [800, 800]],
"dst_pos": [[50, 0], [0, 800], [780, 0], [800, 800]]}},
]
```
其中`method`为扰动方法名，`params`为对应方法的参数。可用的扰动方法及对应参数可在`mindarmour/natural_robustness/natural_noise.py`中查询。
2. 在`serving_client.py`中写客户端的处理脚本，包含输入输出的处理、服务端的调用，可以参考下面的例子。
```python
···
def perturb(perturb_config):
"""invoke servable perturbation method natural_perturbation"""
# 请求的服务端ip及端口、请求的服务名、请求的方法名
client = Client("10.175.122.87:5500", "perturbation", "natural_perturbation")
# 输入数据
instances = []
img_path = '/root/mindarmour/example/adversarial/test_data/1.png'
result_path = '/root/mindarmour/example/adv/result/'
methods_number = 2
outputs_number = 3
img = cv2.imread(img_path)
img = cv2.imencode('.png', img)[1].tobytes() # 图片传输用bytes格式，不支持numpy.ndarray格式
perturb_config = json.dumps(perturb_config) # 配置方法转成json格式
instances.append({"img": img, 'perturb_config': perturb_config, "methods_number": methods_number,
"outputs_number": outputs_number}) # instances中可添加多个输入
# 请求服务，返回结果
result = client.infer(instances)
# 对服务请求得到的结果进行处理，将返回的图片字节流存成图片
file_names = result[0]['file_names'].split(';')
length = result[0]['file_length'].tolist()
before = 0
for name, leng in zip(file_names, length):
res_img = result[0]['results']
res_img = res_img[before:before + leng]
before = before + leng
print('name: ', name)
image = Image.open(BytesIO(res_img))
image.save(os.path.join(result_path, name))
names_dict = result[0]['names_dict']
with open('names_dict.json', 'w') as file:
file.write(names_dict)
```
启动client前，需将服务端的IP地址改成部署server的IP地址，图片路径、结果存储路基替换成用户数据路径。
目前serving数据传输支持的数据类型包括：python的int、float、bool、str、bytes，numpy number, numpy array object。
输入命令`python serving_client.py`开启客户端，如果对应目录下生成扰动样本图片则说明serving服务正确执行。
### 其他
在`serving_logs`目录下可以查看运行日志，辅助debug。

+ 41
- 0
examples/natural_robustness/serving/client/perturb_config.py View File

@@ -0,0 +1,41 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
Configuration of natural robustness methods for server.
"""
perturb_configs = [{"method": "Contrast", "params": {"alpha": 1.5, "beta": 0}},
{"method": "GaussianBlur", "params": {"ksize": 5}},
{"method": "SaltAndPepperNoise", "params": {"factor": 0.05}},
{"method": "Translate", "params": {"x_bias": 0.1, "y_bias": -0.2}},
{"method": "Scale", "params": {"factor_x": 0.7, "factor_y": 0.7}},
{"method": "Shear", "params": {"factor": 2, "direction": "horizontal"}},
{"method": "Rotate", "params": {"angle": 40}},
{"method": "MotionBlur", "params": {"degree": 5, "angle": 45}},
{"method": "GradientBlur", "params": {"point": [50, 100], "kernel_num": 3, "center": True}},
{"method": "GradientLuminance", "params": {"color_start": [255, 255, 255], "color_end": [0, 0, 0],
"start_point": [100, 150], "scope": 0.3,
"bright_rate": 0.3, "pattern": "light",
"mode": "circle"}},
{"method": "GradientLuminance", "params": {"color_start": [255, 255, 255],
"color_end": [0, 0, 0], "start_point": [150, 200],
"scope": 0.3, "pattern": "light", "mode": "horizontal"}},
{"method": "GradientLuminance", "params": {"color_start": [255, 255, 255], "color_end": [0, 0, 0],
"start_point": [150, 200], "scope": 0.3,
"pattern": "light", "mode": "vertical"}},
{"method": "Curve", "params": {"curves": 10, "depth": 10, "mode": "vertical"}},
{"method": "Perspective", "params": {"ori_pos": [[0, 0], [0, 800], [800, 0], [800, 800]],
"dst_pos": [[50, 0], [0, 800], [780, 0], [800, 800]]}},
]

+ 61
- 0
examples/natural_robustness/serving/client/serving_client.py View File

@@ -0,0 +1,61 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""The client of example add."""
import os
import json
from io import BytesIO

import cv2
from PIL import Image
from mindspore_serving.client import Client

from perturb_config import perturb_configs


def perturb(perturb_config):
"""Invoke servable perturbation method natural_perturbation"""
client = Client("0.0.0.0:5500", "perturbation", "natural_perturbation")
instances = []
img_path = 'test_data/1.png'
result_path = 'result/'
if not os.path.exists(result_path):
os.mkdir(result_path)
methods_number = 2
outputs_number = 10
img = cv2.imread(img_path)
img = cv2.imencode('.png', img)[1].tobytes()
perturb_config = json.dumps(perturb_config)
instances.append({"img": img, 'perturb_config': perturb_config, "methods_number": methods_number,
"outputs_number": outputs_number})

result = client.infer(instances)

file_names = result[0]['file_names'].split(';')
length = result[0]['file_length'].tolist()
before = 0
for name, leng in zip(file_names, length):
res_img = result[0]['results']
res_img = res_img[before:before + leng]
before = before + leng
print('name: ', name)
image = Image.open(BytesIO(res_img))
image.save(os.path.join(result_path, name))
names_dict = result[0]['names_dict']
with open('names_dict.json', 'w') as file:
file.write(names_dict)


if __name__ == '__main__':
perturb(perturb_configs)

+ 106
- 0
examples/natural_robustness/serving/server/perturbation/servable_config.py View File

@@ -0,0 +1,106 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""perturbation servable config"""
import json
import copy
import random
from io import BytesIO
import cv2
import numpy as np
from PIL import Image
from mindspore_serving.server import register
from mindarmour.natural_robustness.image import Contrast, GaussianBlur, SaltAndPepperNoise, Scale, Shear, \
Translate, Rotate, MotionBlur, GradientBlur, GradientLuminance, NaturalNoise, Curve, Perspective


CHARACTERS = [chr(i) for i in range(65, 91)]+[chr(j) for j in range(97, 123)]

methods_dict = {'Contrast': Contrast,
'GaussianBlur': GaussianBlur,
'SaltAndPepperNoise': SaltAndPepperNoise,
'Translate': Translate,
'Scale': Scale,
'Shear': Shear,
'Rotate': Rotate,
'MotionBlur': MotionBlur,
'GradientBlur': GradientBlur,
'GradientLuminance': GradientLuminance,
'NaturalNoise': NaturalNoise,
'Curve': Curve,
'Perspective': Perspective}


def check_inputs(img, perturb_config, methods_number, outputs_number):
"""Check inputs."""
if not np.any(img):
raise ValueError("img cannot be empty.")
img = Image.open(BytesIO(img))
img = cv2.cvtColor(np.asarray(img), cv2.COLOR_RGB2BGR)

config = json.loads(perturb_config)
if not config:
raise ValueError("perturb_config cannot be empty.")
for item in config:
if item['method'] not in methods_dict.keys():
raise ValueError("{} is not a valid method.".format(item['method']))

methods_number = int(methods_number)
if methods_number < 1:
raise ValueError("methods_number must more than 0.")
outputs_number = int(outputs_number)
if outputs_number < 1:
raise ValueError("outputs_number must more than 0.")

return img, config, methods_number, outputs_number


def perturb(img, perturb_config, methods_number, outputs_number):
"""Perturb given image."""
img, config, methods_number, outputs_number = check_inputs(img, perturb_config, methods_number, outputs_number)
res_img_bytes = b''
file_names = []
file_length = []
names_dict = {}
for _ in range(outputs_number):
dst = copy.deepcopy(img)
used_methods = []
for _ in range(methods_number):
item = np.random.choice(config)
method_name = item['method']
method = methods_dict[method_name]
params = item['params']
dst = method(**params)(img)
method_params = params

used_methods.append([method_name, method_params])
name = ''.join(random.sample(CHARACTERS, 20))
name += '.png'
file_names.append(name)
names_dict[name] = used_methods

res_img = cv2.imencode('.png', dst)[1].tobytes()
res_img_bytes += res_img
file_length.append(len(res_img))

names_dict = json.dumps(names_dict)

return res_img_bytes, ';'.join(file_names), file_length, names_dict


@register.register_method(output_names=["results", "file_names", "file_length", "names_dict"])
def natural_perturbation(img, perturb_config, methods_number, outputs_number):
"""method natural_perturbation data flow definition, only preprocessing and call model"""
res = register.add_stage(perturb, img, perturb_config, methods_number, outputs_number, outputs_count=4)
return res

+ 35
- 0
examples/natural_robustness/serving/server/serving_server.py View File

@@ -0,0 +1,35 @@
# Copyright 2021 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""The server of example perturbation"""

import os
import sys
from mindspore_serving import server


def start():
"""Start server."""
servable_dir = os.path.dirname(os.path.realpath(sys.argv[0]))

servable_config = server.ServableStartConfig(servable_directory=servable_dir, servable_name="perturbation",
device_ids=(0, 1), num_parallel_workers=4)
server.start_servables(servable_configs=servable_config)

server.start_grpc_server(address="0.0.0.0:5500", max_msg_mb_size=200)
# server.start_restful_server(address="0.0.0.0:5500")


if __name__ == "__main__":
start()

Write Preview
Loading…
Cancel
Save