@@ -1,12 +1,13 @@ | |||||
# 开源软件供应链点亮计划 - 暑期 2020 | # 开源软件供应链点亮计划 - 暑期 2020 | ||||
[开源供应链点亮计划 - 暑期 2020](https://isrc.iscas.ac.cn/summer2020)是由中科院软件所和 [openEuler](https://openeuler.org) 社区共同举办的一项面向高校学生的暑期活动,旨在鼓励在校学生积极参与开源软件的开发维护,促进国内优秀开源软件社区的蓬勃发展。 | |||||
[开源供应链点亮计划 - 暑期 2020](https://summer-ospp.ac.cn/2020/)是由中科院软件所和 [openEuler](https://openeuler.org) 社区共同举办的一项面向高校学生的暑期活动,旨在鼓励在校学生积极参与开源软件的开发维护,促进国内优秀开源软件社区的蓬勃发展。 | |||||
从即日起 [MindSpore](https://www.mindspore.cn) 社区通过[开源供应链点亮计划 - 暑期 2020](https://isrc.iscas.ac.cn/summer2020)活动发布开发任务,在活动过程中有对开发任务感兴趣的在校学生会通过社区 Issue / 邮件等方式和发布任务的导师进行沟通。 2020 年 6 月 30 日导师公布选中参与任务开发的学生,从 7 月 1 日开始到 9 月 30 日之间进行为期三个月的开发,在这个过程中导师对学生进行多种形式的辅导,帮助学生完成开发任务。 | |||||
从即日起 [MindSpore](https://www.mindspore.cn) 社区通过[开源供应链点亮计划 - 暑期 2020](https://summer-ospp.ac.cn/2020/)活动发布开发任务,在活动过程中有对开发任务感兴趣的在校学生会通过社区 Issue / 邮件等方式和发布任务的导师进行沟通。 2020 年 6 月 30 日导师公布选中参与任务开发的学生,从 7 月 1 日开始到 9 月 30 日之间进行为期三个月的开发,在这个过程中导师对学生进行多种形式的辅导,帮助学生完成开发任务。 | |||||
更多活动介绍 | 更多活动介绍 | ||||
1. https://docs.google.com/presentation/d/1XtMfT6XLLMw4yajz0M_Gi9WI71yk-rhWqx3xcT1q5kU | 1. https://docs.google.com/presentation/d/1XtMfT6XLLMw4yajz0M_Gi9WI71yk-rhWqx3xcT1q5kU | ||||
2. https://isrc.iscas.ac.cn/summer2020 | |||||
2. https://summer-ospp.ac.cn/2020/ | |||||
## MindSpore 社区任务提交方式 | ## MindSpore 社区任务提交方式 | ||||
@@ -714,47 +714,34 @@ in which, | |||||
### C++ Example | ### C++ Example | ||||
```cpp | ```cpp | ||||
/// \brief Function to create a CocoDataset. | |||||
/// \note The generated dataset has multi-columns : | |||||
/// - task='Detection', column: [['image', dtype=uint8], ['bbox', dtype=float32], ['category_id', dtype=uint32], | |||||
/// ['iscrowd', dtype=uint32]]. | |||||
/// - task='Stuff', column: [['image', dtype=uint8], ['segmentation',dtype=float32], ['iscrowd', dtype=uint32]]. | |||||
/// - task='Keypoint', column: [['image', dtype=uint8], ['keypoints', dtype=float32], | |||||
/// ['num_keypoints', dtype=uint32]]. | |||||
/// - task='Panoptic', column: [['image', dtype=uint8], ['bbox', dtype=float32], ['category_id', dtype=uint32], | |||||
/// ['iscrowd', dtype=uint32], ['area', dtype=uitn32]]. | |||||
/// \brief Function to create a MnistDataset. | |||||
/// \note The generated dataset has two columns ["image", "label"]. | |||||
/// \param[in] dataset_dir Path to the root directory that contains the dataset. | /// \param[in] dataset_dir Path to the root directory that contains the dataset. | ||||
/// \param[in] annotation_file Path to the annotation json. | |||||
/// \param[in] task Set the task type of reading coco data, now support 'Detection'/'Stuff'/'Panoptic'/'Keypoint'. | |||||
/// \param[in] decode Decode the images after reading. | |||||
/// \param[in] usage Part of dataset of MNIST, can be "train", "test" or "all" (default = "all"). | |||||
/// \param[in] sampler Shared pointer to a sampler object used to choose samples from the dataset. If sampler is not | /// \param[in] sampler Shared pointer to a sampler object used to choose samples from the dataset. If sampler is not | ||||
/// given, a `RandomSampler` will be used to randomly iterate the entire dataset (default = RandomSampler()). | /// given, a `RandomSampler` will be used to randomly iterate the entire dataset (default = RandomSampler()). | ||||
/// \param[in] cache Tensor cache to use (default=nullptr which means no cache is used). | /// \param[in] cache Tensor cache to use (default=nullptr which means no cache is used). | ||||
/// \param[in] extra_metadata Flag to add extra meta-data to row. (default=false). | |||||
/// \return Shared pointer to the CocoDataset. | |||||
/// \return Shared pointer to the MnistDataset. | |||||
/// \par Example | /// \par Example | ||||
/// \code | /// \code | ||||
/// /* Define dataset path and MindData object */ | /// /* Define dataset path and MindData object */ | ||||
/// std::string folder_path = "/path/to/coco_dataset_directory"; | |||||
/// std::string annotation_file = "/path/to/annotation_file"; | |||||
/// std::shared_ptr<Dataset> ds = Coco(folder_path, annotation_file); | |||||
/// std::string folder_path = "/path/to/mnist_dataset_directory"; | |||||
/// std::shared_ptr<Dataset> ds = Mnist(folder_path, "all", std::make_shared<RandomSampler>(false, 20)); | |||||
/// | /// | ||||
/// /* Create iterator to read dataset */ | /// /* Create iterator to read dataset */ | ||||
/// std::shared_ptr<Iterator> iter = ds->CreateIterator(); | /// std::shared_ptr<Iterator> iter = ds->CreateIterator(); | ||||
/// std::unordered_map<std::string, mindspore::MSTensor> row; | /// std::unordered_map<std::string, mindspore::MSTensor> row; | ||||
/// iter->GetNextRow(&row); | /// iter->GetNextRow(&row); | ||||
/// | /// | ||||
/// /* Note: In COCO dataset, each dictionary has keys "image" and "annotation" */ | |||||
/// /* Note: In MNIST dataset, each dictionary has keys "image" and "label" */ | |||||
/// auto image = row["image"]; | /// auto image = row["image"]; | ||||
/// \endcode | /// \endcode | ||||
inline std::shared_ptr<CocoDataset> Coco(const std::string &dataset_dir, const std::string &annotation_file, | |||||
const std::string &task = "Detection", const bool &decode = false, | |||||
const std::shared_ptr<Sampler> &sampler = std::make_shared<RandomSampler>(), | |||||
const std::shared_ptr<DatasetCache> &cache = nullptr, | |||||
const bool &extra_metadata = false) { | |||||
return std::make_shared<CocoDataset>(StringToChar(dataset_dir), StringToChar(annotation_file), StringToChar(task), | |||||
decode, sampler, cache, extra_metadata); | |||||
inline std::shared_ptr<MnistDataset> MS_API | |||||
Mnist(const std::string &dataset_dir, const std::string &usage = "all", | |||||
const std::shared_ptr<Sampler> &sampler = std::make_shared<RandomSampler>(), | |||||
const std::shared_ptr<DatasetCache> &cache = nullptr) { | |||||
return std::make_shared<MnistDataset>(StringToChar(dataset_dir), StringToChar(usage), sampler, cache); | |||||
} | } | ||||
``` | ``` | ||||
The API document page output according to the above comments is [Function mindspore::dataset::Coco](https://www.mindspore.cn/lite/api/en/master/generate/function_mindspore_dataset_Coco-1.html). | |||||
The API document page output according to the above comments is [Function mindspore::dataset::Coco](https://www.mindspore.cn/lite/api/en/master/generate/function_mindspore_dataset_Mnist-1.html). |
@@ -69,7 +69,7 @@ Examples: | |||||
- `Raises`:异常信息,包含异常类型、含义等。 | - `Raises`:异常信息,包含异常类型、含义等。 | ||||
- `Examples`:样例代码。 | - `Examples`:样例代码。 | ||||
**针对算子和Cell的注释,需要在`Examples`前添加`Inputs`、`Outputs`和`Supported Platforms`三项内容。** | |||||
针对算子和Cell的注释,需要在`Examples`前添加`Inputs`、`Outputs`和`Supported Platforms`三项内容。 | |||||
- `Inputs`和`Outputs`:用于描述实例化后,算子的输入和输出的类型和shape,输入名可以和样例相同。建议在注释中给出对应的数学公式。 | - `Inputs`和`Outputs`:用于描述实例化后,算子的输入和输出的类型和shape,输入名可以和样例相同。建议在注释中给出对应的数学公式。 | ||||
- `Supported Platforms`:用于描述算子支持的硬件平台,名称前后需添加``,存在多个时使用空格隔开。 | - `Supported Platforms`:用于描述算子支持的硬件平台,名称前后需添加``,存在多个时使用空格隔开。 | ||||
@@ -713,47 +713,34 @@ class BatchNorm(PrimitiveWithInfer): | |||||
### 完整示例 | ### 完整示例 | ||||
```cpp | ```cpp | ||||
/// \brief Function to create a CocoDataset. | |||||
/// \note The generated dataset has multi-columns : | |||||
/// - task='Detection', column: [['image', dtype=uint8], ['bbox', dtype=float32], ['category_id', dtype=uint32], | |||||
/// ['iscrowd', dtype=uint32]]. | |||||
/// - task='Stuff', column: [['image', dtype=uint8], ['segmentation',dtype=float32], ['iscrowd', dtype=uint32]]. | |||||
/// - task='Keypoint', column: [['image', dtype=uint8], ['keypoints', dtype=float32], | |||||
/// ['num_keypoints', dtype=uint32]]. | |||||
/// - task='Panoptic', column: [['image', dtype=uint8], ['bbox', dtype=float32], ['category_id', dtype=uint32], | |||||
/// ['iscrowd', dtype=uint32], ['area', dtype=uitn32]]. | |||||
/// \brief Function to create a MnistDataset. | |||||
/// \note The generated dataset has two columns ["image", "label"]. | |||||
/// \param[in] dataset_dir Path to the root directory that contains the dataset. | /// \param[in] dataset_dir Path to the root directory that contains the dataset. | ||||
/// \param[in] annotation_file Path to the annotation json. | |||||
/// \param[in] task Set the task type of reading coco data, now support 'Detection'/'Stuff'/'Panoptic'/'Keypoint'. | |||||
/// \param[in] decode Decode the images after reading. | |||||
/// \param[in] usage Part of dataset of MNIST, can be "train", "test" or "all" (default = "all"). | |||||
/// \param[in] sampler Shared pointer to a sampler object used to choose samples from the dataset. If sampler is not | /// \param[in] sampler Shared pointer to a sampler object used to choose samples from the dataset. If sampler is not | ||||
/// given, a `RandomSampler` will be used to randomly iterate the entire dataset (default = RandomSampler()). | /// given, a `RandomSampler` will be used to randomly iterate the entire dataset (default = RandomSampler()). | ||||
/// \param[in] cache Tensor cache to use (default=nullptr which means no cache is used). | /// \param[in] cache Tensor cache to use (default=nullptr which means no cache is used). | ||||
/// \param[in] extra_metadata Flag to add extra meta-data to row. (default=false). | |||||
/// \return Shared pointer to the CocoDataset. | |||||
/// \return Shared pointer to the MnistDataset. | |||||
/// \par Example | /// \par Example | ||||
/// \code | /// \code | ||||
/// /* Define dataset path and MindData object */ | |||||
/// std::string folder_path = "/path/to/coco_dataset_directory"; | |||||
/// std::string annotation_file = "/path/to/annotation_file"; | |||||
/// std::shared_ptr<Dataset> ds = Coco(folder_path, annotation_file); | |||||
/// /* Define dataset path and MindData object */ | |||||
/// std::string folder_path = "/path/to/mnist_dataset_directory"; | |||||
/// std::shared_ptr<Dataset> ds = Mnist(folder_path, "all", std::make_shared<RandomSampler>(false, 20)); | |||||
/// | /// | ||||
/// /* Create iterator to read dataset */ | |||||
/// std::shared_ptr<Iterator> iter = ds->CreateIterator(); | |||||
/// std::unordered_map<std::string, mindspore::MSTensor> row; | |||||
/// iter->GetNextRow(&row); | |||||
/// /* Create iterator to read dataset */ | |||||
/// std::shared_ptr<Iterator> iter = ds->CreateIterator(); | |||||
/// std::unordered_map<std::string, mindspore::MSTensor> row; | |||||
/// iter->GetNextRow(&row); | |||||
/// | /// | ||||
/// /* Note: In COCO dataset, each dictionary has keys "image" and "annotation" */ | |||||
/// auto image = row["image"]; | |||||
/// /* Note: In MNIST dataset, each dictionary has keys "image" and "label" */ | |||||
/// auto image = row["image"]; | |||||
/// \endcode | /// \endcode | ||||
inline std::shared_ptr<CocoDataset> Coco(const std::string &dataset_dir, const std::string &annotation_file, | |||||
const std::string &task = "Detection", const bool &decode = false, | |||||
const std::shared_ptr<Sampler> &sampler = std::make_shared<RandomSampler>(), | |||||
const std::shared_ptr<DatasetCache> &cache = nullptr, | |||||
const bool &extra_metadata = false) { | |||||
return std::make_shared<CocoDataset>(StringToChar(dataset_dir), StringToChar(annotation_file), StringToChar(task), | |||||
decode, sampler, cache, extra_metadata); | |||||
inline std::shared_ptr<MnistDataset> MS_API | |||||
Mnist(const std::string &dataset_dir, const std::string &usage = "all", | |||||
const std::shared_ptr<Sampler> &sampler = std::make_shared<RandomSampler>(), | |||||
const std::shared_ptr<DatasetCache> &cache = nullptr) { | |||||
return std::make_shared<MnistDataset>(StringToChar(dataset_dir), StringToChar(usage), sampler, cache); | |||||
} | } | ||||
``` | ``` | ||||
根据以上注释内容输出的API文档页面为[Function mindspore::dataset::Coco](https://www.mindspore.cn/lite/api/en/master/generate/function_mindspore_dataset_Coco-1.html)。 | |||||
根据以上注释内容输出的API文档页面为[Function mindspore::dataset::Coco](https://www.mindspore.cn/lite/api/en/master/generate/function_mindspore_dataset_Mnist-1.html)。 |
@@ -23,8 +23,8 @@ This is the working repo for the Data special interest group (SIG). This repo co | |||||
## Representative videos | ## Representative videos | ||||
* [mindspore data processing introduction](https://www.bilibili.com/video/BV1RZ4y1W7FL) | * [mindspore data processing introduction](https://www.bilibili.com/video/BV1RZ4y1W7FL) | ||||
* [mindspore data loading and data format conversion](https://www.mindspore.cn/docs/programming_guide/zh-CN/master/quick_start/quick_video/loading_the_dataset_and_converting_data_format.html) | |||||
* [optimize data processing](https://www.mindspore.cn/docs/programming_guide/zh-CN/master/quick_start/quick_video/optimize_data_processing.html) | |||||
* [mindspore data loading and data format conversion](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/teaching_video/video/%E5%8A%A0%E8%BD%BD%E6%95%B0%E6%8D%AE%E9%9B%86%E4%B8%8E%E8%BD%AC%E6%8D%A2%E6%A0%BC%E5%BC%8F.mp4) | |||||
* [optimize data processing](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/teaching_video/video/%E4%BC%98%E5%8C%96%E6%95%B0%E6%8D%AE%E5%A4%84%E7%90%86.mp4) | |||||
## Main issue To be solved | ## Main issue To be solved | ||||
@@ -1,5 +1,5 @@ | |||||
## Resource about Dataset in mindspore.cn | ## Resource about Dataset in mindspore.cn | ||||
* [data loading and processing](https://www.mindspore.cn/tutorials/zh-CN/master/index.html) | * [data loading and processing](https://www.mindspore.cn/tutorials/zh-CN/master/index.html) | ||||
* [dataset sample](https://www.mindspore.cn/docs/programming_guide/zh-CN/master/dataset_sample.html) | |||||
* [dataset sample](https://www.mindspore.cn/tutorials/zh-CN/master/advanced/dataset.html) | |||||
* [dataset API introduction](https://www.mindspore.cn/docs/api/zh-CN/master/api_python/mindspore.dataset.html) | * [dataset API introduction](https://www.mindspore.cn/docs/api/zh-CN/master/api_python/mindspore.dataset.html) |
@@ -37,7 +37,7 @@ Meeting link: https://welink-meeting.zoom.us/j/448774327 | |||||
## Action Items | ## Action Items | ||||
1. DX MEP is added: <https://gitee.com/lyd911/community/tree/master/design/meps/mep-dx> | |||||
1. DX MEP is added: <https://gitee.com/mindspore/community/tree/master/design/meps/mep-dx> | |||||
2. Goals of DX SIG: | 2. Goals of DX SIG: | ||||
2.1 Build user portraits, attract core developers | 2.1 Build user portraits, attract core developers | ||||