You can not select more than 25 topics
Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
3.2 kB
3.2 kB
MindSpore Data Special Interest Group (SIG)
This is the working repo for the Data special interest group (SIG). This repo contains all the artifacts, materials, meeting notes and proposals regarding dataset - data processing and mindrecord - data format in MindSpore. Feedbacks and contributions are welcome.
- Data Processing: You can understand it as a Dataset, which is mainly responsible for reading the user's data into a Dataset, then performing related data enhancement operations (such as: resize, onehot, rotate, shuffle, batch ...), and finally provide the Dataset to the training process.
- Data Format: It can conveniently normalize the user's training data to a unified format (MindRecord). The specific operation steps are as follows: The user can easily convert the training data into MindRecord data by defining the training data schema and calling the Python API interface. The format is then read into a Dataset through MindDataset and provided to the training process.
SIG Leads
- Liu Cunwei (Huawei)
Logistics
- SIG leads will drive the meeting.
- Meeting announcement will be posted on our gitee channel: https://gitee.com/mindspore/community/tree/master/sigs/data
- Feedbacks and topic requests are welcome by all.
Discussion
- Slack channel: https://app.slack.com/client/TUKCY4QDR/C010RPN6QNP?cdn_fallback=2
- Documents and artifacts: https://gitee.com/mindspore/community/tree/master/sigs/data
Representative videos
- mindspore data processing introduction
- mindspore data loading and data format conversion
- optimize data processing
Main issue To be solved
Here we call for developer joining us to develop a better Dataset processing system, following is mainly issue in each season.
Comment in issue please if you have any quetions and for better communication.
Also you can find all the issue in gitee by filter with label comp/data