16 kB
Note
This document is developed based on PEP 8, Huawei Python Coding Style Guide, Huawei Python Secure Coding Standard, and industry consensus. To participate in the MindSpore community, please comply with this style guide (for contents conflict with the PEP 8 style guide), and then with PEP 8.
If you disagree with the rules, you are advised to submit an issue and provide reasons. The issue can take effect after being reviewed, accepted, and modified by the MindSpore community operation team.
Scope
MindSpore open source community
1. Code Style
1.1 Naming
Rule 1.1.1 Package names and module names are in lowercase and cannot contain underscores (_).
Rule 1.1.2 Class names are in the CamelCase style. The first letter must be capitalized, and the prefix is a private class underscore (_).
class _Foo:
_instance = None
pass
Rule 1.1.3 Function names and variable names are in lowercase and separated by underscores (_) when containing multiple words.
def _func_example(path):
pass
Recommendation 1.1.4 Do not use single-character names except for iterators and counters.
1.2 Format
Rule 1.2.1 Ensure that each line contains a maximum of 120 characters.
If a line contains more than 120 characters, start a new line properly.
Rule 1.2.2 Use spaces to indent, four at a time. Tab indent is forbidden.
Rule 1.2.3 The import sequence is as follows: standard library, third-party, and customization module.
Rule 1.2.4 Do not use parentheses in return statements and conditional statements.
Rule 1.2.5 There are two blank lines between a module-level function and a class, and one blank line between class member functions. Add blank lines between comments and code as needed. In principle, there should be no more than two blank lines.
Rule 1.2.6 Delete invalid or redundant code directly. Do not retain the code in the form of comments or TODO. You are advised to submit an issue record.
1.3 Comment
Rule 1.3.1 File header comments must contain copyright statements.
All Python files must contain the following copyright statements:
# Copyright 2019 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""
Add notes.
"""
import xxx
Notes:
Files created in 2020 should containCopyright 2020 Huawei Technologies Co., Ltd.
Files created in 2019 and modified in 2020 should containCopyright 2019-2020 Huawei Technologies Co., Ltd.
Rule 1.3.2 Comply with the comment formats of external classes, methods, operators, and cells:
- The comment formats of
classanddefare the same. Use Python comments which is generally accepted by the industry, and indent the comments under a declaration. Allclassanddefshould be commented. You can write only one introduction for the classes and methods in the module. - For details about the comment formats, see MindSpore Comment Specifications.
Rule 1.3.3 Do not use comments to mask Pylint alarms.
1.4 Log
Rule 1.4.1 Capitalize the first letter of the exception log text.
Rule 1.4.2 Variable names in log texts must be marked with single quotation marks.
2. General Coding
2.1 Interface Declaration
Rule 2.1.1 User interfaces are described in all of a file, and all is placed between import and code.
Rule 2.1.2 Use underscores (_) to prefixe the non-external methods used by the current file. Method names used across modules do not need underscore prefixes. User interfaces are declared in all of a file.
2.2 Data Verification
Rule 2.2.1 Check the validity of all external data, including but not limited to function input parameters, external input named lines, file formats, file sizes, environment variables, and user data.
Recommendation 2.2.2 File paths must be canonicalized before use.
A file path that comes from external data must be canonicalized first. If the file path is not canonicalized, attackers can construct a malicious file path to access the file without authorization.
For example, an attacker can construct ../../../etc/passwd to access any file.
For example, use the realpath() function in Linux and the PathCanonicalize() function in Windows for file path canonicalization.
[Noncompliant Code Example]
The following code obtains the file name from an external system, concatenates the file name into a file path, and directly reads the file content. As a result, the attacker can read the content of any file.
The following is an example of error code:
[Compliant Code Example]
Canonicalize the file path and then check whether the path is valid in the program.
The following is an example of correct code:
[Exceptions]
Command line programs that run on the console, or file paths that are manually entered on the console are exempted from this rule.
Rule 2.2.3 Do not invoke the OS command parser to run commands or programs.
If untrusted input that is not verified is used as a parameter or part of a system command, vulnerability may occur in a command injection. For the command injection vulnerability, the command is executed at the same privilege level as the Python application. It provides a function similar to shells for attackers. In Python, os.system or os.popen is often used to call a new process. If the command to be executed comes from external input, command and parameter injection may occur.
When running the command, pay attention to the following points:
- Do not concatenate the input parameters in the character string for command execution. If the input parameters must be concatenated, perform whitelist filtering on the input parameters.
- Verify the type of the input parameters. For example, integer data can be mandatorily converted into integers.
- Ensure that the formatted string is correct. For example, use %d instead of %s to concatenate parameters of the int type.
[Noncompliant Code Example 1]
The attacker can find the value of the environment variable APPHOME and place the attacking program against the constant INITCMD in the corresponding directory
to execute the attack.
home = os.getenv('APPHOME')
cmd = os.path.join(home, INITCMD)
os.system(cmd)
[Noncompliant Code Example 2]
The value of the backuptype attribute is not verified. The value is entered by the user and may be attacked. For example, the value entered by the user is && del.
c:\dbms\. "：
# The value is obtained from the user configuration.
btype = req.field('backuptype')
cmd = "cmd.exe /K \"c:\\util\\rmanDB.bat " + btype + "&&c:\\util\\cleanup.bat\""
os.system(cmd)
[Noncompliant Code Example 3]
The value of the backuptype attribute is not verified. The value is entered by the user, which may be attacked. For example, the value entered by the user is && del.
c:\dbms\. ":
import os
import sys
try:
print(os.system("ls " + sys.argv[1]))
except Exception as ex:
print('exception:', ex)
Attackers can run the following command to exploit this vulnerability:
python test.py ". && echo bad"
Actually, the following two commands are executed:
ls .
echo bad
[Compliant Code Example]
Do not use os.system. You can use standard APIs instead of running system commands to complete the tasks.
import os
import sys
try:
print(os.listdir(sys.argv[1]))
except Exception as ex:
print(ex)
2.3 Abnormal Behavior
Rule 2.3.1 Exceptions must be properly handled. Do not suppress or ignore exceptions found in the check results.
Ensure that the programs in each except block continue to operate only when they are valid. The except block must either recover from an exception or throw another exception suitable for the context of the current catch block to allow the most adjacent outer try-except statement block to recover.
[Compliant Code Example]
The correct method is to avoid using os.system. You can use standard APIs instead of running system commands to complete the tasks.
validFlag = False
while not validFlag:
try:
# If requested file does not exist, throws FileNotFoundError
# If requested file exists, sets validFlag to true
validFlag = True
except FileNotFoundError:
import traceback
traceback.print_exc()
[Exceptions]
- If the resource release failure does not affect the subsequent program behavior, the exception that occurs during resource release can be suppressed. Examples of releasing resources include closing files, network sockets, threads, and so on. These resources are usually released in the except or fianlly block and will not be used during subsequent program operation. Therefore, unless the resources are exhausted, these exceptions cannot affect the subsequent behavior of the program. When the resource exhaustion is resolved, you only need to purify the exceptions and record logs (for future improvement). In this case, there is no need to handle other errors.
- If it is impossible to recover from an exception at a specific abstraction level, the code at that level does not need to handle the exception. Instead, the code at that level should throw an appropriate exception so that higher-level code can catch the exception and attempt to recover it. In this case, the most common implementation method is to omit the catch statement block and allow the exception to be broadcast.
Rule 2.3.2 When using try…except… to protect the code, use finally… to ensure that operation objects are released after an exception occurs.
When using try…except… to protect the code, if an exception occurs during code execution, use finally… to ensure that operation objects can be released.
[Compliant Code Example]
handle = open(r"/tmp/sample_data.txt") # May raise IOError
try:
data = handle.read() # May raise UnicodeDecodeError
except UnicodeDecodeError as decode_error:
print(decode_error)
finally:
handle.close() # Always run after try:
Rule 2.3.3 Do not capture all exceptions by executing the "except:" statement.
Note that Python is tolerant of exceptions. The "except:" statement can capture any errors, including those in Python syntax. Executing the "except:" statement hides potential bugs. Therefore, specify exceptions to be handled when using try…except… to protect the code. The Exception class is the base class of most runtime exceptions and should not be used in the "except:" statement. The "try" statement should contain only exceptions that must be handled at the current location. The "except:" statement should only capture exceptions that must be handled. For example, for the code for opening files, the "try" statement should contain only the "open" statement. The "except:" statement only captures the FileNotFoundError exceptions. Other unexpected exceptions are captured by functions in the upper layer, or are transparently transmitted to external programs for exposure.
[Noncompliant Code Example]
Two types of exceptions may occur in the following code. When executing the "except:" statement for unified handle, if exceptions occur in the open statement execution, and the "except:" statement handle is invalid, the close method will be called and an error that the reported handle is undefined will be reported.
try:
handle = open(r"/tmp/sample_data.txt") # May raise IOError
data = handle.read() # May raise UnicodeDecodeError
except:
handle.close()
[Compliant Code Example]
try:
handle = open(r"/tmp/sample_data.txt") # May raise IOError
try:
data = handle.read() # May raise UnicodeDecodeError
except UnicodeDecodeError as decode_error:
print(decode_error)
finally:
handle.close()
except(FileNotFoundError, IOError) as file_open_except:
print(file_open_except)
Rule 2.3.4 The raise keyword that is not contained in the "except:" statement must have exceptions specified.
Note: The raise keyword can be used only in the "try-except" statement and re-throw exceptions captured by the "except:" statement.
[Noncompliant Code Example]
a = 1
if a==1:
raise
[Compliant Code Example 1] Raise an exception or a custom exception.
a = 1
if a==1:
raise Exception
[Compliant Code Example 2] Use the raise keyword in the "try-except" statement.
try:
f = open('myfile.txt')
s = f.readline()
i = int(s.strip())
except IOError as e:
print("I/O error({0}): {1}".format(e.errno, e.strerror))
except ValueError:
print("Could not convert data to an integer.")
except Exception:
print("Unexpected error:", sys.exc_info()[0])
raise
2.4 Serialization and Deserialization
Rule 2.4.1 When pickle has security risks, do not use the pickle.load, cPickle.load, or shelve module to load untrusted data.
Rule 2.4.2 Use secure random numbers.
Python implements the random number generation function in the random module, and implements various distributed pseudo-random number generators. The generated random numbers can be
evenly distributed, in Gaussian distribution, logarithmic normal distribution, negative exponential distribution, alpha distribution, or beta distribution manners. However, these random numbers are pseudo-random numbers, and
cannot be used for applications requiring security encryption.
Use /dev/random to generate secure random numbers, or use the secrets module introduced by Python 3.6 to generate secure random numbers.
[Noncompliant Code Example]
import random
# Pseudo-random numbers
func = random.SystemRandom()
print(func.random())
print(func.randint(0, 10))
[Compliant Code Example]
import platform
# For details about the length, see the cryptographic algorithm specifications. The length varies according to the scenario.
randLength = 16
if platform.system() == 'Linux':
with open("/dev/random", 'rb') as file:
sr = file.read(randLength)
print(sr)
Rule 2.4.3 The assert statement is usually used only in test code. Do not include the assert function in released versions.
The assert statement is used only for internal tests during R&D. If AssertionError occurs, it indicates that errors exist in software design or the code.
The software should be modified to resolve this issue. Do not include the assert function in externally released versions for production.