You can not select more than 25 topics
Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
13 kB
13 kB
Results with minimal test RMSE for each kernel on dataset Asyclic
All kernels expect for Cyclic pattern kernel are tested on dataset Asyclic, which consists of 185 molecules (graphs). (Cyclic pattern kernel is tested on dataset MAO and PAH.)
The criteria used for prediction are SVM for classification and kernel Ridge regression for regression.
For predition we randomly divide the data in train and test subset, where 90% of entire dataset is for training and rest for testing. 10 splits are performed. For each split, we first train on the train data, then evaluate the performance on the test set. We choose the optimal parameters for the test set and finally provide the corresponding performance. The final results correspond to the average of the performances on the test sets.
All the results were run under Python 3.5.2, in a machine of 64 bit with one Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz, Memory of 32GB, and Ubuntu 16.04.3 LTS OS.
Summary
| Kernels | RMSE(℃) | STD(℃) | Parameter | k_time |
|---|---|---|---|---|
| Shortest path | 35.19 | 4.50 | - | 14.58" |
| Marginalized | 18.02 | 6.29 | p_quit = 0.1 | 4'19" |
| Path | 14.00 | 6.94 | - | 37.58" |
| WL subtree | 7.55 | 2.33 | height = 1 | 0.84" |
| WL shortest path | 35.16 | 4.50 | height = 2 | 40.24" |
| WL edge | 33.41 | 4.73 | height = 5 | 5.66" |
| Treelet | 8.31 | 3.38 | - | 0.50" |
| Path up to d | 7.43 | 2.69 | depth = 2 | 0.52" |
| Tree pattern | 7.27 | 2.21 | lamda = 1, h = 2 | 37.24" |
| Cyclic pattern | 0.9 | 0.11 | cycle bound = 100 | 0.31" |
- RMSE stands for arithmetic mean of the root mean squared errors on all splits.
- STD stands for standard deviation of the root mean squared errors on all splits.
- Paremeter is the one with which the kenrel achieves the best results.
- k_time is the time spent on building the kernel matrix.
- The targets of training data are normalized before calculating path kernel and treelet kernel.
Detailed results of each kernel
In each table below:
- The unit of the RMSEs and stds is ℃, The unit of the k_time is s.
- k_time is the time spent on building the kernel matrix.
shortest path kernel
RMSE_test std_test RMSE_train std_train k_time
----------- ---------- ------------ ----------- --------
35.192 4.49577 28.3604 1.35718 14.5768
Marginalized kernel
The table below shows the results of the marginalized under different termimation probability.
p_quit RMSE_test std_test RMSE_train std_train k_time
-------- ----------- ---------- ------------ ----------- --------
0.1 18.0243 6.29247 12.1863 7.03899 258.77
0.2 18.3376 5.85454 13.9554 7.54407 256.327
0.3 18.496 5.73492 13.9391 7.95812 255.614
0.4 19.4491 5.3713 16.2593 6.69358 254.897
0.5 19.7857 5.55054 17.0181 6.84437 256.757
0.6 20.1922 5.59122 17.6618 6.56718 256.557
0.7 21.6614 6.02685 20.5882 5.74601 254.953
0.8 22.996 6.08347 23.5943 3.80637 252.804
0.9 24.4241 4.95119 25.8082 3.31207 256.738
Path kernel
The targets of training data are normalized before calculating the kernel.
RMSE_test std_test RMSE_train std_train k_time
----------- ---------- ------------ ----------- --------
14.0015 6.93602 3.76191 0.702594 37.5759
Weisfeiler-Lehman subtree kernel
The table below shows the results of the WL subtree under different subtree heights.
height RMSE_test std_test RMSE_train std_train k_time
-------- ----------- ---------- ------------ ----------- --------
0 15.6859 4.1392 17.6816 0.713183 0.360443
1 7.55046 2.33179 6.27001 0.654734 0.837389
2 9.72847 2.05767 4.45068 0.882129 1.25317
3 11.2961 2.79994 2.27059 0.481516 1.79971
4 12.8083 3.44694 1.07403 0.637823 2.35346
5 14.0179 3.67504 0.700602 0.57264 2.78285
6 14.9184 3.80535 0.691515 0.56462 3.20764
7 15.6295 3.86539 0.691516 0.56462 3.71648
8 16.2144 3.92876 0.691515 0.56462 3.99213
9 16.7257 3.9931 0.691515 0.56462 4.26315
10 17.1864 4.05672 0.691516 0.564621 5.00918
Weisfeiler-Lehman shortest path kernel
The table below shows the results of the WL subtree under different subtree heights.
height rmse_test std_test rmse_train std_train k_time
-------- ----------- ---------- ------------ ----------- --------
0 35.192 4.49577 28.3604 1.35718 13.5041
1 35.1808 4.50045 27.9335 1.44836 26.8292
2 35.1632 4.50205 28.1113 1.50891 40.2356
3 35.1946 4.49801 28.3903 1.36571 54.6704
4 35.1753 4.50111 27.9746 1.46222 67.1522
5 35.1997 4.5071 28.0184 1.45564 80.0881
6 35.1645 4.49849 28.3731 1.60057 92.1925
7 35.1771 4.5009 27.9604 1.45742 105.812
8 35.1968 4.50526 28.1991 1.5149 119.022
9 35.1956 4.50197 28.2665 1.30769 131.228
10 35.1676 4.49723 28.4163 1.61596 144.964
Weisfeiler-Lehman edge kernel
The table below shows the results of the WL subtree under different subtree heights.
height rmse_test std_test rmse_train std_train k_time
-------- ----------- ---------- ------------ ----------- ---------
0 33.4077 4.73272 29.9975 0.90234 0.853002
1 33.4235 4.72131 30.1603 1.09423 1.71751
2 33.433 4.72441 29.9286 0.787941 2.66032
3 33.4073 4.73243 30.0114 0.909674 3.47763
4 33.4256 4.72166 30.1842 1.1089 4.54367
5 33.4067 4.72641 30.0411 1.01845 5.66178
6 33.419 4.73075 29.9056 0.782179 6.14803
7 33.4248 4.72155 30.1759 1.10382 7.60354
8 33.4122 4.71554 30.1365 1.07485 7.97222
9 33.4071 4.73193 30.0329 0.921065 9.07084
10 33.4165 4.73169 29.9242 0.790843 10.0254
Treelet kernel
The targets of training data are normalized before calculating the kernel.
RMSE_test std_test RMSE_train std_train k_time
----------- ---------- ------------ ----------- --------
8.3079 3.37838 2.90887 1.2679 0.500302
Path kernel up to depth d
The table below shows the results of the path kernel up to different depth d.
The first table is the results using Tanimoto kernel, where The targets of training data are normalized before calculating the kernel..
depth rmse_test std_test rmse_train std_train k_time
------- ----------- ---------- ------------ ----------- ---------
0 41.6202 6.453 43.6169 2.13212 0.0904737
1 38.8446 6.44648 40.8329 3.44147 0.175414
2 35.2915 4.7813 35.7461 1.61134 0.344896
3 29.4845 3.90351 28.4646 3.00137 0.553939
4 22.6693 6.28053 19.2517 3.42893 0.770649
5 21.7956 5.5225 16.886 2.60519 1.01558
6 20.6049 5.49983 13.1097 2.58431 1.33302
7 20.3479 5.17631 12.0152 2.5928 1.60266
8 19.8228 5.13769 10.7981 2.13082 1.81218
9 19.8734 5.10369 10.7997 2.09549 2.21726
10 19.8708 5.09217 10.7787 2.10002 2.41006
The second table is the results using MinMax kernel.
depth rmse_test std_test rmse_train std_train k_time
------- ----------- ---------- ------------ ----------- --------
0 12.58 2.73235 12.1209 0.500467 0.377576
1 12.6215 2.18866 10.2243 0.734261 0.456332
2 7.42903 2.69395 2.71885 0.732922 0.585278
3 9.02468 2.50808 1.54 1.13813 0.706556
4 10.0811 3.6477 1.36029 1.42399 0.847957
5 11.3005 4.44163 1.08518 1.06206 1.00086
6 12.186 4.88816 1.06443 1.00191 1.19792
7 12.7534 5.14529 1.19912 1.34031 1.4372
8 13.0471 5.27184 1.35822 1.84315 1.68449
9 13.1789 5.27707 1.36002 1.84834 1.96545
10 13.2538 5.26425 1.36208 1.85426 2.24943
Tree pattern kernel
Until N kernel when h = 2:
lmda rmse_test std_test rmse_train std_train k_time
----------- ----------- ---------- ------------ ----------- --------
1e-10 7.46524 1.71862 5.99486 0.356634 38.1447
1e-09 7.37326 1.77195 5.96155 0.374395 37.4921
1e-08 7.35105 1.78349 5.96481 0.378047 37.9971
1e-07 7.35213 1.77903 5.96728 0.382251 38.3182
1e-06 7.3524 1.77992 5.9696 0.3863 39.6428
1e-05 7.34958 1.78141 5.97114 0.39017 37.3711
0.0001 7.3513 1.78136 5.94251 0.331843 37.3967
0.001 7.35822 1.78119 5.9326 0.32534 36.7357
0.01 7.37552 1.79037 5.94089 0.34763 36.8864
0.1 7.32951 1.91346 6.42634 1.29405 36.8382
1 7.27134 2.20774 6.62425 1.2242 37.2425
10 7.49787 2.36815 6.81697 1.50182 37.8286
100 7.42887 2.64789 6.68766 1.34809 36.3701
1000 7.24914 2.65554 6.81906 1.41008 36.1695
10000 7.08183 2.6248 6.93431 1.38441 37.5723
100000 8.021 3.43694 8.69813 0.909839 37.8158
1e+06 8.49625 3.6332 9.59333 0.96626 38.4688
1e+07 10.9067 3.17593 11.5642 2.07792 36.9926
1e+08 61.1524 10.4355 65.3527 13.9538 37.1321
1e+09 99.943 13.6994 98.8848 5.27014 36.7443
1e+10 100.083 13.8503 97.9168 3.22768 37.096
Cyclic pattern kernel
This kernel is not tested on dataset Acyclic
Results on dataset MAO:
cycle_bound accur_test std_test accur_train std_train k_time
------------- ------------ ---------- ------------- ----------- --------
0 0.642857 0.146385 0.54918 0.0167983 0.187052
50 0.871429 0.1 0.698361 0.116889 0.300629
100 0.9 0.111575 0.732787 0.0826366 0.309837
150 0.9 0.111575 0.732787 0.0826366 0.31808
200 0.9 0.111575 0.732787 0.0826366 0.317575
Results on dataset PAH:
cycle_bound accur_test std_test accur_train std_train k_time
------------- ------------ ---------- ------------- ----------- --------
0 0.61 0.113578 0.629762 0.0135212 0.521801
10 0.61 0.113578 0.629762 0.0135212 0.52589
20 0.61 0.113578 0.629762 0.0135212 0.548528
30 0.64 0.111355 0.633333 0.0157935 0.535311
40 0.64 0.111355 0.633333 0.0157935 0.61764
50 0.67 0.09 0.658333 0.0345238 0.733868
60 0.68 0.107703 0.671429 0.0365769 0.871147
70 0.67 0.100499 0.666667 0.0380208 1.12625
80 0.78 0.107703 0.709524 0.0588534 1.19828
90 0.78 0.107703 0.709524 0.0588534 1.21182