# Results with minimal test RMSE for each kernel on dataset Asyclic All kernels expect for Cyclic pattern kernel are tested on dataset Asyclic, which consists of 185 molecules (graphs). (Cyclic pattern kernel is tested on dataset MAO and PAH.) The criteria used for prediction are SVM for classification and kernel Ridge regression for regression. For predition we randomly divide the data in train and test subset, where 90% of entire dataset is for training and rest for testing. 10 splits are performed. For each split, we first train on the train data, then evaluate the performance on the test set. We choose the optimal parameters for the test set and finally provide the corresponding performance. The final results correspond to the average of the performances on the test sets. All the results were run under Python 3.5.2, in a machine of 64 bit with one Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz, Memory of 32GB, and Ubuntu 16.04.3 LTS OS. ## Summary | Kernels | RMSE(℃) | STD(℃) | Parameter | k_time | |------------------|:-------:|:------:|------------------:|-------:| | Shortest path | 35.19 | 4.50 | - | 14.58" | | Marginalized | 18.02 | 6.29 | p_quit = 0.1 | 4'19" | | Path | 14.00 | 6.94 | - | 37.58" | | WL subtree | 7.55 | 2.33 | height = 1 | 0.84" | | WL shortest path | 35.16 | 4.50 | height = 2 | 40.24" | | WL edge | 33.41 | 4.73 | height = 5 | 5.66" | | Treelet | 8.31 | 3.38 | - | 0.50" | | Path up to d | 7.43 | 2.69 | depth = 2 | 0.52" | | Tree pattern | 7.27 | 2.21 | lamda = 1, h = 2 | 37.24" | | Cyclic pattern | 0.9 | 0.11 | cycle bound = 100 | 0.31" | * RMSE stands for arithmetic mean of the root mean squared errors on all splits. * STD stands for standard deviation of the root mean squared errors on all splits. * Paremeter is the one with which the kenrel achieves the best results. * k_time is the time spent on building the kernel matrix. * The targets of training data are normalized before calculating *path kernel* and *treelet kernel*. ## Detailed results of each kernel In each table below: * The unit of the *RMSEs* and *stds* is *℃*, The unit of the *k_time* is *s*. * k_time is the time spent on building the kernel matrix. ### shortest path kernel ``` RMSE_test std_test RMSE_train std_train k_time ----------- ---------- ------------ ----------- -------- 35.192 4.49577 28.3604 1.35718 14.5768 ``` ### Marginalized kernel The table below shows the results of the marginalized under different termimation probability. ``` p_quit RMSE_test std_test RMSE_train std_train k_time -------- ----------- ---------- ------------ ----------- -------- 0.1 18.0243 6.29247 12.1863 7.03899 258.77 0.2 18.3376 5.85454 13.9554 7.54407 256.327 0.3 18.496 5.73492 13.9391 7.95812 255.614 0.4 19.4491 5.3713 16.2593 6.69358 254.897 0.5 19.7857 5.55054 17.0181 6.84437 256.757 0.6 20.1922 5.59122 17.6618 6.56718 256.557 0.7 21.6614 6.02685 20.5882 5.74601 254.953 0.8 22.996 6.08347 23.5943 3.80637 252.804 0.9 24.4241 4.95119 25.8082 3.31207 256.738 ``` ### Path kernel **The targets of training data are normalized before calculating the kernel.** ``` RMSE_test std_test RMSE_train std_train k_time ----------- ---------- ------------ ----------- -------- 14.0015 6.93602 3.76191 0.702594 37.5759 ``` ### Weisfeiler-Lehman subtree kernel The table below shows the results of the WL subtree under different subtree heights. ``` height RMSE_test std_test RMSE_train std_train k_time -------- ----------- ---------- ------------ ----------- -------- 0 15.6859 4.1392 17.6816 0.713183 0.360443 1 7.55046 2.33179 6.27001 0.654734 0.837389 2 9.72847 2.05767 4.45068 0.882129 1.25317 3 11.2961 2.79994 2.27059 0.481516 1.79971 4 12.8083 3.44694 1.07403 0.637823 2.35346 5 14.0179 3.67504 0.700602 0.57264 2.78285 6 14.9184 3.80535 0.691515 0.56462 3.20764 7 15.6295 3.86539 0.691516 0.56462 3.71648 8 16.2144 3.92876 0.691515 0.56462 3.99213 9 16.7257 3.9931 0.691515 0.56462 4.26315 10 17.1864 4.05672 0.691516 0.564621 5.00918 ``` ### Weisfeiler-Lehman shortest path kernel The table below shows the results of the WL subtree under different subtree heights. ``` height rmse_test std_test rmse_train std_train k_time -------- ----------- ---------- ------------ ----------- -------- 0 35.192 4.49577 28.3604 1.35718 13.5041 1 35.1808 4.50045 27.9335 1.44836 26.8292 2 35.1632 4.50205 28.1113 1.50891 40.2356 3 35.1946 4.49801 28.3903 1.36571 54.6704 4 35.1753 4.50111 27.9746 1.46222 67.1522 5 35.1997 4.5071 28.0184 1.45564 80.0881 6 35.1645 4.49849 28.3731 1.60057 92.1925 7 35.1771 4.5009 27.9604 1.45742 105.812 8 35.1968 4.50526 28.1991 1.5149 119.022 9 35.1956 4.50197 28.2665 1.30769 131.228 10 35.1676 4.49723 28.4163 1.61596 144.964 ``` ### Weisfeiler-Lehman edge kernel The table below shows the results of the WL subtree under different subtree heights. ``` height rmse_test std_test rmse_train std_train k_time -------- ----------- ---------- ------------ ----------- --------- 0 33.4077 4.73272 29.9975 0.90234 0.853002 1 33.4235 4.72131 30.1603 1.09423 1.71751 2 33.433 4.72441 29.9286 0.787941 2.66032 3 33.4073 4.73243 30.0114 0.909674 3.47763 4 33.4256 4.72166 30.1842 1.1089 4.54367 5 33.4067 4.72641 30.0411 1.01845 5.66178 6 33.419 4.73075 29.9056 0.782179 6.14803 7 33.4248 4.72155 30.1759 1.10382 7.60354 8 33.4122 4.71554 30.1365 1.07485 7.97222 9 33.4071 4.73193 30.0329 0.921065 9.07084 10 33.4165 4.73169 29.9242 0.790843 10.0254 ``` ### Treelet kernel **The targets of training data are normalized before calculating the kernel.** ``` RMSE_test std_test RMSE_train std_train k_time ----------- ---------- ------------ ----------- -------- 8.3079 3.37838 2.90887 1.2679 0.500302 ``` ### Path kernel up to depth *d* The table below shows the results of the path kernel up to different depth *d*. The first table is the results using *Tanimoto kernel*, where **The targets of training data are normalized before calculating the kernel.**. ``` depth rmse_test std_test rmse_train std_train k_time ------- ----------- ---------- ------------ ----------- --------- 0 41.6202 6.453 43.6169 2.13212 0.0904737 1 38.8446 6.44648 40.8329 3.44147 0.175414 2 35.2915 4.7813 35.7461 1.61134 0.344896 3 29.4845 3.90351 28.4646 3.00137 0.553939 4 22.6693 6.28053 19.2517 3.42893 0.770649 5 21.7956 5.5225 16.886 2.60519 1.01558 6 20.6049 5.49983 13.1097 2.58431 1.33302 7 20.3479 5.17631 12.0152 2.5928 1.60266 8 19.8228 5.13769 10.7981 2.13082 1.81218 9 19.8734 5.10369 10.7997 2.09549 2.21726 10 19.8708 5.09217 10.7787 2.10002 2.41006 ``` The second table is the results using *MinMax kernel*. ``` depth rmse_test std_test rmse_train std_train k_time ------- ----------- ---------- ------------ ----------- -------- 0 12.58 2.73235 12.1209 0.500467 0.377576 1 12.6215 2.18866 10.2243 0.734261 0.456332 2 7.42903 2.69395 2.71885 0.732922 0.585278 3 9.02468 2.50808 1.54 1.13813 0.706556 4 10.0811 3.6477 1.36029 1.42399 0.847957 5 11.3005 4.44163 1.08518 1.06206 1.00086 6 12.186 4.88816 1.06443 1.00191 1.19792 7 12.7534 5.14529 1.19912 1.34031 1.4372 8 13.0471 5.27184 1.35822 1.84315 1.68449 9 13.1789 5.27707 1.36002 1.84834 1.96545 10 13.2538 5.26425 1.36208 1.85426 2.24943 ``` ### Tree pattern kernel Until N kernel when h = 2: ``` lmda rmse_test std_test rmse_train std_train k_time ----------- ----------- ---------- ------------ ----------- -------- 1e-10 7.46524 1.71862 5.99486 0.356634 38.1447 1e-09 7.37326 1.77195 5.96155 0.374395 37.4921 1e-08 7.35105 1.78349 5.96481 0.378047 37.9971 1e-07 7.35213 1.77903 5.96728 0.382251 38.3182 1e-06 7.3524 1.77992 5.9696 0.3863 39.6428 1e-05 7.34958 1.78141 5.97114 0.39017 37.3711 0.0001 7.3513 1.78136 5.94251 0.331843 37.3967 0.001 7.35822 1.78119 5.9326 0.32534 36.7357 0.01 7.37552 1.79037 5.94089 0.34763 36.8864 0.1 7.32951 1.91346 6.42634 1.29405 36.8382 1 7.27134 2.20774 6.62425 1.2242 37.2425 10 7.49787 2.36815 6.81697 1.50182 37.8286 100 7.42887 2.64789 6.68766 1.34809 36.3701 1000 7.24914 2.65554 6.81906 1.41008 36.1695 10000 7.08183 2.6248 6.93431 1.38441 37.5723 100000 8.021 3.43694 8.69813 0.909839 37.8158 1e+06 8.49625 3.6332 9.59333 0.96626 38.4688 1e+07 10.9067 3.17593 11.5642 2.07792 36.9926 1e+08 61.1524 10.4355 65.3527 13.9538 37.1321 1e+09 99.943 13.6994 98.8848 5.27014 36.7443 1e+10 100.083 13.8503 97.9168 3.22768 37.096 ``` ### Cyclic pattern kernel **This kernel is not tested on dataset Acyclic** Results on dataset MAO: ``` cycle_bound accur_test std_test accur_train std_train k_time ------------- ------------ ---------- ------------- ----------- -------- 0 0.642857 0.146385 0.54918 0.0167983 0.187052 50 0.871429 0.1 0.698361 0.116889 0.300629 100 0.9 0.111575 0.732787 0.0826366 0.309837 150 0.9 0.111575 0.732787 0.0826366 0.31808 200 0.9 0.111575 0.732787 0.0826366 0.317575 ``` Results on dataset PAH: ``` cycle_bound accur_test std_test accur_train std_train k_time ------------- ------------ ---------- ------------- ----------- -------- 0 0.61 0.113578 0.629762 0.0135212 0.521801 10 0.61 0.113578 0.629762 0.0135212 0.52589 20 0.61 0.113578 0.629762 0.0135212 0.548528 30 0.64 0.111355 0.633333 0.0157935 0.535311 40 0.64 0.111355 0.633333 0.0157935 0.61764 50 0.67 0.09 0.658333 0.0345238 0.733868 60 0.68 0.107703 0.671429 0.0365769 0.871147 70 0.67 0.100499 0.666667 0.0380208 1.12625 80 0.78 0.107703 0.709524 0.0588534 1.19828 90 0.78 0.107703 0.709524 0.0588534 1.21182 ```