linlin
GitHub
6 years ago
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 93 additions and 98 deletions
Split View
Diff Options
-
+93 -98README.md
+ 93
- 98
README.md
View File
| @@ -3,106 +3,101 @@ A python package for graph kernels. | |||
| ## Requirements | |||
| python==3.6.5 | |||
| numpy==1.15.2 | |||
| scipy==1.1.0 | |||
| matplotlib==3.0.0 | |||
| networkx==2.2 | |||
| scikit-learn==0.20.0 | |||
| tabulate==0.8.2 | |||
| tqdm==4.26.0 | |||
| suffix-tree==0.0.6 (for the path kernel up to length h only) | |||
| control==0.8.0 (for generalized random walk kernels only) | |||
| slycot===0.3.2.dev-5263ada (for generalized random walk kernels only, requires fortran compiler, gfortran for example) | |||
| Modules better be upgraded. | |||
| ipython==7.0.1 | |||
| ipykernel=5.1.0 | |||
| ipython-genutils=0.2.0 | |||
| ipywidgets==7.4.2 | |||
| ## Results with minimal test RMSE for each kernel on dataset Asyclic | |||
| All kernels expect for Cyclic pattern kernel are tested on dataset Asyclic, which consists of 185 molecules (graphs). (Cyclic pattern kernel is tested on dataset MAO and PAH.) | |||
| The criteria used for prediction are SVM for classification and kernel Ridge regression for regression. | |||
| For prediction we randomly divide the data in train and test subset, where 90\% of entire dataset is for training and rest for testing. 30 splits are performed. For each split, we first train on the train data, then evaluate the performance on the test set. We choose the optimal parameters for the test set and finally provide the corresponding performance. The final results correspond to the average of the performances on the test sets. | |||
| | Kernels | train_perf | valid_perf | test_perf | Parameters | gram_matrix_time | | |||
| |---------------------------|------------|------------|------------|--------------------------------------------------|---------------------------| | |||
| | Shortest path | 28.77±0.60 | 38.31±0.92 | 39.40±6.32 | 'alpha': '1.00' | 13.54" | | |||
| | Marginalized | 12.95±0.37 | 19.02±1.73 | 18.24±5.00 | 'p_quit': 0.2, 'alpha': '1.00e-04' | 437.04"/447.44"±5.32" | | |||
| | Extension of Marginalized | 20.65±0.44 | 26.06±1.83 | 26.84±4.81 | 'p_quit': 0.1, 'alpha': '5.62e-04' | 6388.50"/6266.67"±149.16" | | |||
| | Path | 8.71±0.63 | 19.28±1.75 | 17.42±6.57 | 'alpha': '2.82e-02' | 21.94" | | |||
| | WL subtree | 13.90±0.35 | 18.47±1.36 | 18.08±4.70 | 'height': 1.0, 'alpha': '1.50e-03' | 0.79"/1.32"±0.76" | | |||
| | WL shortest path | 28.74±0.60 | 38.20±0.62 | 39.02±6.09 | 'height': 10.0, 'alpha': '1.00' | 146.83"/80.63"±45.04" | | |||
| | WL edge | 30.21±0.64 | 36.53±1.02 | 38.42±6.42 | 'height': 5.0, 'alpha': '6.31e-01' | 5.24"/5.15"±2.83" | | |||
| | Treelet | 7.33±0.64 | 13.86±0.80 | 15.38±3.56 | 'alpha': '1.12e+01' | 0.48" | | |||
| | Path up to d | 5.76±0.27 | 9.89±0.87 | 10.21±4.16 | 'depth': 2.0, 'k_func': 'MinMax', 'alpha ': '0.1' | 0.56"/1.16"±0.75" | | |||
| | Cyclic pattern | | | | | | | |||
| | Walk up to n | 20.88±0.74 | 23.34±1.11 | 24.46±6.57 | 'n': 2.0, 'alpha': '1.00e-03' | 0.56"/331.70"±753.44" | | |||
| In table above,last column is the time consumed to calculate the gram matrix. Note for | |||
| kernels which need to tune hyper-parameters that are required to calculate gram | |||
| matrices, average time consumption and its confidence are obtained over the | |||
| hyper-parameters grids, which are shown after "/". The time shown before "/" | |||
| is the one spent on building the gram matrix corresponding to the best test | |||
| performance. | |||
| * See detail results in [results.md](pygraph/kernels/results.md). | |||
| * python==3.6.5 | |||
| * numpy==1.15.2 | |||
| * scipy==1.1.0 | |||
| * matplotlib==3.0.0 | |||
| * networkx==2.2 | |||
| * scikit-learn==0.20.0 | |||
| * tabulate==0.8.2 | |||
| * tqdm==4.26.0 | |||
| * control==0.8.0 (for generalized random walk kernels only) | |||
| * slycot==0.3.3 (for generalized random walk kernels only, which requires a fortran compiler, gfortran for example) | |||
| ## How to use? | |||
| Simply clone this repository and voilà! Then check [`notebooks`](https://github.com/jajupmochi/py-graph/tree/ljia/notebooks) directory for demos: | |||
| * [`notebooks`](https://github.com/jajupmochi/py-graph/tree/ljia/notebooks) directory includes test codes of graph kernels based on linear patterns; | |||
| * [`notebooks/tests`](https://github.com/jajupmochi/py-graph/tree/ljia/notebooks/tests) directory includes codes that test some libraries and functions; | |||
| * [`notebooks/utils`](https://github.com/jajupmochi/py-graph/tree/ljia/notebooks/utils) directory includes some useful tools, such as a Gram matrix checker and a function to get properties of datasets; | |||
| * [`notebooks/else`](https://github.com/jajupmochi/py-graph/tree/ljia/notebooks/else) directory includes other codes that we used for experiments. | |||
| ## List of graph kernels | |||
| * Based on walks | |||
| * The common walk kernel [1] | |||
| * Exponential | |||
| * Geometric | |||
| * The marginalized kenrel | |||
| * With tottering [2] | |||
| * Without tottering [7] | |||
| * The generalized random walk kernel [3] | |||
| * Sylvester equation | |||
| * Conjugate gradient | |||
| * Fixed-point iterations | |||
| * Spectral decomposition | |||
| * Based on paths | |||
| * The shortest path kernel [4] | |||
| * The structural shortest path kernel [5] | |||
| * The path kernel up to length h [6] | |||
| * The Tanimoto kernel | |||
| * The MinMax kernel | |||
| ## Computation optimization methods | |||
| * Python’s `multiprocessing.Pool` module is applied to perform **parallelization** on the computations of all kernels as well as the model selection. | |||
| * **The Fast Computation of Shortest Path Kernel (FCSP) method** [8] is implemented in *the random walk kernel*, *the shortest path kernel*, as well as *the structural shortest path kernel* where FCSP is applied on both vertex and edge kernels. | |||
| * **The trie data structure** [9] is employed in *the path kernel up to length h* to store paths in graphs. | |||
| ## Issues | |||
| * This library uses `multiprocessing.Pool.imap_unordered` function to do the parallelization, which may not be able to run correctly under Windows system. For now, Windows users may need to comment the parallel codes and uncomment the codes below them which run serially. We will consider adding a parameter to control serial or parallel computations as needed. | |||
| * Some modules (such as `Numpy`, `Scipy`, `sklearn`) apply [`OpenBLAS`](https://www.openblas.net/) to perform parallel computation by default, which causes conflicts with other parallelization modules such as `multiprossing.Pool`, highly increasing the computing time. By setting its thread to 1, `OpenBLAS` is forced to use a single thread/CPU, thus avoids the conflicts. For now, this procedure has to be done manually. Under Linux, type this command in terminal before running the code: | |||
| ``` | |||
| $ export OPENBLAS_NUM_THREADS=1 | |||
| ``` | |||
| Or add `export OPENBLAS_NUM_THREADS=1` at the end of your `~/.bashrc` file, then run | |||
| ``` | |||
| $ source ~/.bashrc | |||
| ``` | |||
| to make this effective permanently. | |||
| ## Results | |||
| Check this paper for detailed description of graph kernels and experimental results: | |||
| Linlin Jia, Benoit Gaüzère, and Paul Honeine. Graph Kernels Based on Linear Patterns: Theoretical and Experimental Comparisons. working paper or preprint, March 2019. URL https://hal-normandie-univ.archives-ouvertes.fr/hal-02053946. | |||
| ## References | |||
| [1] K. M. Borgwardt and H.-P. Kriegel. Shortest-path kernels on graphs. In Proceedings of the International Conference on Data Mining, pages 74-81, 2005. | |||
| [1] Thomas Gärtner, Peter Flach, and Stefan Wrobel. On graph kernels: Hardness results and efficient alternatives. Learning Theory and Kernel Machines, pages 129–143, 2003. | |||
| [2] H. Kashima, K. Tsuda, and A. Inokuchi. Marginalized kernels between labeled graphs. In Proceedings of the 20th International Conference on Machine Learning, Washington, DC, United States, 2003. | |||
| [3] Suard F, Rakotomamonjy A, Bensrhair A. Kernel on Bag of Paths For Measuring Similarity of Shapes. InESANN 2007 Apr 25 (pp. 355-360). | |||
| [4] N. Shervashidze, P. Schweitzer, E. J. van Leeuwen, K. Mehlhorn, and K. M. Borgwardt. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12:2539-2561, 2011. | |||
| [5] Gaüzère B, Brun L, Villemin D. Two new graphs kernels in chemoinformatics. Pattern Recognition Letters. 2012 Nov 1;33(15):2038-47. | |||
| [6] Liva Ralaivola, Sanjay J Swamidass, Hiroto Saigo, and Pierre Baldi. Graph kernels for chemical informatics. Neural networks, 18(8):1093–1110, 2005. | |||
| [7] Pierre Mahé and Jean-Philippe Vert. Graph kernels based on tree patterns for molecules. Machine learning, 75(1):3–35, 2009. | |||
| [8] Tamás Horváth, Thomas Gärtner, and Stefan Wrobel. Cyclic pattern kernels for predictive graph mining. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 158–167. ACM, 2004. | |||
| [9] Thomas Gärtner, Peter Flach, and Stefan Wrobel. On graph kernels: Hardness results and efficient alternatives. Learning Theory and Kernel Machines, pages 129–143, 2003. | |||
| ## Updates | |||
| ### 2018.02.28 | |||
| * ADD *walk kernel up to n* and its result on dataset Asyclic. | |||
| * MOD training process, use nested cross validation for model selection. Recalculate performance of all kernels. | |||
| ### 2018.02.08 | |||
| * ADD *tree pattern kernel* and its result on dataset Asyclic. | |||
| * ADD *cyclic pattern kernel* and its result on classification datasets. | |||
| ### 2018.01.24 | |||
| * ADD *path kernel up to depth d* and its result on dataset Asyclic. | |||
| * MOD treelet kernel, retrieve canonkeys of all graphs before calculate kernels, wildly speed it up. | |||
| ### 2018.01.17 | |||
| * ADD comments to code of treelet kernel. | |||
| ### 2018.01.16 | |||
| * ADD *treelet kernel* and its result on dataset Asyclic. | |||
| * MOD the way to calculate WL subtree kernel, correct its results. | |||
| * ADD *kernel_train_test* and *split_train_test* to wrap training and testing process. | |||
| * MOD readme.md file, add detailed results of each kernel. - linlin | |||
| ### 2017.12.22 | |||
| * ADD calculation of the time spend to acquire kernel matrices for each kernel. | |||
| * MOD floydTransformation function, calculate shortest paths taking into consideration user-defined edge weight. | |||
| * MOD implementation of nodes and edges attributes genericity for all kernels. | |||
| * ADD detailed results file results.md. | |||
| ### 2017.12.21 | |||
| * MOD Weisfeiler-Lehman subtree kernel and the test code. | |||
| ### 2017.12.20 | |||
| * ADD *Weisfeiler-Lehman subtree kernel* and its result on dataset Asyclic. | |||
| ### 2017.12.07 | |||
| * ADD *mean average path kernel* and its result on dataset Asyclic. | |||
| * ADD delta kernel. - linlin | |||
| * MOD reconstruction the code of marginalized kernel. | |||
| ### 2017.12.05 | |||
| * ADD *marginalized kernel* and its result. - linlin | |||
| * ADD list required python packages in file README.md. | |||
| ### 2017.11.24 | |||
| * ADD *shortest path kernel* and its result. | |||
| [3] Vishwanathan, S.V.N., Schraudolph, N.N., Kondor, R., Borgwardt, K.M., 2010. Graph kernels. Journal of Machine Learning Research 11, 1201–1242. | |||
| [4] K. M. Borgwardt and H.-P. Kriegel. Shortest-path kernels on graphs. In Proceedings of the International Conference on Data Mining, pages 74-81, 2005. | |||
| [5] Liva Ralaivola, Sanjay J Swamidass, Hiroto Saigo, and Pierre Baldi. Graph kernels for chemical informatics. Neural networks, 18(8):1093–1110, 2005. | |||
| [6] Suard F, Rakotomamonjy A, Bensrhair A. Kernel on Bag of Paths For Measuring Similarity of Shapes. InESANN 2007 Apr 25 (pp. 355-360). | |||
| [7] Mahé, P., Ueda, N., Akutsu, T., Perret, J.L., Vert, J.P., 2004. Extensions of marginalized graph kernels, in: Proc. the twenty-first international conference on Machine learning, ACM. p. 70. | |||
| [8] Lifan Xu, Wei Wang, M Alvarez, John Cavazos, and Dongping Zhang. Parallelization of shortest path graph kernels on multi-core cpus and gpus. Proceedings of the Programmability Issues for Heterogeneous Multicores (MultiProg), Vienna, Austria, 2014. | |||
| [9] Edward Fredkin. Trie memory. Communications of the ACM, 3(9):490–499, 1960. | |||
| ## Authors | |||
| * [Linlin Jia](https://github.com/jajupmochi), LITIS, INSA Rouen Normandie | |||
| * [Benoit Gaüzère](http://pagesperso.litislab.fr/~bgauzere/#contact_en), LITIS, INSA Rouen Normandie | |||
| * [Paul Honeine](http://honeine.fr/paul/Welcome.html), LITIS, Université de Rouen Normandie | |||
| ## Citation | |||
| Still waiting... | |||
| ## Acknowledgments | |||
| This research was supported by CSC (China Scholarship Council) and the French national research agency (ANR) under the grant APi (ANR-18-CE23-0014). The authors would like to thank the CRIANN (Le Centre Régional Informatique et d’Applications Numériques de Normandie) for providing computational resources. | |||