[jbosstools-issues] [JBoss JIRA] (ERT-550) Benchmark old and new JDT Indexers

[ https://issues.jboss.org/browse/ERT-550?page=com.atlassian.jira.plugin.sy... ] Roland Grunberg edited comment on ERT-550 at 11/28/17 1:18 PM: --------------------------------------------------------------- Commented on jdt-core-dev asking for some clarification and it would seem that the new indexer is not intended to outperform the old one "across the board". I am finding it difficult to produce cases where the new one outperforms the old one so I may have to keep trying later on. Methadology : See https://github.com/rgrunber/eclipse.jdt.core/tree/index_performance (should be top-most commit) We take a set of jars, and split them (roughly) evenly among the number of projects to exist in the workspace. When then perform a type hierarchy search on java.lang.Object from one of the projects. The results are calculated from an average of 5 runs. There's also testing of the actual time it takes to build the indexes but these seem pretty consistent across all scenarios. The new indexer is usually 2x as large and takes 3-4x as long to build. Results so far for search indexing : 1 workspace project / 1500 jars / 43000 classes Old : 5s New : 22s 1 workspace project / 1 jar / 18000 classes Old : 1s New : 3s 1 workspace project / 1 jar / 24000 classes Old : 2s New : 7s 1 workspace project / 1500 jars / 75000 classes Old : 7s New : 34s 10 workspace project / 1500 jars / 75000 classes Old : 4s New : 7s 100 workspace project / 1500 jars / 75000 classes Old : 3s New : 6s 1 workspace project / 4000 jars / 120000 classes Old : 44s New : 320s 10 workspace project / 4000 jars / 120000 classes Old : 9s New : 15s 100 workspace project / 4000 jars / 120000 classes Old : 8s New : 12s The results do seem to indicate that the new indexer works much faster when the number of projects increases, but its improvements never seem to do better than the old indexer. Notes / Definitions : jars - referring to .jar files, so N jars is exactly that many jars on the filesystem classes - this is the number of classes detected by the indexer, and doesn't correspond to total number of unique classes of all jars was (Author: rgrunber): Commented on jdt-core-dev asking for some clarification and it would seem that the new indexer is not intended to outperform the old one "across the board". I am finding it difficult to produce cases where the new one outperforms the old one so I may have to keep trying later on. Methadology : See https://github.com/rgrunber/eclipse.jdt.core/tree/index_performance (should be top-most commit) We take a set of jars, and split them (roughly) evenly among the number of projects to exist in the workspace. When then perform a type hierarchy search on java.lang.Object from one of the projects. The results are calculated from an average of 10 runs. There's also testing of the actual time it takes to build the indexes but these seem pretty consistent across all scenarios. The new indexer is usually 2x as large and takes 3-4x as long to build. Results so far for search indexing : 1 workspace project / 1500 jars / 43000 classes Old : 5s New : 22s 1 workspace project / 1 jar / 18000 classes Old : 1s New : 3s 1 workspace project / 1 jar / 24000 classes Old : 2s New : 7s 1 workspace project / 1500 jars / 75000 classes Old : 7s New : 34s 10 workspace project / 1500 jars / 75000 classes Old : 4s New : 7s 100 workspace project / 1500 jars / 75000 classes Old : 3s New : 6s 1 workspace project / 4000 jars / 120000 classes Old : 44s New : 320s 10 workspace project / 4000 jars / 120000 classes Old : 9s New : 15s 100 workspace project / 4000 jars / 120000 classes Old : 8s New : 12s The results do seem to indicate that the new indexer works much faster when the number of projects increases, but its improvements never seem to do better than the old indexer. Notes / Definitions : jars - referring to .jar files, so N jars is exactly that many jars on the filesystem classes - this is the number of classes detected by the indexer, and doesn't correspond to total number of unique classes of all jars -- This message was sent by Atlassian JIRA (v7.5.0#75005)