Hi,
the configuration looks not bad.
How do you measure? Do you use the same application and add only the getConnection()? This might avoid other bottlenecks.
Do you use a DB query or only connect?
Do you have GC logging enabled that we can sure this is not the issue?
From the statistics it looks like that only 100 connections are used (MaxUsedCount) and there is only less wait for a connection (AverageBlockingTime / TotalBlockingTime)
You might add time statistic to your code that you are able to see where the time is spend.
Also you might set min-pool-size==max-pool-size to avoid releasing connections and start after the pool is filled up.
Maybe the bottleneck is the network or database?