| One problem with these "progress bar" approaches - and also our current implementation - is that often the full SQL COUNT we have to perform in advance in many databases will take as long as the whole process itself. The reason being that to implement the proper isolation semantics expected by a standard COUNT it needs to somehow iterate all data; fetching it at the same time only adds a small additional cost which often is negligible. So we do that first, and then repeat again for the itaration.. doubling the total time it takes just to have a reasonable estimate half way - we could be done by that time. When thinking about these evolutions, I think we need to take into consideration an alternative approach which skips the count altogether - for many of my personal experiments I regularly find myself commenting out some code, obvsiouly there could be better alternatives. |