I still haven't resolved this problem, but looking into it further the calling side is behaving correctly. Attempting to use the future after the calling cancel results in a CancellationException.
The specs for Future have the following information for cancel:
"Attempts to cancel execution of this task. This attempt will fail if the task has already completed, already been cancelled, or could not be cancelled for some other reason"
Does anyone know what "some other reason" could be?
Could the container be doing something smart and choosing not to interrupt the executing thread?