This definitely needs improvement, but it's not straightforward either. I'll use this space to discuss first before incorporating a better solution.

The problem is mostly because the real world dimensions for the device are not available. 640px on a mobile device is different from 640px on a desktop or laptop. We have the following options today:

• Use the Modernizr.touch test instead of device width test. This is not clean, since this verifies for WebKit style touch support. It fails on IE10 Metro.
• Use the script provided by detectmobilebrowsers.com. Released under public domain. The downside is that this is user-agent sniffing and needs updating as new mobile browsers are released.
• Add a orientation criteria to the test, like Modernizr.mq('(max-device-width: XYZpx) and (orientation: portrait)'). The downside is that the test would fail in landscape mode.