Hi Henri!
The server has its own ports that it listens to in the bindings xml, as you know. Unfortunately, the tools cannot automatically know exactly how you launched your server and everything about it. If you open the server editor, (double-click the server), you can see a section labeled "Ports". Here, the tools outline what ports they EXPECT to communicate with the server on.
These are automatically discovered via xpaths, but, if you change your binding ports via launch args, then odds are the toolset is using the wrong xpath. IT's most likely using the xpaths for the default bindings, scanning your xml files for that xpath, and then taking that value. You can get around this by unchecking "Automatically Detect" and instead putting in the direct port you want the eclipse plugins to use when communicating with that server.
Another option is to click on "configure", and, in the dialog that comes up, select one of the other xpaths that are listed. (To add your own xpaths, in the servers view, expand the server, and the section labeled "XML Configuration". Inside, you should see a Ports folder, and, in there, you can add your own xpaths.
Hope this helps!