It seems I couldn't drag and drop the existing projects to server adapter, Is there any configuration or plugin required to do this action?
no, there's no additional configuration needed. If you're not able to drag and drop your project, then your project's not an Eclipse WTP project. If your project is a maven project then this means that you're mssing the maven integration for Eclipse: http://www.eclipse.org/m2e-wtp/
Another question is for the debug, when I created the new remote java application in the debug configurations, nothing listed if choose jboss maven container in the source tab, Do I need to install plugins or other steps to make it avaiable in the jboss maven container?
If you're choosing your project then the source code should get associated. I honestly dont have "jboss maven container" if I try to replicate what you're reporting. To me it looks like there's something very weird with your project?