It doesn't come back when I update the maven configuration.
When I've tried to update the configuration before the change at the Deployment Assembly the test folder was still there.
I guess it's been included while I was trying to figure out the right configuration to use in the project and it's never been deleted. I'm sure I've never added it manually.