in the link below,
http://community.jboss.org/people/bpmn2user/blog/2011/03/28/jbpm5--work-itemdomain-specific-process-hello-process-example
at step 4 you created a class corresponding to the work item inside the WorkItemHandler implementation class there is a method you have to implement public void executeWorkItem(..) inside this method you can put custom logic, i.e. custom business logic, call another application via a jar, or call a web service etc.
If for example you call another class or application which is inside a jar then you'll have to include the jar (and any other dependencies of the jar to run) in the libs of your jbpm application or if you're using the gwt-server implementation with the console then you'll have to place the jar file (and any other dependencies of the jar to run) inside the gwt-console-server.war/lib along with the other jars.
Also don't forget to include the workitem jar itself inside the lib of gwt-console-server.war or your jbpm application's classpath, depending how you use jbpm5.