Problem with WorkingMemory serialization ---------------------------------------- Key: JBRULES-592 URL: http://jira.jboss.com/jira/browse/JBRULES-592 Project: JBoss Rules Issue Type: Bug Security Level: Public(Everyone can see) Components: Reteoo Affects Versions: 3.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.0.5 Reporter: Edson Tirelli Assigned To: Edson Tirelli Fix For: 3.1-m1, 3.0.6 REPORTED BY MICHAEL SUZIO: ----------------------------------------------- I've run into a problem. Strangely enough, I swear this worked in the past, but then we went into a long coding phase and emerged from it to find out our test cases didn't actually work "in the wild". Took some time, but eventually I figured out the why of that -- and coded better tests. In any case, here is what is happening: - We create a rulebase from some DRL text (doesn't matter how it gets created, my test code below is *one* example case) - A Working Memory object is created: WorkingMemory wm =3D3D3D rulebase.newWorkingMemory(); - The WorkingMemory is serialized to disk via normal ObjectOutputStream methods Now, *while the VM is running*, I can create new RuleBase objects, read that WorkingMemory back in, and attach it to those new RuleBase objects, and it always works. However, if I stop the VM, and re-run the sample program below, it fails with this stack trace: java.lang.ClassNotFoundException: [Lorg.drools.reteoo.ObjectTypeNode; at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) at java.lang.ClassLoader.loadClass (ClassLoader.java:251) at org.drools.rule.CompositePackageClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at org.drools.common.ObjectInputStreamWithLoader.resolveClass (Unknown Source) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1544) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1466) at java.io.ObjectInputStream.readArray (ObjectInputStream.java:1591) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1299) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) at java.util.HashMap.readObject (HashMap.java:1067) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav a:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessor Impl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:946) at java.io.ObjectInputStream.readSerialData (ObjectInputStream.java:1809) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1719 ) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1305) at java.io.ObjectInputStream.readArray (ObjectInputStream.java:1634) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1299) at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1634) at java.io.ObjectInputStream.readObject0 (ObjectInputStream.java:1299) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1908) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1832) at java.io.ObjectInputStream.readOrdinaryObject (ObjectInputStream.java:1719 ) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1305) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1908) at java.io.ObjectInputStream.readSerialData (ObjectInputStream.java:1832) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1719 ) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1305) at java.io.ObjectInputStream.defaultReadFields (ObjectInputStream.java:1908) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1832) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1719 ) at java.io.ObjectInputStream.readObject0 (ObjectInputStream.java:1305) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348) at org.drools.common.AbstractRuleBase.newWorkingMemory(Unknown Source) at org.drools.common.AbstractRuleBase.newWorkingMemory (Unknown Source) at DroolsFailureTest.runTest(DroolsFailureTest.java:52) at DroolsFailureTest.main(DroolsFailureTest.java:25) So, it appears that WorkingMemory objects cannot (at least for me in this case) be serialized, then read back into a new VM. That always fails, and it's quite vexing. Our intention is to be running rules on large amounts of data that periodically come in, and we need to save our progress as we go along and be able to restore that data anywhere in the data cluster that the job gets parceled out to (so, ultimately, the WorkingMemory data is going to go into a database or possibly get written out to a shared filesystem among the drones). If I can't save/restore data transparently between runs, this is pretty much a show-stopper. Technical details: Java 1.5.0_10 (but I tested this on 1.5.0_04, 1.5.0_06, 1.5.0_08, and 1.5.0_09 -- basically, whatever we had available on other people's workstations), all on Windows boxes. JBoss Rules 3.0.3 and 3.0.5 both seemed to fail consistently. Here is the sample code (it's two classes, save them to appropriate files). I think these should be reproducible cases. Run the DroolsFailureTest twice, and it will work the first time (showing that reads/writes within the VM work), and then fail the second time it is run (when the cached WorkingMemory on disk then is getting read back into a new VM). import java.io.*; import org.drools.*; import org.drools.compiler.*; public class DroolsFailureTest { public static void main(String[] args) { boolean removeFile = false; if (args.length > 0) { removeFile = args[0].equalsIgnoreCase("remove"); } if (removeFile) { WorkingMemorySaver memSaver = getWorkingMemorySaver(); memSaver.clearWorkingMemory(); } runTest(); // run 1 -- should always work runTest(); // run 2 -- fails? System.exit(0); } public static void runTest() { System.out.println ("Getting working memory handler"); WorkingMemorySaver memSaver = getWorkingMemorySaver(); RuleBase rb = getRuleBase(); // Uncomment this -- prove we get a different RuleBase object // each time // // System.err.println("Rulebase hashcode = " + rb); InputStream is = memSaver.getWorkingMemoryInputStream(); System.out.println("Got input stream " + is); WorkingMemory wm = null; if (is != null) { System.err.println("Trying to restore object from input stream"); try { wm = rb.newWorkingMemory(is); System.err.println("Got " + wm); } catch (Exception e) { e.printStackTrace(); } } if (wm == null) { wm = rb.newWorkingMemory(); System.err.println("No working memory, creating new one = " + wm); } System.out.println("Asserted data, running the rules"); // wm.fireAllRules(); System.err.println ("Saving working memory"); memSaver.saveWorkingMemory(wm); } private static RuleBase getRuleBase() { System.setProperty("drools.compiler", "JANINO"); RuleBaseLoader rbl = RuleBaseLoader.getInstance(); String ruleText = getRuleText(); RuleBase rb = null; try { rb = rbl.loadFromReader(new StringReader(ruleText)); } catch (CheckedDroolsException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } return rb; } private static String getRuleText() { String ruleText = "package test;\n" + "rule TestRule\n" + "when\n" + " sObj: String()\n" + "then\n" + " System.out.println(\"Saw \" + sObj);\n" + "end"; return ruleText; } private static WorkingMemorySaver getWorkingMemorySaver() { WorkingMemorySaver memSaver = new WorkingMemorySaver(); memSaver.setWorkingMemoryDir("C:\\"); return memSaver; } } import java.io.*; import org.drools.*; /** * */ public class WorkingMemorySaver { private File workingMemoryDir = null; public WorkingMemorySaver() { } public InputStream getWorkingMemoryInputStream() { File workingMemoryFile = getWorkingMemoryFile(); InputStream stream = null; if ( workingMemoryFile.exists()) { try { stream = new FileInputStream(getWorkingMemoryFile()); } catch (FileNotFoundException e) { e.printStackTrace(); } } return stream; } public void saveWorkingMemory(WorkingMemory wm) { try { //noinspection IOResourceOpenedButNotSafelCuyClosed ObjectOutputStream oStream = new ObjectOutputStream((new FileOutputStream(getWorkingMemoryFile()))); oStream.writeObject(wm); } catch (IOException e) { e.printStackTrace(); } } /** * used in case the file becomes corrupted */ public void clearWorkingMemory() { try { File workingMemoryFile = getWorkingMemoryFile(); if (workingMemoryFile != null) { boolean deleted = workingMemoryFile.delete(); if (deleted) { System.err.println("Deleted working memory file " + workingMemoryFile.getName()); } } } catch (Exception e) { e.printStackTrace(); } } public File getWorkingMemoryDir() { return workingMemoryDir; } public void setWorkingMemoryDir(File workingMemoryDir) { this.workingMemoryDir = workingMemoryDir; } public void setWorkingMemoryDir(String workingMemoryDirName) { this.workingMemoryDir = new File(workingMemoryDirName); } private File getWorkingMemoryFile() { return new File(workingMemoryDir, "working_memory.bin"); } } A possible related bug is in this thread: http://www.mail-archive.com/user@drools.codehaus.org/msg01731.html Which points out this Sun bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=3D3D3D6434149 In fact, it looks like my problem is just a variant on Raffael's. Like Raffael, I've tried JBoss 3.0.5 (no help) and Java 1.6 (similar results -- different "stream corrupted" error in my case, but essentially the same). I don't see a resolution to his bug on the archives -- Raffael, did you find a fix? Please let me know! Sorry for such a long post, but I hope someone can point out what I can do to fix this issue.