[jboss-user] [JBoss Tools (users)] - Re: WTP2.0.1, JBossAS-Tools-1.0.0.beta4 not publishing corre

[dlmiles]

"max.andersen at jboss.com" wrote : No - we don't use renameTo(), we use copy. What we do in case of updates I'll need to check in code - or maybe even get rob to chime in.

Do you copy in situation ?  This is what tomcat driver used to do in WTP0.7 thru WTP1.0 or so.  Copy in situation means to me you take the existing file, truncate it, then write blocks of data repeatably to it, then close the file.

The file instance is the same after the copy as it was before the copy.  This can cause the JVM to crash when applied to JARs which change.  I would get between 5 JVM crashes a day on Linux when developing for Tomcat, this is always because I had a contributing J2EE Utility project which I was also developing in parallel to the web-app.

The problem is that the JVM holds an open descriptor to the file and possibly has a mmap() in place for the contents and continues to access it, expecting the old contents to be there.  You can't change contents from under the JVM, you need to make a new file instance and replace (on unix, on windows file locking becomes an issue and was the reason for the broken methodology in the TC driver in the first place)


anonymous wrote : And I've never bumped into these issues and I run both on Linux and Windows...maybe my machine is just too fast ...

True the window of time problem can be made visible with a slower machine.  Getting a faster machine can cover up the problem but none the less there is a design issue.


anonymous wrote : The descriptor is not updated before the user actually updates it, but I can see how it might be relevant when doing a full update.

Okay another beef I have here is that tools like XDoclet which modify descriptor don't do a replace-if-modified check after they build the new descriptor file.  This means the timestamps on descriptors can keep incrementing when the contents were computed to be the same as the previous version.  So there are cases where the user may not have actually modified a descriptor but the system did it for them (or rather the system modified a dependency which causes a rebuild of the descriptor but it turns out no change to the descriptor resulted)..  This is very annoying and non productive to a developer.

The tooling uses incremental updates anyway for the large parts everything I'm saying relates to that.


anonymous wrote : With respect to the deployer ignoring updates if a certain file is present that is something I always thought about doing but is it really an issue in day-to-day dev ? 
  | 
  | I think you raise some valid issues, I'll point Rob to this thread; but I would also really like to have concrete examples of things actually failing because of these things (your missing files is not one of them as far as i can see - they might be a cause of something else which we definitly needs fixing)

Its too soon for me to say if IMHO its a issue in day-to-day, certainly in tomcat the develop-run-test cycle is faster especially in relation to JSPs.  So it did make a difference there.

This is early days for me using JBossTools-AS the large part of this comment is really clarifying the issues I found with TC which has now been fixed in WTP so I'm happy to provide the heads up and make you think in this realm about these matters.

But sure I don't have any concrete examples of bad things with JBossTools-AS at this moment because I've not gotten past the basic functionality of deploying an app. :)



"max.andersen at jboss.com from another post" wrote : Another thought is that a ant build file doing incremental updates would have the exact same issues....

My thoughts here are I don't care as I always considered such ant tasks broken by design and have done deployment by copying a file to "default/deploy/MyEAR.tmp" and then renaming the file to "default/deploy/MyEAR.ear" to make it visible to the deployer with a trivial script.


One problem here is that Windows really doesn't have a transactional file system, it does not allow files that are open by another process to be deleted or renamed (not in Win2000 and Win98SE etc..) the concept was an after thought to the windows API added to WinXP.  It does not allow a transactional rename, i.e. because the target of the rename has to be deleted first, and it can only be deleted if no other process has it open, so there is a window of time when another process will see no file exists.

Unix on the other hand does have a transactional file system the semantics of which are mandated by POSIX.  It does allow a file than is open by another process to be deleted and/or renamed.  It does have a rename() call which will replace the existing target atomically.

Unix has had very specific semantics for dealing with updating files atomically for many years, take the example of updating /etc/passwd to add a new account, what horrors would result if the /etc/passwd file for just a moment didn't exist, because another process deleted it before doing a rename of the /etc/passwd.tmp to /etc/passwd.  The correct way of dealing with these issues was solved many years ago, its a shame none of it rubbed off on the windows platform.

IMHO J2EE stuff should have slightly different implementations to deal with each systems  capabilities and quirks.  The unix implemention would be pretty simple, the windows implementation can deal with file locking and other such hoops you're made to jump through.


"max.andersen at jboss.com from another post" wrote : btw. timestamps are updated and since the existing file won't change timestamp before the touch it should be fine (in most cases) 

The timestamp issue is mainly about not missing a lost update.  i.e. imagine there are 2 updates in the workspace for a file queued in the event system, due to the speed of the system the first update is being pushed out to server runtime now.

Due to the copy in situation method of updating the modification date of the file is related to the write's currently being performed on the file.  This timestamp is after the timestamp of the 2nd update already queued into the system and the runtime notices the timestamp on the file and takes action.

Now the IDE runtime driver completes the update, and keeps processing and eventually gets to work on another update on the same file.  This time it writes the file out in situation again and then fixes the last modification timestamp after the last write.  This time the server runtime due to it being busy only sees the file after the last modification was fixed, but this modification stamp is older than the last one so it ignores the update.

Copying to a temporary first, fixing the timestamp, then moving the file to its rightful position closes the door on this issue too.

IMHO the deployers should not care if a timestamp is older or newer, it should only care if the timestamp is different (as in !=) to the one its holding for the last deployment action.


All of these issues have further reaching consequences to provide robustness of implementation in the J2EE environment than just server tooling, it doesn't hurt to think and appreciate these matters.

"rob.stryker at jboss.com from another post" wrote : I'm relatively certain eclipse doesn't fire any resource changed events until after it's sure the local filesystem has finished writing those changes... so that only leaves the step between the workspace / fs and then runtime / fs. 
  | 
  | The only thing I can think of right now that'd be best for incremental deployment (or even a new full publish) is to see if I can turn off the deployment scanner during a publish and turn it back on directly thereafter. This would probably be done with jmx. 

I'm only talking of the copy from workspace (or rather "project space" build area) to runtime in all of my comments here.

Sure on JMX if thats a way to go, the existence of a file idea was simply the most lightweight method I could think (that would also work with tomcat), but ideally both are good as JMX can work across a network (for when you have a remote deployment feature).  Also the choice on using a .properties file would also allow communication between the IDE and the deployer for quick easy parsing, there are a number of useful issues that an IDE might want to convey to a deployer, for example in the case of Tomcat when taglibs are updated asking the container to flush its tag cache.  So you end up with 2  well known file name paths, one used when the IDE is active and that file is then renamed to another filename when the IDE is inactive but has instructions it want the deployer to know about by parsing the .properties file.

This would then lead on to the deployer also writing out a .properties file to communicate with the IDE in the opposite direction.  So the IDE didn't modify when the deployer was active.

The feature if it carries any performance bottleneck might be best served with a tickbox to disable it within the server runtime instance configuration in the server tooling.


Hopefully food for thought.

View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4092824#4092824

Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4092824