<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0" /> <base href="https://hibernate.atlassian.net" />
<title>Message Title</title>
</head>
<body class="jira" style="color: #333; font-family: Arial, sans-serif; font-size: 14px; line-height: 1.429">
<table id="background-table" cellpadding="0" cellspacing="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; background-color: #f5f5f5; border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<!-- header here -->
<tr>
<td id="header-pattern-container" style="padding: 0px; border-collapse: collapse; padding: 10px 20px">
<table id="header-pattern" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="header-avatar-image-container" valign="top" style="padding: 0px; border-collapse: collapse; vertical-align: top; width: 32px; padding-right: 8px"> <img id="header-avatar-image" class="image_fix" src="https://secure.gravatar.com/avatar/12df4da7e3351be801bc16b66caf8038?d=mm&s=48" height="32" width="32" border="0" style="border-radius: 3px; vertical-align: top" />
</td>
<td id="header-text-container" valign="middle" style="padding: 0px; border-collapse: collapse; vertical-align: middle; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px"> <a class="user-hover" rel="hardy.ferentschik" id="email_hardy.ferentschik" href="https://hibernate.atlassian.net/secure/ViewProfile.jspa?name=hardy.ferentschik" style="color:#6c797f;; color: #3b73af; text-decoration: none">Hardy Ferentschik</a> <strong>commented</strong> on an issue
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td id="email-content-container" style="padding: 0px; border-collapse: collapse; padding: 0 20px">
<table id="email-content-table" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; border-spacing: 0; border-collapse: separate">
<tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<td class="email-content-rounded-top mobile-expand" style="padding: 0px; border-collapse: collapse; color: #fff; padding: 0 15px 0 16px; height: 15px; background-color: #fff; border-left: 1px solid #ccc; border-top: 1px solid #ccc; border-right: 1px solid #ccc; border-bottom: 0; border-top-right-radius: 5px; border-top-left-radius: 5px; height: 10px; line-height: 10px; padding: 0 15px 0 16px; mso-line-height-rule: exactly">
</td>
</tr>
<tr>
<td id="text-paragraph-pattern-top" class="email-content-main mobile-expand comment-top-pattern" style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 15px 16px; background-color: #fff; border-bottom: 1px solid #ccc; border-bottom: none; padding-bottom: 0px">
<table class="text-paragraph-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 2px">
<tr>
<td class="text-paragraph-pattern-container mobile-resize-text" style="padding: 0px; border-collapse: collapse; padding: 0px; padding-left: 16px; padding-bottom: 10px; border-left: 1px solid #ccc">
<p style="margin: 10px 0 0 0">Related IRC discussion:</p>
<div class="preformatted panel" style="border-width: 1px;; border: 1px solid #ccc; background: #f5f5f5; font-size: 12px; line-height: 1.333; font-family: monospace; border: 1px solid #ccc; -moz-border-radius: 3px 3px 3px 3px; border-radius: 3px 3px 3px 3px; margin: 9px 0">
<div class="preformattedContent panelContent" style="padding: 9px 12px">
<pre style="margin: 10px 0 0 0; max-height: 30em; overflow: auto; white-space: pre-wrap; word-wrap: normal; white-space: pre; word-break: normal; word-wrap: break-word; word-break: break-word; white-space: pre-wrap">hardy: sannegrinovero: ping
[4:55pm] sannegrinovero: hardy: hi
[4:56pm] hardy: do you have some time to talk ErrorHandler
[4:56pm] hardy: I know you won't like it, but I think for Search 5 we should make some changes
[4:56pm] sannegrinovero: sure.. but didn't you send a pull already?
[4:56pm] hardy: sure sure
[4:56pm] hardy: it goes beyond that
[4:56pm] sannegrinovero: ah ok
[4:57pm] hardy: more of a general discussion
[4:57pm] sannegrinovero: Ok, what's the idea?
[4:57pm] hardy: btw, didn't you already fix the issue regarding the documentation of the ErrorHandler?
[4:57pm] hardy: I kind of still missing some more detailed docs on this is supposed to work
[4:58pm] sannegrinovero: hardy: I don't remember closing that issue. Did I?
[4:58pm] hardy: but my first question, is the ErrorHandler supposed to be used for all type of indexing (batch, even driven and programmatic API_?
[4:59pm] hardy: don't know, I just thought we created it and somehow I thought you fixed it directly
[4:59pm] hardy: have not even looked up the issue yet
[5:00pm] hardy: agh, here it is HSEARCH-1353
[5:00pm] hardy: still open
[5:00pm] sannegrinovero: right
[5:00pm] hardy: so when does the ErrorHandler apply
[5:00pm] hardy: for some reason I thought it is a batch indexing thingy
[5:01pm] hardy: but that's not how it is implemented
[5:01pm] sannegrinovero: I think it was meant to be applied always
[5:01pm] hardy: ok
[5:01pm] hardy: that would align with the online docs "Hibernate Search allows you to configure how exceptions are handled during the indexing process."
[5:01pm] sannegrinovero: but since it gives the application a hook to deal with a failing transaction (conceptually), its application gets tricky when in the scope of massindexing.
[5:01pm] hardy: and partly that is what's happening as well
[5:02pm] hardy: at least when it comes to the processing of LuceneWork instances (backend work so to say)
[5:03pm] hardy: but it does not apply when an error occurs earlier in the indexing process, for example when building the Lucene document
[5:03pm] hardy: the pull request addresses this for the case of the mass indexer
[5:03pm] sannegrinovero: so the MassIndexer was meant as an "operations" tool for people to use offline. If a failure would happen, people would be able to deal with it, as opposing to an online indexing operation on which any compensating work needs to be automated.
[5:03pm] hardy: in EntityConsumerLuceneWorkProducer
[5:04pm] hardy: sure, fixing it for the mass indexer makes sense
[5:04pm] hardy: my issue is with event based indexing or even the programmatic index API
[5:05pm] sannegrinovero: with "online indexing" above I mean event based / programmatic index API
[5:05pm] hardy: as of now, if there is a error in the backend (due to Lucene resources) even event based indexing just logs the error
[5:05pm] hardy: at least nk sp
[5:05pm] hardy: ups
[5:05pm] hardy: at least I think so
[5:06pm] hardy: however, an exception during document creation will be propagated and roll back the transaction
[5:06pm] hardy: is that not inconsistent?
[5:06pm] sannegrinovero: you have to look at the scope of execution in terms of threads
[5:07pm] sannegrinovero: the document creation is done in the user thread, so we can throw any meaningfull exception to the end user
[5:07pm] sannegrinovero: and he will be able to handle it in "traditional" way
[5:07pm] sannegrinovero: rolling back the transaction is fine in this case, as we can only fail for two reasons:
[5:08pm] sannegrinovero: 1) Code bug. e.g. some Bridge implementation goes in NPE
[5:08pm] hardy: keep going, just putting some coffee on
[5:08pm] sannegrinovero: 2)critical system failure (out of memory, etc..)
[5:08pm] sannegrinovero: wo milk please, thanks
[5:09pm] sannegrinovero: so in case of 2) you really don't want to catch the error, but have it fail naturally without messing up with the error messages, etc..
[5:09pm] sannegrinovero: in case of 1) hey they should really fix their code, rolling back a transaction is safer than not doing so.
[5:10pm] sebersole: i know sannegrinovero and gmorling gave some very early feedback
[5:10pm] sannegrinovero: sebersole: no sorry missed it
[5:10pm] hardy: bugger, in fact I have no milk
[5:10pm] sannegrinovero: sebersole: glad to look at it over the weekend if you could send me fresh link
[5:11pm] sannegrinovero: hardy: sorry
[5:11pm] hardy:
[5:11pm] hardy: back to error handling in this case
[5:11pm] sebersole: sannegrinovero: http://in.relation.to/Bloggers/CallableStatementSupport
[5:11pm] sannegrinovero: hardy: so the ErrorHandler is meant to capture the exceptions happening in the backend threads, as otherwise we have no chance to let the user know of issues (other than logging them) as it's not the user's thread.
[5:12pm] hardy: sannegrinovero: ok, the way you say it kind of makes sense. It's just not what the documentation is saying
[5:12pm] hardy: so at the very least the documentation is too vague and needs to be ways more specific
[5:13pm] sannegrinovero: hardy: the issue is related with the fact that without the ErrorHandler a user bug would kill our internal threads.. critically bad
[5:13pm] hardy: since right now it implies any error would be handled via the Errorhandler
[5:13pm] hardy: i get that
[5:13pm] hardy: but going back to event based indexing
[5:14pm] sannegrinovero: hardy: agreed, the wording is poor. I think the "indexing process" it refers to is strictly referring to the backend contained operations, but that's not clear indeed.
[5:14pm] hardy: right
[5:14pm] hardy: It might make sense to you and me
[5:14pm] hardy: but for a user indexing is indexing
[5:14pm] hardy: he does not thing about technical details and which threads are involved
[5:15pm] sannegrinovero: hardy: so ok to fix the docs. Did you have more in mind for 5.0 ?
[5:15pm] hardy: we need to make this ways more explicit
[5:15pm] hardy: hang on a sec
[5:15pm] hardy: so event based indexing
[5:15pm] hardy: an error occurs during document creation -> rollback
[5:16pm] hardy: an error occurs during LuceneWork processing -> error gets logged, nothing else happened
[5:16pm] sannegrinovero: hardy: funny that you fix the spaces in this message without fixing the message "e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates."
[5:17pm] hardy: so what we are saying here is that if a backend error occurs in the default configuration the user ends up with missing data in the index
[5:17pm] hardy: and the only potential feedback is in the logs?
[5:18pm] sannegrinovero: hardy: "an error occurs during document creation -> rollback" : consider that during "document creation" the most likely failure (Assuming no dumb user implementation bugs) is a failure to read additional needed data from the DB in the transaction.
[5:18pm] hardy: it does not matter what causes the problem
[5:18pm] hardy: could be many things
[5:18pm] hardy: think the TIKA bridge
[5:18pm] hardy: or other custom bridges
[5:19pm] hardy: or in fact some ORM based problem
[5:19pm] hardy: who knows
[5:20pm] hardy: what bugs me thought is that a backend error does not throw an excpetion
[5:21pm] sannegrinovero: sure. but that's technically impossible.
[5:21pm] hardy: and the issue is really that indexing occurs while the transaction has already completed (in a separate thread)
[5:21pm] sannegrinovero: (let's leave TIKA out as I have a problem with how that works in terms of thread owner
[5:22pm] sannegrinovero: well we know that. that's by design and often a good thing.
[5:23pm] sannegrinovero: hardy: if people would need maximum reliability we would need HSEARCH-668
[5:23pm] hardy: sannegrinovero: sure, another improvement to the TIKA processing would be to do the processing in the backend
[5:25pm] hardy: sannegrinovero: at the very least we need to refine the javadocs and online docs
[5:25pm] hardy: it must become clear when an ErrorHandler comes into play
[5:25pm] sannegrinovero: hardy: so right, a backend failure brings the index out of sync. But let's consider when that happens: 1) when the disk is full 2) critical failure in Lucene (index possibly destroyed) . Can we do something about it?
[5:25pm] hardy: and we need to highlight this whole threading issue mrie
[5:27pm] sannegrinovero: hardy: the new executor I have been playing with could be used to refactor the backends to avoid threading altogether, at least when not using the "async" option. Let's play with that in 5.0 as I'm convinved we have too much threading around.
[5:28pm] sannegrinovero: hardy: what I'm wondering now is if you have an implementation change in mind or if you're happy with docs improvements only.
[5:30pm] sannegrinovero: hardy: in case of synch backend we could consider to - even if we fail to remove threads - to capture the backend stack and forward it to the user thread, but that adds up quite some locking costs.
[5:30pm] hardy: sannegrinovero: regarding this particular question doc changes should be fine
[5:30pm] hardy: sannegrinovero: I am not against this option
[5:30pm] hardy: might make snese
[5:30pm] hardy: sense
[5:31pm] sannegrinovero: hardy: just figuring if you had a better idea. Go buy some milk so I have time to review your pull (unless you want to merge mine first)
[5:31pm] hardy: and would make the sync backend more consistent imo
[5:32pm] sannegrinovero: hardy: well yes but then the sync backend would be inconsistent with the async backend..
[5:32pm] hardy: sannegrinovero: not when we follow your whole threading argument
[5:33pm] hardy: sannegrinovero: now regarding the ErrorHandler itself
[5:33pm] sannegrinovero: hardy: while you get the milk you could start thinking about where the Async/TIKA could fit in this
[5:33pm] hardy:
[5:34pm] hardy: I kind of dislike handle and handleException
[5:34pm] hardy: I mean it confuses me on why both are there and when they are needed
[5:34pm] hardy: and really I always want to create a context
[5:35pm] hardy: for my fix I had to use handleException (ErrorContext did not make any sense)
[5:35pm] hardy: so the user ends up with a message and throwable
[5:35pm] hardy: I cold have provided more
[5:36pm] hardy: I was wondering whether it would not be nicer to just have a single handle(ErrorContext context), with potential different subclasses of ErrorContext
[5:38pm] hardy: sannegrinovero: still there
[5:38pm] sannegrinovero: hardy: and force users to use Visitor ?
[5:38pm] hardy: visitor or the unwrap approach
[5:39pm] hardy: if you go along with a single handle method I let you have your Visitor
[5:40pm] hardy: and last but not least ErrorContext#getFailingOperations and ErrorContext#getOperationAtFault
[5:40pm] hardy: this one boggles my mind as well
[5:40pm] hardy: afaikt getOperationAtFault is the primary error
[5:41pm] hardy: I think renaming getOperationAtFault into getPrimaryFailedOperation would already make things much clearer
[5:42pm] sannegrinovero: well there are no secondary failures
[5:43pm] sannegrinovero: the concept is that we have a set of N changes. One of them killed the backend, so all N are lost.
[5:43pm] hardy: so what is it then ErrorContext#getOperationAtFault does compared to ErrorContext#getFailingOperations
[5:43pm] sannegrinovero: getFailingOperations returns a list of all LuceneWork instances which where not applied, because of getOperationAtFault screwed up.
[5:44pm] hardy: their naming implies they are LuceneWork instances with errors, but once I deal with a single instance and once with a list
[5:44pm] hardy: right, let's rename it then
[5:45pm] hardy: getFailingOperations -> getNotAppliedWork()
[5:46pm] hardy: getOperationAtFault -> getExceptionCausingWork (something like that)
[5:47pm] sannegrinovero: hardy: does it really bother you much? I don't see a big usability win (assuming we improve the javadocs of course)
[5:47pm] hardy: with the rename I can make sense out of this without consulting the javadocs
[5:48pm] hardy: but I agree a proper javadocs is a given
[5:48pm] hardy: but preferably I would address both
[5:48pm] hardy: and the usability win is in making it easier for a developer to write a custom ErrorHandler
[5:49pm] hardy: so basically there are three things which bother me in this error handling thing atm
[5:49pm] sannegrinovero: hardy: yes I see your point but the exact name being more/less appropriate is arguable, while a name change is 100% annoying to a user, so unless there is a very good reason we could just keep the status quo as there are other bleeding edges which we could polish with greated benefit for all.
[5:50pm] hardy: #1 The documentation implies that ErrorHandler is always used. We need to make the workings of the ErrorHandler more explicit in relation to backends etc
[5:50pm] sannegrinovero: hardy: also this is an advanced SPI, people are going to be much happier with you dumping the metadata to console.
[5:51pm] sannegrinovero: (that's just an example of other things which are nice to do)
[5:51pm] hardy: #2 I think a single ErrorHandler#handle(ErrorContext) with subclasses for ErrorContext would be benefitial
[5:52pm] hardy: #2a if not, I recommend at least to drop 'Exception' in handleException
[5:52pm] hardy: #3 getFailingOperations and getOperationAtFault in ErrorContext should be renamed to make their intend clearer
[5:53pm] sannegrinovero: hardy: I agree with you that the names you proposed are better. I just am not totally convinved that a name change outweights the drawbacks, especially given a better javadoc. Let's think about it during 5.0 ok?
[5:53pm] hardy: #4 all this with better javadocs (HSEARCH-1353)
[5:54pm] sannegrinovero: hardy: so considering the above issue stays in 4.4 ^, could you record the #3 and #4 on jira please?
[5:54pm] hardy: Sure, it is a 5.0 thing
[5:54pm] hardy: but we might want to add some warnings/deprecations already now
[5:54pm] hardy: and I tend to disagree that is it better to lee the status quo
[5:54pm] sannegrinovero: hardy: ah right nice, that definitely lowers my barriers to name changes.
[5:56pm] sannegrinovero: hardy: I guess that since I'm living the Lucene4 migration pain myself, I can see how people get put off when a library changes too much in one go. There is only a certain amount of complexity that people are willing to face for a version upgrade.
[5:56pm] hardy: sure, but there is no complexity there
[5:56pm] hardy: Lucene 4 is also an architectural change
[5:57pm] hardy: here we are talking mostly about name changes
[5:57pm] hardy: ignoring #2 that is
[5:57pm] sannegrinovero: hardy: by complexity I mean purely a "high number of compile failures": doesn't really matter if most are trivial, it still creates a mental barrier. Not rejecting the proposal though as I see the merit.
[5:58pm] sannegrinovero: hardy: technically I wonder if we should make a transition release, with 5.0 having sill lots of deprecated methods to help people out. Let's see what we can get to do.
[5:58pm] hardy: for me there is a difference in type of compilation error
[5:59pm] hardy: if I just have to rename, it is pretty much done in a minute and if I as developer can also the benefit in this change (aka the API is clearer) I don't mind this change
[5:59pm] sannegrinovero: sure there is. but if some are complex and you can't finish a migration in a day, and still have 100 compile errors before getting sleep.. it doesn't matter if they are easy, you might very well abort.
[6:00pm] hardy: also, we are taking Search 4 to 5
[6:00pm] hardy: for me that is still the one place where we can do this
[6:00pm] sannegrinovero: ok then
[6:00pm] hardy: once 5 is out we have are stuck again
[6:00pm] sannegrinovero: as long as we won't have a long list of such "small changes"
[6:00pm] sannegrinovero: right. almost wish to start quick on 6.0
[6:01pm] hardy:
[6:01pm] sannegrinovero: but nothing is quick as I'd like
[6:01pm] hardy: I know
[6:01pm] hardy: too much to do
[6:01pm] hardy: anyways, these are my thoughts around error handler
[6:02pm] hardy: I thought I dump them on you while fresh in my mind
[6:02pm] hardy: I create an issue and we can take it from there
[6:03pm] sannegrinovero: hardy: right, thanks. Going for a cofee myself to get on with your pull please merge mine?
[6:03pm] hardy: the biggest change would be to have a single ErrorHandler#handle method
[6:03pm] hardy: the other things are documentation and method renames
[6:03pm] hardy: sannegrinovero: ok
[6:11pm] hardy: sannegrinovero: fyi HSEARCH-1375
</pre>
</div>
</div>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 15px 16px; background-color: #fff">
<table id="page-title-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="page-title-pattern-first-line" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; padding-top: 16px"> <a href="https://hibernate.atlassian.net/browse/HSEARCH" style="color: #3b73af; text-decoration: none">Hibernate Search</a> / <a href="https://hibernate.atlassian.net/browse/HSEARCH-1375" style="color: #3b73af; text-decoration: none"><img src="https://hibernate.atlassian.net/images/icons/issuetypes/bug.png" height="16" width="16" border="0" align="absmiddle" alt="Bug" /></a> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1375" style="color: #3b73af; text-decoration: none">HSEARCH-1375</a>
</td>
</tr>
<tr>
<td style="vertical-align: top;; padding: 0px; border-collapse: collapse; padding-right: 5px; font-size: 20px; line-height: 30px; mso-line-height-rule: exactly" id="page-title-pattern-header-container"> <span id="page-title-pattern-header" style="font-family: Arial, sans-serif; padding: 0; font-size: 20px; line-height: 30px; mso-text-raise: 2px; mso-line-height-rule: exactly; vertical-align: middle"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1375" style="color: #3b73af; text-decoration: none">Improve design of ErrorHandler</a> </span>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 15px 16px; background-color: #fff">
<table id="actions-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px">
<tr>
<td id="actions-pattern-container" valign="middle" style="padding: 0px; border-collapse: collapse; padding: 0 0 0 24px; vertical-align: middle; padding-left: 0px">
<table align="left" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td class="actions-pattern-action-icon-container" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 0px; vertical-align: middle"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1375#add-comment" target="_blank" title="{getText($action.text)}" style="color: #3b73af; text-decoration: none"> <img class="actions-pattern-action-icon-image" src="https://hibernate.atlassian.net/images/mail/comment-icon.png" alt="Add Comment" title="{Add Comment}-icon" height="16" width="16" border="0" style="vertical-align: middle" /> </a>
</td>
<td class="actions-pattern-action-text-container" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 4px; padding-left: 5px"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1375#add-comment" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none">Add Comment</a>
</td>
</tr>
</table>
</td>
</tr>
</table>
</td>
</tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<tr>
<td class="email-content-rounded-bottom mobile-expand" style="padding: 0px; border-collapse: collapse; color: #fff; padding: 0 15px 0 16px; height: 5px; line-height: 5px; background-color: #fff; border-top: 0; border-left: 1px solid #ccc; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; border-bottom-right-radius: 5px; border-bottom-left-radius: 5px; mso-line-height-rule: exactly">
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td id="footer-pattern" style="padding: 0px; border-collapse: collapse; padding: 12px 20px">
<table id="footer-pattern-container" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="footer-pattern-text" class="mobile-resize-text" width="100%" style="padding: 0px; border-collapse: collapse; color: #999; font-size: 12px; line-height: 18px; font-family: Arial, sans-serif; mso-line-height-rule: exactly; mso-text-raise: 2px">
This message is automatically generated by JIRA.<br /> If you think it was sent incorrectly, please contact your JIRA administrators<br /> For more information on JIRA, see: <a style="color:#6c797f;; color: #3b73af; text-decoration: none" href="http://www.atlassian.com/software/jira">http://www.atlassian.com/software/jira</a>
</td>
<td id="footer-pattern-logo-desktop-container" valign="top" style="padding: 0px; border-collapse: collapse; padding-left: 20px; vertical-align: top">
<table style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="footer-pattern-logo-desktop-padding" style="padding: 0px; border-collapse: collapse; padding-top: 3px"> <img id="footer-pattern-logo-desktop" src="https://hibernate.atlassian.net/images/mail/footer-desktop-logo.png" alt="JIRA logo" title="JIRA logo" width="92" height="36" class="image_fix" />
</td>
</tr>
</table>
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>