<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0" /> <base href="https://hibernate.atlassian.net" />
<title>Message Title</title>
</head>
<body class="jira" style="color: #333; font-family: Arial, sans-serif; font-size: 14px; line-height: 1.429">
<table id="background-table" cellpadding="0" cellspacing="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; background-color: #f5f5f5; border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<!-- header here -->
<tr>
<td id="header-pattern-container" style="padding: 0px; border-collapse: collapse; padding: 10px 20px">
<table id="header-pattern" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="header-avatar-image-container" valign="top" style="padding: 0px; border-collapse: collapse; vertical-align: top; width: 32px; padding-right: 8px"> <img id="header-avatar-image" class="image_fix" src="https://secure.gravatar.com/avatar/57637a2eb871b34eba14e700c78c6a5d?d=mm&s=48" height="32" width="32" border="0" style="border-radius: 3px; vertical-align: top" />
</td>
<td id="header-text-container" valign="middle" style="padding: 0px; border-collapse: collapse; vertical-align: middle; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px"> <a class="user-hover" rel="sanne" id="email_sanne" href="https://hibernate.atlassian.net/secure/ViewProfile.jspa?name=sanne" style="color:#6c797f;; color: #3b73af; text-decoration: none">Sanne Grinovero</a> <strong>commented</strong> on <a href="https://hibernate.atlassian.net/browse/HSEARCH-1792" style="color: #3b73af; text-decoration: none"><img src="cid:jira-generated-image-static-newfeature-31d445f5-4a1c-4702-b977-59709d5e4064" height="16" width="16" border="0" align="absmiddle" alt="New Feature" /> HSEARCH-1792</a>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td id="email-content-container" style="padding: 0px; border-collapse: collapse; padding: 0 20px">
<table id="email-content-table" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; border-spacing: 0; border-collapse: separate">
<tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<td class="email-content-rounded-top mobile-expand" style="padding: 0px; border-collapse: collapse; color: #fff; padding: 0 15px 0 16px; height: 15px; background-color: #fff; border-left: 1px solid #ccc; border-top: 1px solid #ccc; border-right: 1px solid #ccc; border-bottom: 0; border-top-right-radius: 5px; border-top-left-radius: 5px; height: 10px; line-height: 10px; padding: 0 15px 0 16px; mso-line-height-rule: exactly">
</td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #fff">
<table class="page-title-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td style="vertical-align: top;; padding: 0px; border-collapse: collapse; padding-right: 5px; font-size: 20px; line-height: 30px; mso-line-height-rule: exactly" class="page-title-pattern-header-container"> <span class="page-title-pattern-header" style="font-family: Arial, sans-serif; padding: 0; font-size: 20px; line-height: 30px; mso-text-raise: 2px; mso-line-height-rule: exactly; vertical-align: middle"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1792" style="color: #3b73af; text-decoration: none">Re: Support multi-tenancy</a> </span>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td id="text-paragraph-pattern-top" class="email-content-main mobile-expand comment-top-pattern" style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #fff; border-bottom: none; padding-bottom: 0">
<table class="text-paragraph-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 2px">
<tr>
<td class="text-paragraph-pattern-container mobile-resize-text " style="padding: 0px; border-collapse: collapse; padding: 0 0 10px 0">
<p style="margin: 10px 0 0 0">Hi <a href="https://hibernate.atlassian.net/secure/ViewProfile.jspa?name=dunx" class="user-hover" rel="dunx" style="color: #3b73af; text-decoration: none">Yoseph Stephen</a>, let me give you some insight on the internals, as you made some very valid comments but the proposed custom <tt>DirectoryProvider</tt> won't work.</p>
<p style="margin: 10px 0 0 0">There are two main problems:</p>
<ul class="alternate" type="square">
<li>
all backend operations (like writing changes to the <tt>Directory</tt>) will happen in a different thread, so <tt>ThreadLocal</tt> isn't an option.
</li>
<li>
depending on backend configuration, the lifecycle of an <tt>IndexWriter</tt> might batch multiple writes, but will invoke <tt>directoryProvider.getDirectory()</tt> only once.
</li>
</ul>
<p style="margin: 10px 0 0 0">So while I agree we should work on a better solution, that is not the right path.</p>
<p style="margin: 10px 0 0 0">The <tt>ShardIdentifierProvider</tt> based solution however will work, and has some nice consequences. First off, Lucene's scoring formulas include factors like how frequent each term/keyword is across the whole index, so having a separate index per tenant seems a property you might want to have, to keep the scores fully independent across tenants.<br /> I wrote that documentation section about sharding, and what I meant primarily is that having too many shards makes query less efficient - as people normally would then need to aggregate results from multiple shards (but you wouldn't do that in this case). Also, each shard would have its own Directory instance, which would keep some file handles open, so having too many Directory instances might exhaust the total number of file handles available to your process/system. On Linux servers you can reconfigure it to get your more file handles, so it's not a blocking issue but makes things a bit more complex. And obviously file handles take a bit of memory too, so it would be good to avoid them when not strongly needed.</p>
<p style="margin: 10px 0 0 0">But your proposal would also require a <tt>Directory</tt> instance per tenant, so you'd have the same drawbacks.</p>
<p style="margin: 10px 0 0 0">One more limitation of having a per-tenant sharding strategy, is that you wouldn't be able to do additional sharding in each tenant.. but considering we'd already have a large amount of {{Directory} instances I wouldn't recommend that so I think that's an acceptable limitation.</p>
<p style="margin: 10px 0 0 0">Regarding the `tenantId` token: you're right, and I wasn't suggesting to do that. I was more wondering how you were handling the separation today, and assuming that you're using a `tenantId` token.<br /> Note that with a <tt>Filter</tt> the drawback wouldn't be too bad, as the filter instances are cacheable and you'd end up searching only in the subset of the index which relates to your tenant.. but these filters take memory and caching by LRU implies that sometimes you wouldn't have a cache hit, and at each write on the index these cached filters would be invalidated.</p>
<p style="margin: 10px 0 0 0">Note that if you were to experiment with the custom sharding strategy, you'd have the option to narrow down query execution to the specific shard only, using this special "filter" definition:<br /> <a href="http://docs.jboss.org/hibernate/search/5.0/reference/en-US/html_single/#query-filter-shard" class="external-link" rel="nofollow" style="color: #3b73af; text-decoration: none">http://docs.jboss.org/hibernate/search/5.0/reference/en-US/html_single/#query-filter-shard</a></p>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #fff">
<table id="actions-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px">
<tr>
<td id="actions-pattern-container" valign="middle" style="padding: 0px; border-collapse: collapse; padding: 10px 0 10px 24px; vertical-align: middle; padding-left: 0">
<table align="left" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td class="actions-pattern-action-icon-container" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 0px; vertical-align: middle"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1792#add-comment" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none"> <img class="actions-pattern-action-icon-image" src="cid:jira-generated-image-static-comment-icon-05d9ef99-4171-4bb9-ae29-d8023ac6aff6" alt="Add Comment" title="Add Comment" height="16" width="16" border="0" style="vertical-align: middle" /> </a>
</td>
<td class="actions-pattern-action-text-container" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 4px; padding-left: 5px"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1792#add-comment" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none">Add Comment</a>
</td>
</tr>
</table>
</td>
</tr>
</table>
</td>
</tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<tr>
<td class="email-content-rounded-bottom mobile-expand" style="padding: 0px; border-collapse: collapse; color: #fff; padding: 0 15px 0 16px; height: 5px; line-height: 5px; background-color: #fff; border-top: 0; border-left: 1px solid #ccc; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; border-bottom-right-radius: 5px; border-bottom-left-radius: 5px; mso-line-height-rule: exactly">
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td id="footer-pattern" style="padding: 0px; border-collapse: collapse; padding: 12px 20px">
<table id="footer-pattern-container" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="footer-pattern-text" class="mobile-resize-text" width="100%" style="padding: 0px; border-collapse: collapse; color: #999; font-size: 12px; line-height: 18px; font-family: Arial, sans-serif; mso-line-height-rule: exactly; mso-text-raise: 2px">
This message was sent by Atlassian JIRA <span id="footer-build-information">(v6.4-OD-13-026#64011-<span title="fa13a3e6fa1dbfa1a9bf18a4be4bb763b15a8ef5" data-commit-id="fa13a3e6fa1dbfa1a9bf18a4be4bb763b15a8ef5}">sha1:fa13a3e</span>)</span>
</td>
<td id="footer-pattern-logo-desktop-container" valign="top" style="padding: 0px; border-collapse: collapse; padding-left: 20px; vertical-align: top">
<table style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="footer-pattern-logo-desktop-padding" style="padding: 0px; border-collapse: collapse; padding-top: 3px"> <img id="footer-pattern-logo-desktop" src="cid:jira-generated-image-static-footer-desktop-logo-375b8b85-ca96-40d1-b989-be691a74422a" alt="Atlassian logo" title="Atlassian logo" width="169" height="36" class="image_fix" />
</td>
</tr>
</table>
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>