<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0" /> <base href="https://hibernate.atlassian.net" />
<title>Message Title</title>
</head>
<body class="jira" style="color: #333; font-family: Arial, sans-serif; font-size: 14px; line-height: 1.429">
<table id="background-table" cellpadding="0" cellspacing="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; background-color: #f5f5f5; border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<!-- header here -->
<tr>
<td id="header-pattern-container" style="padding: 0px; border-collapse: collapse; padding: 10px 20px">
<table id="header-pattern" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="header-avatar-image-container" valign="top" style="padding: 0px; border-collapse: collapse; vertical-align: top; width: 32px; padding-right: 8px"> <img id="header-avatar-image" class="image_fix" src="https://secure.gravatar.com/avatar/57637a2eb871b34eba14e700c78c6a5d?d=mm&s=48" height="32" width="32" border="0" style="border-radius: 3px; vertical-align: top" />
</td>
<td id="header-text-container" valign="middle" style="padding: 0px; border-collapse: collapse; vertical-align: middle; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px"> <a class="user-hover" rel="sanne" id="email_sanne" href="https://hibernate.atlassian.net/secure/ViewProfile.jspa?name=sanne" style="color:#6c797f;; color: #3b73af; text-decoration: none">Sanne Grinovero</a> <strong>commented</strong> on <a href="https://hibernate.atlassian.net/browse/HSEARCH-1792" style="color: #3b73af; text-decoration: none"><img src="cid:jira-generated-image-static-newfeature-1e468a92-0b32-4599-8cf4-f6e0ddaa7889" height="16" width="16" border="0" align="absmiddle" alt="New Feature" /> HSEARCH-1792</a>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td id="email-content-container" style="padding: 0px; border-collapse: collapse; padding: 0 20px">
<table id="email-content-table" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; border-spacing: 0; border-collapse: separate">
<tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<td class="email-content-rounded-top mobile-expand" style="padding: 0px; border-collapse: collapse; color: #fff; padding: 0 15px 0 16px; height: 15px; background-color: #fff; border-left: 1px solid #ccc; border-top: 1px solid #ccc; border-right: 1px solid #ccc; border-bottom: 0; border-top-right-radius: 5px; border-top-left-radius: 5px; height: 10px; line-height: 10px; padding: 0 15px 0 16px; mso-line-height-rule: exactly">
</td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #fff">
<table class="page-title-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td style="vertical-align: top;; padding: 0px; border-collapse: collapse; padding-right: 5px; font-size: 20px; line-height: 30px; mso-line-height-rule: exactly" class="page-title-pattern-header-container"> <span class="page-title-pattern-header" style="font-family: Arial, sans-serif; padding: 0; font-size: 20px; line-height: 30px; mso-text-raise: 2px; mso-line-height-rule: exactly; vertical-align: middle"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1792" style="color: #3b73af; text-decoration: none">Re: Support multi-tenancy</a> </span>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td id="text-paragraph-pattern-top" class="email-content-main mobile-expand comment-top-pattern" style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #fff; border-bottom: none; padding-bottom: 0">
<table class="text-paragraph-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 2px">
<tr>
<td class="text-paragraph-pattern-container mobile-resize-text " style="padding: 0px; border-collapse: collapse; padding: 0 0 10px 0">
<blockquote style="margin: 10px 0 0 0; border-left: 1px solid #ccc; color: #707070; margin-left: 19px; padding: 10px 20px">
<p style="margin: 10px 0 0 0">I wonder, what is the consideration not put the "multi-tenancy" feature in Hibernate Search core until now?</p>
</blockquote>
<p style="margin: 10px 0 0 0">Priorities. Nobody seemed too interested in the combination "Search + multitenancy". But I agree it should be done, not just for you but as you say it's probably a fair expectation since the Hibernate ORM core project does support it, so happy to fix the mismatch.. it has been bothering me for some time and your use case description is very valuable for us to make sure we implement it "the right way".</p>
<blockquote style="margin: 10px 0 0 0; border-left: 1px solid #ccc; color: #707070; margin-left: 19px; padding: 10px 20px">
<p style="margin: 10px 0 0 0">How many works need to be done to implement this multi-tenancy feature in Hibernate Search? Could it make to the 5.next release?</p>
</blockquote>
<p style="margin: 10px 0 0 0">The 5.next release is very soon, but don't worry there will be another "next" after that <img class="emoticon" src="https://hibernate.atlassian.net/images/icons/emoticons/smile.gif" height="16" width="16" align="absmiddle" alt="" border="0" /><br /> We probably need to clarify what exactly would be needed; to address this issue I'd expect it to work "transparently" in some way which needs minimal intervention from the user (the application developer), for example we should automatically filter on the current Session.<br /> Still, if you're prepared to enable such a Filter yourself you don't necessarily need that level of integration.</p>
<p style="margin: 10px 0 0 0">Since you mention "millions" of tenants, I guess it would not be wise to have millions of indexes. Technically it could work, but the way Hibernate Search is structured today, for each index there is a managing thread, which keeps file handles open quite aggressively for performance reasons. That wouldn't scale to millions of IndexManager instances, your case would need some kind of "sleep" stand for indexes which are not being used.</p>
<p style="margin: 10px 0 0 0">Such a "sleep" mode for non-active indexes would never be very efficient, so I think the best solution for your use case would be to add a marker keyword in each indexed entry, and combine that with a filter which only matches the currently active marker.</p>
<p style="margin: 10px 0 0 0">Would that work for you? Technically there is nothing stopping you to get going with such an approach - I think - other than <a href="https://hibernate.atlassian.net/browse/HSEARCH-1649" title="MassIndexer not compatible with ORM multitenancy" class="issue-link" data-issue-key="HSEARCH-1649" style="color: #3b73af; text-decoration: none">HSEARCH-1649</a> which should be merged shortly.</p>
<blockquote style="margin: 10px 0 0 0; border-left: 1px solid #ccc; color: #707070; margin-left: 19px; padding: 10px 20px">
<p style="margin: 10px 0 0 0">1. Hibernate search needs to write indexes of requested tenant data only and store it in different locations (not mixed files for all tenants)</p>
</blockquote>
<p style="margin: 10px 0 0 0">Sorry I'm not sure if that's a question or a requirement you have. From a technical perspective, as suggested above I think you'd prefer to have a single consolidated index to avoid having too many backends active at once.</p>
<blockquote style="margin: 10px 0 0 0; border-left: 1px solid #ccc; color: #707070; margin-left: 19px; padding: 10px 20px">
<p style="margin: 10px 0 0 0">2. Hibernate search needs to search in a subset data of the requested tenant only dynamically (no need to define all active tenants first)</p>
</blockquote>
<p style="margin: 10px 0 0 0">That should be possible via the dynamic sharding feature. The initial loading defined by <tt>org.hibernate.search.store.ShardIdentifierProvider#getAllShardIdentifiers()</tt> is meant to satisfy a global query, so that it can search in "all shards". But since you'd never want to do that, your custom implementation would be good to go by returning an empty set: you'd not initialize any shard until there is need for, for example via <tt>org.hibernate.search.store.ShardIdentifierProvider.getShardIdentifiersForQuery(FullTextFilterImplementor[])</tt>.</p>
<blockquote style="margin: 10px 0 0 0; border-left: 1px solid #ccc; color: #707070; margin-left: 19px; padding: 10px 20px">
<p style="margin: 10px 0 0 0">3. Hibernate search needs a way to release all resources linked to the requested tenant (example: opened directory, cached objects, etc.). This API will called manually by application code (example: each time the tenant signed off), and automatically </p>
</blockquote>
<p style="margin: 10px 0 0 0">Right, in its default configuration an open index will stay open until shutdown. You can however reconfigure the ReaderProvider and the IndexWriter strategies to release and close files ASAP; that's possible. What is currently not possible is to fully shutdown an indexing backend, so you'd have an idle thread for each unused index.</p>
<p style="margin: 10px 0 0 0">To summarize there are two possible approaches:</p>
<ul class="alternate" type="square">
<li>
if you're ok to have your tenants to share an index, you should be able to implement this already.
</li>
<li>
if that's not acceptable, considering the scale you mention, we'd probably want to change the "one thread per backend" strategy to a limited pool. This would be slightly worse for the general purpose but would scale with tenants, so we could make it a configuration option.
</li>
</ul>
<p style="margin: 10px 0 0 0">For the second option, which requires some changes, I won't be able to help much in the next 2 weeks as I have various other commitments and travel. I would suggest trying the first approach <img class="emoticon" src="https://hibernate.atlassian.net/images/icons/emoticons/wink.gif" height="16" width="16" align="absmiddle" alt="" border="0" /></p>
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0px; border-collapse: collapse; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #fff">
<table id="actions-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px">
<tr>
<td id="actions-pattern-container" valign="middle" style="padding: 0px; border-collapse: collapse; padding: 10px 0 10px 24px; vertical-align: middle; padding-left: 0">
<table align="left" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td class="actions-pattern-action-icon-container" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 0px; vertical-align: middle"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1792#add-comment" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none"> <img class="actions-pattern-action-icon-image" src="cid:jira-generated-image-static-comment-icon-98752e27-3a8d-4804-9493-a4580082eb1a" alt="Add Comment" title="Add Comment" height="16" width="16" border="0" style="vertical-align: middle" /> </a>
</td>
<td class="actions-pattern-action-text-container" style="padding: 0px; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 4px; padding-left: 5px"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-1792#add-comment" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none">Add Comment</a>
</td>
</tr>
</table>
</td>
</tr>
</table>
</td>
</tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<tr>
<td class="email-content-rounded-bottom mobile-expand" style="padding: 0px; border-collapse: collapse; color: #fff; padding: 0 15px 0 16px; height: 5px; line-height: 5px; background-color: #fff; border-top: 0; border-left: 1px solid #ccc; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; border-bottom-right-radius: 5px; border-bottom-left-radius: 5px; mso-line-height-rule: exactly">
</td>
</tr>
</table>
</td>
</tr>
<tr>
<td id="footer-pattern" style="padding: 0px; border-collapse: collapse; padding: 12px 20px">
<table id="footer-pattern-container" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="footer-pattern-text" class="mobile-resize-text" width="100%" style="padding: 0px; border-collapse: collapse; color: #999; font-size: 12px; line-height: 18px; font-family: Arial, sans-serif; mso-line-height-rule: exactly; mso-text-raise: 2px">
This message was sent by Atlassian JIRA <span id="footer-build-information">(v6.4-OD-13-026#64011-<span title="fa13a3e6fa1dbfa1a9bf18a4be4bb763b15a8ef5" data-commit-id="fa13a3e6fa1dbfa1a9bf18a4be4bb763b15a8ef5}">sha1:fa13a3e</span>)</span>
</td>
<td id="footer-pattern-logo-desktop-container" valign="top" style="padding: 0px; border-collapse: collapse; padding-left: 20px; vertical-align: top">
<table style="border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt">
<tr>
<td id="footer-pattern-logo-desktop-padding" style="padding: 0px; border-collapse: collapse; padding-top: 3px"> <img id="footer-pattern-logo-desktop" src="cid:jira-generated-image-static-footer-desktop-logo-c2685515-6446-4799-b9b0-a01c8043d820" alt="Atlassian logo" title="Atlassian logo" width="169" height="36" class="image_fix" />
</td>
</tr>
</table>
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>