<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0">
<base href="https://hibernate.atlassian.net">
<title>Message Title</title>
</head>
<body class="jira" style="color: #333333; font-family: Arial, sans-serif; font-size: 14px; line-height: 1.429">
<table id="background-table" cellpadding="0" cellspacing="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0; background-color: #f5f5f5; border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0" bgcolor="#f5f5f5">
<!-- header here -->
<tbody>
<tr>
<td id="header-pattern-container" style="padding: 0; border-collapse: collapse; padding: 10px 20px">
<table id="header-pattern" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0">
<tbody>
<tr>
<td id="header-avatar-image-container" valign="top" style="padding: 0; border-collapse: collapse; vertical-align: top; width: 32px; padding-right: 8px" width="32"> <img id="header-avatar-image" class="image_fix" src="https://avatar-cdn.atlassian.com/2a8bdd4ffd282b7185c74b52ab452617?s=48&d=https%3A%2F%2Fsecure.gravatar.com%2Favatar%2F2a8bdd4ffd282b7185c74b52ab452617%3Fd%3Dmm%26s%3D48%26noRedirect%3Dtrue" height="32" width="32" border="0" style="border-radius: 3px; vertical-align: top"> </td>
<td id="header-text-container" valign="middle" style="padding: 0; border-collapse: collapse; vertical-align: middle; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px"> <a class="user-hover" rel="yrodiere" id="email_yrodiere" href="https://hibernate.atlassian.net/secure/ViewProfile.jspa?name=yrodiere" style="color:#6c797f;; color: #3b73af; text-decoration: none">Yoann Rodière</a> <strong>commented</strong> on <a href="https://hibernate.atlassian.net/browse/HSEARCH-3390?atlOrigin=eyJpIjoiZDkwNDhmMjc2OTNlNDBmMjljMmE3OTA0Y2MxMTZiYzEiLCJwIjoiaiJ9" style="color: #3b73af; text-decoration: none"><img src="cid:jira-generated-image-avatar-820594a4-e1c0-405d-bb37-14082ab2ea37" height="16" width="16" border="0" align="absmiddle" alt="New Feature"> HSEARCH-3390</a> </td>
</tr>
</tbody>
</table> </td>
</tr>
<tr>
<td id="email-content-container" style="padding: 0; border-collapse: collapse; padding: 0 20px">
<table id="email-content-table" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0; border-spacing: 0; border-collapse: separate">
<tbody>
<tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<td class="email-content-rounded-top mobile-expand" style="padding: 0; border-collapse: collapse; color: #ffffff; padding: 0 15px 0 16px; height: 15px; background-color: #ffffff; border-left: 1px solid #cccccc; border-top: 1px solid #cccccc; border-right: 1px solid #cccccc; border-bottom: 0; border-top-right-radius: 5px; border-top-left-radius: 5px; height: 10px; line-height: 10px; padding: 0 15px 0 16px; mso-line-height-rule: exactly" height="10" bgcolor="#ffffff"> </td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0; border-collapse: collapse; border-left: 1px solid #cccccc; border-right: 1px solid #cccccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #ffffff" bgcolor="#ffffff">
<table class="page-title-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0">
<tbody>
<tr>
<td style="vertical-align: top;; padding: 0; border-collapse: collapse; padding-right: 5px; font-size: 20px; line-height: 30px; mso-line-height-rule: exactly" class="page-title-pattern-header-container"> <span class="page-title-pattern-header" style="font-family: Arial, sans-serif; padding: 0; font-size: 20px; line-height: 30px; mso-text-raise: 2px; mso-line-height-rule: exactly; vertical-align: middle"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-3390?atlOrigin=eyJpIjoiZDkwNDhmMjc2OTNlNDBmMjljMmE3OTA0Y2MxMTZiYzEiLCJwIjoiaiJ9" style="color: #3b73af; text-decoration: none">Re: Composite projections</a> </span> </td>
</tr>
</tbody>
</table> </td>
</tr>
<tr>
<td id="text-paragraph-pattern-top" class="email-content-main mobile-expand comment-top-pattern" style="padding: 0; border-collapse: collapse; border-left: 1px solid #cccccc; border-right: 1px solid #cccccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #ffffff; border-bottom: none; padding-bottom: 0" bgcolor="#ffffff">
<table class="text-paragraph-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 2px">
<tbody>
<tr>
<td class="text-paragraph-pattern-container mobile-resize-text " style="padding: 0; border-collapse: collapse; padding: 0 0 10px"> <p style="margin: 10px 0 0; margin-top: 0"><a href="https://hibernate.atlassian.net/secure/ViewProfile.jspa?name=gsmet" class="user-hover" rel="gsmet" style="color: #3b73af; text-decoration: none">Guillaume Smet</a> Some thoughts I had this weekend over implementing this. I hope it helps. Let's discuss that when you start?</p> <p style="margin: 10px 0 0">Hit collector cannot work for composite projections, we need to change them. The main problem being that hit collectors assume a "flat" structure for hits, but composite projections will obviously allow tree-like structures that don't fit well in this model.</p> <p style="margin: 10px 0 0">So, we need something else.</p> <p style="margin: 10px 0 0">I think one thing that could work would be to move the responsibility of "collecting" hit elements out of the engine, and leave it to the backend (which knows what each projection is supposed to do).</p> <p style="margin: 10px 0 0">The idea would be that:</p>
<ol>
<li>the aggregator just offers a "add(T)" method (T being the type of the reference, loaded object, projection, ...) instead of a "nextCollector" method.</li>
<li>the projections do not add their result to a projection, but return it.</li>
<li>the hit-scoped hit collector disappears in favor of a query-scoped "hit mapper". The "hit mapper" will not collect the hits, but instead it will allow to:</li>
</ol>
<ul>
<li>convert a reference to the type expected by the mapper (e.g. DocumentReference => PojoReference) using its <tt>Object convertReference(DocumentReference)</tt> method.</li>
<li>inform the mapper that an object will have to be loaded through its <tt>Object planLoading(DocumentReference)</tt> method, which returns a key for later retrieval of the loaded object.</li>
<li>trigger loading of objects through its <tt>LoadResult load()</tt> method that returns another SPI, which exposes a <tt>Object getLoaded(Object key)</tt> method.</li>
</ul> <p style="margin: 10px 0 0">This should allow to work with (potentially nested) composite projections more easily.</p> <p style="margin: 10px 0 0">However, the hit collector is needed for several reasons, so we'll have to remove it carefully.</p> <p style="margin: 10px 0 0">Here are the three main changes I think will be necessary in order to even begin to implement composite projections. Unfortunately we'll probably have to implement them all in the same commit, as they are somewhat intertwined.</p> <p style="margin: 10px 0 0">First, the HitCollector is needed because it abstracts over the number of elements per hit: some projections only return one element per hit, some others return multiple elements per hit (a list).<br> We can work around that by removing <tt>asProjections</tt> (plural) from SPIs and using composite projections instead wherever it was needed in the ORM/JavaBean mappers (namely in <tt>org.hibernate.search.mapper.javabean.search.dsl.query.impl.JavaBeanQueryResultDefinitionContextImpl#asProjections</tt> and in <tt>org.hibernate.search.mapper.orm.search.impl.FullTextQueryResultDefinitionContextImpl#asProjections</tt>).</p> <p style="margin: 10px 0 0">Second, the HitCollector is needed because we have multiple different methods of result collection: <tt>asReference</tt>, <tt>asObject</tt>, ... and thus multiple types of collectors. We can work around that by having multiple types of <tt>HitMapper</tt> just as we had multiple types of <tt>HitCollector</tt>: <tt>ReferenceHitCollector</tt> will become <tt>ReferenceHitMapper</tt> and will only expose <tt>convertReference()</tt>, <tt>LoadingHitCollector</tt> will become <tt>LoadingHitMapper</tt> and will only expose <tt>planLoading()</tt> and <tt>load()</tt>, and <tt>ProjectionHitCollector</tt> will become <tt>ProjectionHitMapper</tt> and will extend both <tt>ReferenceHitMapper</tt> and <tt>LoadingHitMapper</tt>. I would recommend implementing the various "mappers" in the engine for now, probably instantiating them in <tt>org.hibernate.search.engine.common.impl.MappedIndexSearchTargetImpl#queryAsProjection</tt> et al.</p> <p style="margin: 10px 0 0">Third, the <tt>HitCollector</tt> is needed to, well, collect hits. Without it, we need to change the implementation of projections to return the hits differently. Thus we'll have to make the projections return their result from their <tt>extract</tt> method. However, it's not that simple, as generating the results of a full-text query is a multiple-step process: first we extract everything from the query, then we load whatever we need from the database, then we insert what was loaded from the database into the extracted results.<br> I think we'll have to make projection expose two methods:</p>
<ul>
<li><tt>Object extract(ProjectionHitMapper, <backene specific parameters>)</tt> extracts data from the hits, plans loading and returns a temporary extracted result.</li>
<li><tt>T transform(Object extractedData, LoadingResult loadingResult)</tt>, gets passed the extracted data returned by <tt>extract</tt>, and turns the temporary result into the type of the projection. Most of the time it just casts and returns the extracted data, but it can also consider the extracted data as a loading key and use the LoadingResult to return a loaded object. It can also (and that'll be useful for composite projections) apply a function to the extracted data.</li>
</ul> <p style="margin: 10px 0 0">The two methods will be called in two passes: first call <tt>extract()</tt> for every single hit, then call <tt>LoadingHitMapper#load()</tt>, then call <tt>transform()</tt> for every single hit.</p> <p style="margin: 10px 0 0">Miscellaneous notes:</p> <p style="margin: 10px 0 0">1. Initially implement composite projections without the transforming function, it can be added later<br> 2. We may want to take this opportunity to simplify the architecture by removing the <tt>asObject</tt> method from SPIs and considering mappers should simply use projections for that. But we can't really do it for <tt>asReference</tt>, because that would force mappers to provide an <tt>ObjectLoader</tt> even when using <tt>asReference</tt> and not needing any object loading.<br> 3. Important: make sure that we can add support for more parameters (type-safe transforming function) without breaking APIs. One way to do that would be to give a different name to the DSL method that create a composite projection from a vararg of projections, e.g. <tt>compositeList()</tt> vs <tt>composite()</tt>, or <tt>composite()</tt> vs. <tt>compositeTransformed()</tt>, or <tt>compositeList()</tt> vs. <tt>compositeTransformed()</tt>. Another way would be to use the same name, but not allow a function in that same method. </p> </td>
</tr>
</tbody>
</table> </td>
</tr>
<tr>
<td class="email-content-main mobile-expand " style="padding: 0; border-collapse: collapse; border-left: 1px solid #cccccc; border-right: 1px solid #cccccc; border-top: 0; border-bottom: 0; padding: 0 15px 0 16px; background-color: #ffffff" bgcolor="#ffffff"> <script type="application/ld+json">
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"description": "View Issue",
"potentialAction": {
"@type": "ViewAction",
"target": "https://hibernate.atlassian.net/browse/HSEARCH-3390?inbox=true&focusedCommentId=104044&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-104044",
"name": "View Comment"
},
"publisher": {
"@type": "Organization",
"name": "Atlassian",
"url": "https://www.atlassian.com"
}
}
</script>
<table id="actions-pattern" cellspacing="0" cellpadding="0" border="0" width="100%" style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 1px">
<tbody>
<tr>
<td id="actions-pattern-container" valign="middle" style="padding: 0; border-collapse: collapse; padding: 10px 0 10px 24px; vertical-align: middle; padding-left: 0">
<table align="left" style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0">
<tbody>
<tr>
<td class="actions-pattern-action-icon-container" style="padding: 0; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 0; vertical-align: middle"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-3390#add-comment?atlOrigin=eyJpIjoiZDkwNDhmMjc2OTNlNDBmMjljMmE3OTA0Y2MxMTZiYzEiLCJwIjoiaiJ9" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none"> <img class="actions-pattern-action-icon-image" src="cid:jira-generated-image-static-comment-icon-2204a17c-803f-4953-878d-28d9f39b8e52" alt="Add Comment" title="Add Comment" height="16" width="16" border="0" style="vertical-align: middle"> </a> </td>
<td class="actions-pattern-action-text-container" style="padding: 0; border-collapse: collapse; font-family: Arial, sans-serif; font-size: 14px; line-height: 20px; mso-line-height-rule: exactly; mso-text-raise: 4px; padding-left: 5px"> <a href="https://hibernate.atlassian.net/browse/HSEARCH-3390#add-comment?atlOrigin=eyJpIjoiZDkwNDhmMjc2OTNlNDBmMjljMmE3OTA0Y2MxMTZiYzEiLCJwIjoiaiJ9" target="_blank" title="Add Comment" style="color: #3b73af; text-decoration: none">Add Comment</a> </td>
</tr>
</tbody>
</table> </td>
</tr>
</tbody>
</table> </td>
</tr>
<!-- there needs to be content in the cell for it to render in some clients -->
<tr>
<td class="email-content-rounded-bottom mobile-expand" style="padding: 0; border-collapse: collapse; color: #ffffff; padding: 0 15px 0 16px; height: 5px; line-height: 5px; background-color: #ffffff; border-top: 0; border-left: 1px solid #cccccc; border-bottom: 1px solid #cccccc; border-right: 1px solid #cccccc; border-bottom-right-radius: 5px; border-bottom-left-radius: 5px; mso-line-height-rule: exactly" height="5" bgcolor="#ffffff"> </td>
</tr>
</tbody>
</table> </td>
</tr>
<tr>
<td id="footer-pattern" style="padding: 0; border-collapse: collapse; padding: 12px 20px">
<table id="footer-pattern-container" cellspacing="0" cellpadding="0" border="0" style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0">
<tbody>
<tr>
<td id="footer-pattern-mobile-separated-links" class="mobile-resize-text" width="100%" colspan="2" style="padding: 0; border-collapse: collapse; color: #999999; font-size: 12px; line-height: 18px; font-family: Arial, sans-serif; mso-line-height-rule: exactly; mso-text-raise: 2px"> Get Jira notifications on your phone! Download the Jira Cloud app for <a href="https://play.google.com/store/apps/details?id=com.atlassian.android.jira.core&referrer=utm_source%3DNotificationLink%26utm_medium%3DEmail" style="color: #3b73af; text-decoration: none">Android</a> or <a href="https://itunes.apple.com/app/apple-store/id1006972087?pt=696495&ct=EmailNotificationLink&mt=8" style="color: #3b73af; text-decoration: none">iOS</a>
<hr> </td>
</tr>
<tr>
<td id="footer-pattern-text" class="mobile-resize-text" width="100%" style="padding: 0; border-collapse: collapse; color: #999999; font-size: 12px; line-height: 18px; font-family: Arial, sans-serif; mso-line-height-rule: exactly; mso-text-raise: 2px"> This message was sent by Atlassian Jira <span id="footer-build-information">(v1001.0.0-SNAPSHOT#100095-<span title="93136a0b3a14b2baeaaff8f04475fb5e2e1ebc62" data-commit-id="93136a0b3a14b2baeaaff8f04475fb5e2e1ebc62}">sha1:93136a0</span>)</span> </td>
<td id="footer-pattern-logo-desktop-container" valign="top" style="padding: 0; border-collapse: collapse; padding-left: 20px; vertical-align: top">
<table style="border-collapse: collapse; mso-table-lspace: 0; mso-table-rspace: 0">
<tbody>
<tr>
<td id="footer-pattern-logo-desktop-padding" style="padding: 0; border-collapse: collapse; padding-top: 3px; opacity: 0.150"> <img id="footer-pattern-logo-desktop" src="cid:jira-generated-image-static-footer-desktop-logo-bf7bfcb7-1b6d-44fa-bcb9-e5b8d890594c" alt="Atlassian logo" title="Atlassian logo" width="192" height="24" class="image_fix"> </td>
</tr>
</tbody>
</table> </td>
</tr>
</tbody>
</table> </td>
</tr>
</tbody>
</table>
<img border="0" width="1" height="1" alt="" src="http://atlassian.et.e.sparkpost.com/q/el9tI0ctYIVmc1AikUzQLQ~~/AAAAAQA~/RgRdymxIPlcLYXRsYXNzaWFudXNCCgAAyDjpW0FnbmFSIGhpYmVybmF0ZS1pc3N1ZXNAbGlzdHMuamJvc3Mub3JnWAQAAAAE">
</body>
</html>