>> What is DynamicBoost?<br>
<pre><br>I use it for computing boost at runtime - not just a static boost number.<br>Since my boost can depend on current state of different resources.<br><br>For example - I need my articles that are new, pushed higher than the
<br>old one's.<br><br>public @interface DynamicBoost {<br><br> float coef() default 1.0f;<br><br> String transformer() default "number";<br><br> String limit() default "identity";<br><br> String formula() default "quotient";
<br><br> String query() default "";<br><br> boolean isHQL() default true;<br><br>}<br><br> public Float getBoost(Object value, DynamicBoost db) {<br> Transformer t = transformers.get(db.transformer());
<br> double fd = t.toDouble(value);<br> Limitizer limitizer = limitizers.get(db.limit());<br> double limit = limitizer.limit(value, db, t);<br> BoostFormula formula = formulas.get(db.formula());<br>
return formula.calculate(db, fd, limit);<br> }<br><br>>> I fixed the concurrency issues in the Lucene event a while back <br>using a reentrant lock per DirectoryProvider.<br><br>:-)<br>Doing the same thing.
<br><br>public class SynchLuceneEventListener implements <br>PostDeleteEventListener, PostInsertEventListener,<br> PostUpdateEventListener, Initializable {<br><br> private Map<String, ExtendedDocumentBuilder> documentBuilders = new
<br>HashMap<String, ExtendedDocumentBuilder>();<br> private Map<File, IndexWriter> tmpWriterMap = new TreeMap<File, <br>IndexWriter>();<br><br> private boolean initialized;<br><br> protected final Log log =
LogFactory.getLog(getClass());<br><br> /**<br> * Using JNDI to access DynamicBooster instance.<br> * Can have problems with serializability.<br> * @see JBossSynchLuceneEventListener for MBean implementation<br>
*/<br> protected DynamicBooster lookupDynamicBooster(Properties properties) <br>throws Exception {<br> String lookupName = DynamicBooster.JNDI_NAME;<br> String boosterJndiName = <br>properties.getProperty
(DynamicBooster.PROPERTY_JNDI_NAME);<br> if (boosterJndiName != null) {<br> lookupName = boosterJndiName;<br> }<br> Context jndiContext = NamingHelper.getInitialContext(properties);<br> return (DynamicBooster)jndiContext.lookup(lookupName);
<br> }<br><br> /**<br> * No need to synchronize, since this is the only intialized once.<br> * When used in multiple .par files, they are initialized sequentially.<br> */<br> public void initialize(Configuration cfg) {
<br> if (initialized) return;<br><br> try {<br> final Properties properties = cfg.getProperties();<br> DirectoryLocker directoryLocker = DirectoryLocker.getInstance();<br> directoryLocker.initialize
(properties);<br> DynamicBooster dynamicBooster = <br>lookupDynamicBooster(properties);<br><br> Iterator iter = cfg.getClassMappings();<br> while (iter.hasNext()) {<br> PersistentClass pc = (PersistentClass)iter.next();
<br> Class mappedClass = pc.getMappedClass();<br> if (mappedClass != null) {<br> if (mappedClass.getAnnotation(Indexed.class) != null) {<br> String entityName =
pc.getEntityName();<br> final ExtendedDocumentBuilder documentBuilder = <br>new ExtendedDocumentBuilder(entityName, mappedClass);<br> documentBuilder.setDynamicBooster(dynamicBooster);
<br> documentBuilders.put(entityName, documentBuilder);<br> File file = documentBuilder.getFile();<br> try {<br> IndexWriter iw =
tmpWriterMap.get(file);<br> if (iw == null) {<br> boolean create = !file.exists();<br> SynchDirectory sdir = <br>DirectoryLocker.getInstance
().getDirectory(file, create);<br> try {<br> iw = new IndexWriter(sdir, <br>documentBuilder.getAnalyzer(), create);<br> } finally {
<br> sdir.unlock();<br> }<br> tmpWriterMap.put(file, iw);<br> }<br> } catch (IOException ioe) {
<br> throw new HibernateException(ioe);<br> }<br> log.info("index: " + <br>documentBuilder.getFile().getAbsolutePath() + " - " + entityName);
<br> }<br> }<br> }<br> initialized = true;<br> } catch (Exception e) {<br> throw new HibernateException(e);<br> } finally {<br> for (IndexWriter iw :
tmpWriterMap.values()) {<br> LuceneUtils.close(iw);<br> }<br> tmpWriterMap.clear();<br> tmpWriterMap = null;<br> }<br> }<br><br> public void onPostInsert(PostInsertEvent event) {
<br> final Object entity = event.getEntity();<br> ExtendedDocumentBuilder builder = <br>documentBuilders.get(event.getPersister().getEntityName());<br> if (builder != null) {<br> add(entity, builder,
event.getId());<br> }<br> }<br><br> public void onPostUpdate(PostUpdateEvent event) {<br> final Object entity = event.getEntity();<br> ExtendedDocumentBuilder builder = <br>documentBuilders.get(event.getPersister
().getEntityName());<br> if (builder != null) {<br> final Serializable id = event.getId();<br> remove(builder, id);<br> add(entity, builder, id);<br> }<br> }<br><br> public void onPostDelete(PostDeleteEvent event) {
<br> ExtendedDocumentBuilder builder = <br>documentBuilders.get(event.getPersister().getEntityName());<br> if (builder != null) {<br> remove(builder, event.getId());<br> }<br> }<br><br> private void remove(ExtendedDocumentBuilder builder, Serializable id) {
<br> Term idTerm = builder.getTerm(id);<br> File file = builder.getFile();<br> log.info("removing: " + idTerm + ", " + file);<br> try {<br> IndexReader reader = null;<br>
SynchDirectory sdir = <br>DirectoryLocker.getInstance().getDirectory(file);<br> try {<br> reader = IndexReader.open(sdir);<br> reader.delete(idTerm);<br> } finally {
<br> LuceneUtils.close(reader);<br> sdir.unlock();<br> }<br> } catch (IOException ioe) {<br> throw new HibernateException(ioe);<br> }<br> }<br><br> private void add(final Object entity, final ExtendedDocumentBuilder
<br>builder, final Serializable id) {<br> Document doc = builder.getDocument(entity, id);<br> File file = builder.getFile();<br> Analyzer analyzer = builder.getAnalyzer();<br> log.info("adding: " + doc + ", " + file + ", " +
<br>analyzer.getClass().getName());<br> try {<br> IndexWriter writer = null;<br> SynchDirectory sdir = <br>DirectoryLocker.getInstance().getDirectory(file);<br> try {<br> writer = new IndexWriter(sdir, analyzer, false);
<br> writer.addDocument(doc);<br> } finally {<br> LuceneUtils.close(writer);<br> sdir.unlock();<br> }<br> } catch (IOException ioe) {<br> throw new HibernateException(ioe);
<br> }<br> }<br><br>}<br><br>public class SynchDirectory extends Directory {<br><br> private static final Log log = LogFactory.getLog(SynchDirectory.class);<br> private Directory directory;<br> private String name;
<br> private java.util.concurrent.locks.Lock lock;<br><br> public SynchDirectory(Directory directory) {<br> this.directory = directory;<br> lock = new ReentrantLock();<br> }<br><br> void lock(String name) {
<br> <a href="http://this.name">this.name</a> = name;<br> log.debug("Locking synch directory: " + name);<br> lock.lock();<br> }<br><br> public void unlock() {<br> lock.unlock();<br>
log.debug("Unlocking synch directory: " + name);<br> <a href="http://this.name">this.name</a> = null;<br> }<br><br> public String[] list() throws IOException {<br> return directory.list();<br> }
<br><br> public boolean fileExists(String name) throws IOException {<br> return directory.fileExists(name);<br> }<br><br> public long fileModified(String name) throws IOException {<br> return directory.fileModified
(name);<br> }<br><br> public void touchFile(String name) throws IOException {<br> directory.touchFile(name);<br> }<br><br> public void deleteFile(String name) throws IOException {<br> directory.deleteFile
(name);<br> }<br><br> public void renameFile(String from, String to) throws IOException {<br> directory.renameFile(from, to);<br> }<br><br> public long fileLength(String name) throws IOException {<br> return
directory.fileLength(name);<br> }<br><br> public OutputStream createFile(String name) throws IOException {<br> return directory.createFile(name);<br> }<br><br> public InputStream openFile(String name) throws IOException {
<br> return directory.openFile(name);<br> }<br><br> public Lock makeLock(String name) {<br> return directory.makeLock(name);<br> }<br><br> public void close() throws IOException {<br> directory.close
();<br> }<br><br>}<br><br>>> What is additional POJO handling?<br><br>I'm using predetermined HQL to select only Lucene indexable properties, <br>associations - minimizing the stuff that is pulled out when running a
<br>full new indexation.<br><br>public @interface LuceneReadQuery {<br><br> String value();<br><br>}<br><br>@LuceneReadQuery("select new Article(<a href="http://a.id">a.id</a>, a.title, a.body, a.intro, <br>a.subtitle
, <a href="http://a.source.name">a.source.name</a>, <a href="http://a.category.id">a.category.id</a>, a.publishDate, a.locale) from <br>Article a")<br><br><br>I also need different Analyzer instances for different POJOs - some are
<br>localizable, some are plain english text, some are just a bunch of <br>numbers, ...<br><br>public @interface Analyzer {<br><br> Class<? extends org.apache.lucene.analysis.Analyzer> analyzerClass();<br><br>}<br>
<br>I had some issues when different pojos where indexed in different files <br>- I needed them to be in the same one. Ok, you can do this by setting <br>index attribute.<br>But I needed to handle identity stuff - so that based on given result I
<br>could pull out the right pojo from Session.<br><br> doc.add(Field.Keyword(ENTITY_FIELD_NAME, entityName));<br> doc.add(Field.Keyword(IDENTITY_FIELD_NAME, <br>createIdentityString(id)));<br><br> private String createIdentityString(Serializable id) {
<br> return (entityName + "#" + id);<br> }<br><br> //possible identifying fields<br> String entityName = <br>doc.get(ExtendedDocumentBuilder.ENTITY_FIELD_NAME);<br><br>
if (entityName != null) {<br> // todo - currently expecting only integer id's<br> Object entity = session.get(<br> entityName,<br>
<br>Integer.parseInt(doc.get(ExtendedDocumentBuilder.ID_FIELD_NAME))<br> );<br><br><br>>> PS I remember an old JIRA issue of you talking about a filter <br>capability. I thing it's a nice idea, would probably make sense after
<br>the core work is stable.<br><br>Yep.<br>Having @Permission annotation - used the following way:<br><br> Permission p = currClass.getAnnotation(Permission.class);<br> if (p != null) {<br>
permissions.add(p.permission());<br> }<br><br> Permissions ps = currClass.getAnnotation(Permissions.class);<br> if (ps != null) {<br> permissions.addAll(Arrays.asList(ps.permission
()));<br> }<br><br><br> if (!permissions.isEmpty()) {<br> doc.add(Field.UnStored(<br> PERMISSION_FIELD_NAME,<br> StringHelper.join(",", permissions.iterator
()))<br> );<br> }<br><br>And this filter takes care of applying permissions to Lucene search.<br><br>public class SinglePermissionFilter extends Filter {<br><br> private String permission;<br><br> public SinglePermissionFilter() {
<br> }<br><br> public SinglePermissionFilter(String permission) {<br> this.permission = permission;<br> }<br><br> public BitSet bits(IndexReader reader) throws IOException {<br> BitSet bits = new BitSet(
reader.maxDoc());<br> String fieldName = ExtendedDocumentBuilder.PERMISSION_FIELD_NAME;<br> TermDocs td = reader.termDocs(new Term(fieldName, permission));<br> while(td.next()) {<br> bits.set(td.doc
());<br> }<br> return bits;<br> }<br><br> public String getPermission() {<br> return permission;<br> }<br><br> public void setPermission(String permission) {<br> this.permission = permission;
<br> }<br><br>}<br><br>Executing search with permission filter:<br><br> Hits hits = searcher.search(sqh.getQuery(), permissionFilter);<br><br><br>Ok, if I remember some other issues that I had - will email them. :-)
<br><br><br>Rgds, Ales<br><br><br>Emmanuel Bernard wrote:<br>> Hi Ales<br>><br>> Yes let's move that to the mailing list<br>><br>> What is DynamicBoost?<br>> I've added support for @Boost on both attributes and entity
<br>> I have also introduced a Bridge notion that do the translation between <br>> the property and the Lucene field. I'm almost done with that one. This <br>> is very much like the Hibernate Type in it's flexibility and I'll add
<br>> some specific annotations support for numeric padding and date resolution<br>><br>> I fixed the concurrency issues in the Lucene event a while back using <br>> a reentrant lock per DirectoryProvider. Did you check the code in
<br>> <a href="http://anonsvn.jboss.org/repos/hibernate/branches/Lucene_Integration/">http://anonsvn.jboss.org/repos/hibernate/branches/Lucene_Integration/</a><br>> I'm interested in seeing some additional issue if any.
<br>><br>> What is additional POJO handling?<br>><br>> PS I remember an old JIRA issue of you talking about a filter <br>> capability. I thing it's a nice idea, would probably make sense after <br>> the core work is stable.
<br>><br>> Ales Justin wrote:<br>>> Hey,<br>>><br>>> I extended the initial usage quite a lot - needed some additional stuff.<br>>> Some additional annotations - Boost, DynamicBoost, DateField, ...
<br>>> Rewritten current LuceneListener - had problems with concurrency, <br>>> additional pojo handling, ...<br>>><br>>> I can sum up some stuff (non customized stuff) and send it - if <br>>> you're interested.
<br>>> If so - to you or to the mailing list?<br>>><br>>> Rgds, Ales<br>>><br>>> ps: coming to JBW Berlin?<br>><br><br></pre>