Hi again Thomas,

I just want to ask again up front: can we have this discussion on errai-dev or forge-dev? I think there are a bunch of interested people who could provide additional valuable feedback.

On Thu, Feb 28, 2013 at 5:14 PM, Thomas Frühbeck <fruehbeck@aon.at> wrote:

Hi Jonathan,
you shouldnt count _every_  artifact related to an entity:
    - 1 Service Interface
    - 1 Service Impl (1 serivce impl is to be omitted)
    - 1 Mapping
    - 1 Wrapper
    - 1 Reference for relationship transport
    - 1 edit form
    - 1 search form
    - 1 html file
did I miss something? you cant do CRUD on server w/o some server logic, and honestly, do you mean to stuff all UI into one class? can't imagine that. It's feasable, I tried to go that direction, but the handling of @Bound (@Data) fields becomes quite tricky, taking into account all the validation and compilation steps.

No, I wouldn't want to stuff all the UI into one class. I like the idea of having a separate @Templated form for each entity. For that matter, I even like the idea of keeping separate search, edit, and read-only @Templated classes. But I don't think I would end up needing all of those for each entity type.

But I agree, quite verbose :-/ Just now I am extending a generated project for a real world project, and I start to suffer already :-)

Yes. As a practical example, I'm imagining what I would do to improve the usability of the Sakila app. Starting with the scaffolding as generated, I would probably delete more than half of what was generated. Quick brain dump:

I'd keep the Films, Actors, Stores, Staffs and Customers as top-level places in the UI.

I'd make Address, Language, Category, Country, and City work as auto-complete fields instead of hard references that you have to create first. Maybe in an admin screen, there could be the opportunity to list these things and delete them.. but probably not. I'd probably just write a routine that deletes the orphaned entries periodically.

I'd integrate Rentals and Inventories into the Films, Stores, and Customers views, showing the details appropriate to each of those contexts (for example, in a list of Stores, we'd just see the rental count; in an individual store page, we could see more details perhaps in a table of Rental entries grouped by film title and sorted by % inventory utilization or something).

One thing that I notice from this little thought experiment: it would be useful to have a generator for an autocomplete textbox for a particular entity type. This would search by one or more attributes of the entity, resolving the existing entry if the user picks a completion, or creating a new one if the user doesn't pick a completion. I've already had to implement this a few times by hand in various Errai demos. For example, see the 'department' field in ItemForm: https://github.com/errai/errai/blob/master/errai-jpa/demos/errai-jpa-demo-grocery-list/src/main/java/org/jboss/errai/demo/grocery/client/local/ItemForm.java

The other thing I notice is that composability of the @Templated components is really important. I think they're already nicely composable, so this is more of an observation than a recommendation. :)

 

It _really should_ be possible to hide complexity and boiler plate at least for the Wrapper/EntityReference classes, they are really bad!

Yes, I think we should lean on the Errai framework itself to deal with these issues (either by 3-way diff or generated proxies or a mix of both). I don't think it's good to have the generated proxies (or really any generated code that's not meant to be edited by hand) mixed in with the application sources.

Let me say: Errai is really great, I'm glad we met!!
But there are hard limits, which are not easy to come by (no server only code, static factory, field dependency for mapping generation)

I don't understand what you're referring to here. (but I do agree there will inevitably be hard limits to what the framework can do) 

Regarding entity relationship: I will have a look at generated proxies, I hadn't seen the dynamic code generation yet. I am just starting reading about it.
Do you think it is possible to generate a proxy object for an entity, which "unfolds" itself when accessed by retrieving its data on demand from server?
Perhaps you have an example?

Yes, it's definitely possible to do that, but the fetching of new data would have to be asynchronous. The proxies wouldn't be terribly transparent.. code that uses them would need to use callbacks when traversing lazy-loaded relationships.

A feature like this would overlap the capabilities of the datasync feature I'm planning quite a bit. That's not necessarily a bad thing, of course!

I do not really like the "wrappers", but they are at least manageable, controllable.
Even if I used dynamic proxies, how can I assure, that the relationship is transparently transportable? Are you sure this is not becoming EJB 1.1 RemoteEntityBeans Revival?

That's a good point. I still want to continue forward with the syncable dataset idea and see how far it can take us. As I imagine it, we can make it work without any proxies at all.

 

One more problem I see coming is transaction/atomicity/validation of updates. For JSF and other request/response systems (like my plugin) it's easy to tell, where transaction and validation should take place. When doing continous updates of small changes, when is the correct time to lock/refresh/merge - that's already difficult for a JSF application when working with multiple related entities.

I'd generally want to merge as early and as often as possible. Each merge would have three pieces: the new requested state, the state the client thinks the entity has on the server, and the state actual current state on the server. If the expected state (from the client) differs from the actual state (from the server-side entity manager), then the update is rejected and sent back to the client along with the new server state.

I'd probably avoid locking entirely. Just optimistic refresh/merge. For atomicity, we'd simply guarantee that all updates that come together in the same message will be committed or rolled back (eg. due to merge conflict) as a unit.

 

Thanks for your feedback, looking forward to some tips regarding code generation!

The documentation on Errai's code generation API is still pretty sparse, unfortunately. Probably the best place to start out would be to look at Errai code that uses errai-codegen to generate proxies. BindableProxyGenerator and JaxrsProxyGenerator are two such examples.

Cheers,

Jonathan

Regards,
Thomas

Am 28.02.2013 22:32, schrieb Jonathan Fuerth:

Hi again Thomas,

I spent this morning trying to feel my way through the use of your plugin on a simple data model (2 tables with a FK relationship) that I created yesterday. I was able to use the hibernate-tools forge plugin to generate entities, but I couldn't quite figure out how to use your scaffolding plugin to generate Errai code. Anyway, I know this wasn't really the part you were looking for input on; I'm new to Forge and I'm sure it will be easy enough to figure out when there's a getting started doc for your plugin.

So next, I turned to the Sakila project you emailed me. I got it imported into Eclipse no problem, and I deployed it to AS7 equally easily.

The first thing that struck me is that I think there are too many types being generated for each entity. By my count, there are 9 files generated for every one entity in the datamodel. Based on your README.md, I understand you need a way of controlling which parts of the object graph get sent to the client, and also which attributes are overwritten when the updates are merged back into the server. I see that a number of these files are being generated in support of these concerns.

I think my ideal situation would be to generate just 2 files per entity: a .html template and the corresponding @Templated java class. This would leave the framework responsible for tree pruning (eg. through JPA fetch rules), fine-grained merging of attributes (by diffing or transparent proxies generated at GWT rebind time) and more. I know a proxy is a proxy, but if we can keep it out of the application's src/main/java folder, I think that's a win.

I think we should keep discussing this. Sorry it took me so long to respond... I'll try to be more responsive in the future.

-Jonathan

PS: it would be nice to have these discussions on a public mailing list of some sort. What about errai-dev or a Forge dev list?

On Wed, Feb 27, 2013 at 11:00 PM, Jonathan Fuerth <fuerth@fuerth.ca> wrote:

Hi Thomas,

I still have to play more with your errai scaffolding plugin to give feedback. I created a toy data model today (as a testbed for trying out your plugin), installed the latest Forge, and installed your plugin into it. I have not yet used your plugin to scaffold anything. I expect I will have time tomorrow morning to continue that.

Until then, here are my thoughts on https://github.com/shadogray/plugin-errai/blob/master/README.md:

== Merging in updates from the client ==

I agree EntityManager.merge() is too much of a sledgehammer when it comes to accepting updates from clients. We need a way to detect updates that only affected one or two attributes–perhaps even just an attribute of a related child entity reachable from the main subject of the sync request.

I also think that for scalability reasons we have to make the client responsible for tracking the state of the object it thinks it's merging into. This is the approach I'm taking with my data sync efforts so far: when the client submits an update to the server, it reports not only the new state it's requesting, but the 'expected state' that the server should have now. Any mismatch between the expected state and the actual current state on the server means there is a potential merge conflict. By receiving both the expected state and the requested new state, the server can compute the actual field-by-field difference in the update, and decide if a real merge conflict took place. If so, it can respond to the client with the actual new state, and client code can do what it wants to resolve it (probably by presenting some sort of yes/no decision or even a merge UI to the user).

== Handling of Entity References ==

Yes, this is a major concern. Since my initial plan is to define syncable data sets using JPQL named queries, I thought it would be easiest to use the FETCH clause to define the limits of what gets synced and what stays behind (lazy fetch == don't sync to client).

Since then, the EntityGraph API has found its way into JPA 2.1. As far as I can tell, this is pretty much purpose-built to solve this problem without the need for wrapper DTOs. We could build Errai's Data Sync features against this today; the only downside is that it's going to take a while before we can count on JPA 2.1 APIs being available on the server side.

== Short and Char as ID types ==

Errai JPA does support all numeric types for IDs, but I purposely stopped short of implementing ID Generators for byte, short, and char. I'm happy to add these ID Generators if you've got a use case for them.

Thanks for your patience!

Jonathan