[jboss-jira] [JBoss JIRA] (JGRP-653) Streaming API for large messages

[Andrew Scully (JIRA)]

    [ https://issues.jboss.org/browse/JGRP-653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799487#comment-12799487 ] 

Andrew Scully commented on JGRP-653:
------------------------------------

I also think that an API for streaming messages would be useful.

I've looked into using a "streaming state transfer" to this end, but it appears to be a bad fit for the use case. This is because the getState() callback must transfer the entirety of the data into the provided output stream, rather than returning an InputStream that can be read from in chunks. Also, this usage is not the original intention.

Such an API would need to accept and return InputStream objects. Better yet, ReadableByteStream objects, for compatibility with other NIO code.

Changing the RpcDispatcher to be able to target methods with an InputStream / ReadableByteStream argument (or return type) would also be useful (Hessian, for example, allows you to do this).
                
> Streaming API for large messages
> --------------------------------
>
>                 Key: JGRP-653
>                 URL: https://issues.jboss.org/browse/JGRP-653
>             Project: JGroups
>          Issue Type: Feature Request
>            Reporter: Bela Ban
>            Assignee: Bela Ban
>             Fix For: Future
>
>         Attachments: JGroupsInputStream.java, JGroupsOutputStream.java, StreamTest.java
>
>
> For large messages, to load the entire payload into memory might be bad because the payload might be bigger than the max memory available. It would be useful to have an API which allows for use of input and output streams, so that large payloads can be read iteratively by a user and streamed out to the cluster via the underlying channel breaking the data in the input stream into chunks, which are fed into the input stream on the receivers side.
> Issues: we have to have 1 input stream per sender on the receiver side, because a stream is always defined between 2 parties (sender, receiver). Maybe something like NIO, where we register interest in a stream, are notified of new streams ('accept()') and get notified when data on any of the stream is available, would be beneficial.
> Demo is attached

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira