Hi JR, 

and thanks for the input on this!

I had not thought about user privacy with regards to the endpoints as I was sort of blinded by that we now encode the UAID and CHID. Would it be an option to mandate this in the specification?

1) blob is limited to some size
1MB sounds like good limit. 

2) three strikes and you're out.
How about just leaving the number up to the specific implementation and perhaps let the specification can just recommend a value.
So basically this means that it is possible for an honest client to send a batch of notifications, but for some reason 3 are invalid and this would cause the batch to fail, and the App Server would not know about the failure.

Regarding traffic shaping I don't really have any input on that as we only support a single instance as it stands today. We hope to get this work started shortly though.

I think this all sounds good and it would be nice to try this out and see how it works in practice too. 
How about we add the size limit of the batch notification and the "three strikes and you're out" limit to AGPUSH-338[1] and see how this works in conjunction with the UnifiedPush Server?

[1] https://issues.jboss.org/browse/AGPUSH-338



On 18 September 2013 19:01, jr <jrconlin@gmail.com> wrote:
Hi,

JR from Mozilla here.

We've kicked bulk processing around for a while, really, but have had a few
issues we needed to deal with first. Ultimately, bulk processing originally
came down to two things:

1) User Privacy.
This is a bit less of a concern if a system uses the token hashing, since
the UAID and CHID are encoded. It's a problem if there's no token hashing
because there's meta information leakage that happens, and that could cause
problems.

2) Griefing attacks.
Bad guys who want to disrupt the system could send blocks of bad data that
the servers would have to spin up and check, which could go into the
thousands of records. Yeah, it's possible to do this with traditional ones,
but it's more of a pain in the patoot for the griefers because they have to
spin up a new connection for each bad item they want to send.

3) Traffic shaping
Ideally, endpoints are completely opaque. That gives servers the ability to
send some UAIDs to a completely different cluster by specifying a different
hostname or other mechanism. Generally, i've found that if you let users
disassemble something like an endpoint, there be dragons.

Now, since time stands still for no requirement specification, things have
changed a bit since the very early discussions. I like the proposed solution
of passing a JSON blob to /update. What I'd suggest is the following:

1) blob is limited to some size. I'd recommend something like a 1MB, since
that should be able to hold around 8738 tokens. That should be enough to
handle most loads (except something like eBay or twitter, but I'm betting
those are special cases anyway.)

2) three strikes and you're out. This is something that I'm actually
debating pretty hard about. Really it's the number. There needs be some
point where the server says, "this is crap" and drops everything. Pretty
easy to do if a token doesn't decode, but more tricky when you've got 1 out
of n UAIDs failing (for non-hashed tokens). What also makes this hard is
that you don't want to tell a hacker that they're succeeding, so we can't
give information back to the App Server.

Thoughts?



--
View this message in context: http://aerogear-dev.1069024.n5.nabble.com/aerogear-dev-simplepush-Batch-notifications-tp4701p4770.html
Sent from the aerogear-dev mailing list archive at Nabble.com.
_______________________________________________
aerogear-dev mailing list
aerogear-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-dev