7:24 p.m.
Hi All,
Would like to know your opnion about how we're storing identity information using
the File-based Identity Store and discuss possible alternatives.
Just for background, the motivation behind the file-based store is to provide a fast,
ready-to-use and simple store, ideally for test and development scenarios/environments.
The configuration is minimal and requires the file system.
Today we're basically serializing objects (JDK Serialization API) and storing them
into files. The layout is quite simple:
/tmp/pl-idm/:
total 4
drwxrwxr-x. 2 pedroigor pedroigor 140 Jan 18 15:20
65d62693-953c-43a6-ac43-4b655174bbb4 ----> Each Partitions has its own directory
-rw-rw-r--. 1 pedroigor pedroigor 554 Jan 18 15:20 pl-idm-partitions.db ---->
Serialized data for partitions
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-relationships.db
----> Serialized data for Relationships
/tmp/pl-idm/65d62693-953c-43a6-ac43-4b655174bbb4: ----> Partition directory.
total 8
-rw-rw-r--. 1 pedroigor pedroigor 789 Jan 18 15:20 pl-idm-agents.db ---->
Serialized data for Agents
-rw-rw-r--. 1 pedroigor pedroigor 1134 Jan 18 15:20 pl-idm-credentials.db ---->
Serialized data for Credentials
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-groups.db ---->
Serialized data for Groups
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-roles.db ---->
Serialized data for Roles
Serialization provides us a fast way to store data, but I have some concerns that I
want to share:
- As we're serializing objects, we may have to ensure compatibility with prior
versions. I think Version Control is a option here (btw, Stuart Douglas gave me some tips
about that).
- Is better to use JBoss Marshalling instead of using the JDK Serialization API
directly ? Mainly considering the JBoss ecosystem ?
- Is there a better format to store data ? Such as XML ...
- I had some discussions with Shane about using Infinispan. We agreed that the IDM
cache will be ISPN-based, that is fine. But maybe a ISPN-based store can fits well too.
ISPN allows to store data using different CacheStore implementations, transaction support,
indexing, distributable or local storage, etc.
Wdyt ?
Regards,
Pedro Igor
7:55 p.m.
On 01/18/2013 12:24 PM, Pedro Igor Silva wrote:
Hi All,
Would like to know your opnion about how we're storing identity information
using the File-based Identity Store and discuss possible alternatives.
Just for background, the motivation behind the file-based store is to provide a
fast, ready-to-use and simple store, ideally for test and development
scenarios/environments. The configuration is minimal and requires the file system.
A
potential usage is in JBossAS which may not have default configuration
for databases and ldap. In that case, we have to worry about the
serialization issues with files and versions.
Today we're basically serializing objects (JDK Serialization
API) and storing them into files. The layout is quite simple:
/tmp/pl-idm/:
total 4
drwxrwxr-x. 2 pedroigor pedroigor 140 Jan 18 15:20
65d62693-953c-43a6-ac43-4b655174bbb4 ----> Each Partitions has its own directory
-rw-rw-r--. 1 pedroigor pedroigor 554 Jan 18 15:20 pl-idm-partitions.db ---->
Serialized data for partitions
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-relationships.db
----> Serialized data for Relationships
/tmp/pl-idm/65d62693-953c-43a6-ac43-4b655174bbb4: ----> Partition directory.
total 8
-rw-rw-r--. 1 pedroigor pedroigor 789 Jan 18 15:20 pl-idm-agents.db ---->
Serialized data for Agents
-rw-rw-r--. 1 pedroigor pedroigor 1134 Jan 18 15:20 pl-idm-credentials.db
----> Serialized data for Credentials
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-groups.db ---->
Serialized data for Groups
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-roles.db ---->
Serialized data for Roles
Serialization provides us a fast way to store data, but I have some concerns that I
want to share:
- As we're serializing objects, we may have to ensure compatibility with
prior versions. I think Version Control is a option here (btw, Stuart Douglas gave me some
tips about that).
- Is better to use JBoss Marshalling instead of using the JDK Serialization API
directly ? Mainly considering the JBoss ecosystem ?
- Is there a better format to store data ? Such as XML ...
Compared to
Java serialization, XML would be painful.
- I had some discussions with Shane about using Infinispan.
We agreed that the IDM cache will be ISPN-based, that is fine. But maybe a ISPN-based
store can fits well too. ISPN allows to store data using different CacheStore
implementations, transaction support, indexing, distributable or local storage, etc.
The Default cache should be ISPN. But I am not sold on a ISPN store.
ISPN File Cache Store instead of IDM FileStore Implementation is a
option. But JDBCCacheStore probably not.
9:57 a.m.
+1 For PicketLink to be enabled out of the box within the application
server it is going to have to have a suitable store for that scenario so
going beyond just testing and development.
The store used for the out of the box scenario also needs to be
supportable and should problems occur it will need to be recoverable and
it must be maintainable for backwards compatibility so to me it feels
that purely using JDK based serialization adds too much risk here.
Having said that it has been a while since we last reviewed if there are
any embeddable databases with local file system storage that could also
meet our needs - there are various complications here but it may be
something to review again before we end up writing out own.
Regards,
Darran Lofthouse.
On 01/18/2013 06:55 PM, Anil Saldhana wrote:
On 01/18/2013 12:24 PM, Pedro Igor Silva wrote:
> Hi All,
>
> Would like to know your opnion about how we're storing identity information
using the File-based Identity Store and discuss possible alternatives.
>
> Just for background, the motivation behind the file-based store is to provide a
fast, ready-to-use and simple store, ideally for test and development
scenarios/environments. The configuration is minimal and requires the file system.
A potential usage is in JBossAS which may not have default configuration
for databases and ldap. In that case, we have to worry about the
serialization issues with files and versions.
> Today we're basically serializing objects (JDK Serialization API) and
storing them into files. The layout is quite simple:
>
> /tmp/pl-idm/:
> total 4
> drwxrwxr-x. 2 pedroigor pedroigor 140 Jan 18 15:20
65d62693-953c-43a6-ac43-4b655174bbb4 ----> Each Partitions has its own directory
> -rw-rw-r--. 1 pedroigor pedroigor 554 Jan 18 15:20 pl-idm-partitions.db
----> Serialized data for partitions
> -rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-relationships.db
----> Serialized data for Relationships
>
> /tmp/pl-idm/65d62693-953c-43a6-ac43-4b655174bbb4: ----> Partition
directory.
> total 8
> -rw-rw-r--. 1 pedroigor pedroigor 789 Jan 18 15:20 pl-idm-agents.db
----> Serialized data for Agents
> -rw-rw-r--. 1 pedroigor pedroigor 1134 Jan 18 15:20 pl-idm-credentials.db
----> Serialized data for Credentials
> -rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-groups.db
----> Serialized data for Groups
> -rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-roles.db
----> Serialized data for Roles
>
> Serialization provides us a fast way to store data, but I have some concerns
that I want to share:
>
> - As we're serializing objects, we may have to ensure compatibility
with prior versions. I think Version Control is a option here (btw, Stuart Douglas gave me
some tips about that).
>
> - Is better to use JBoss Marshalling instead of using the JDK Serialization
API directly ? Mainly considering the JBoss ecosystem ?
>
> - Is there a better format to store data ? Such as XML ...
Compared to Java serialization, XML would be painful.
> - I had some discussions with Shane about using Infinispan. We agreed that
the IDM cache will be ISPN-based, that is fine. But maybe a ISPN-based store can fits well
too. ISPN allows to store data using different CacheStore implementations, transaction
support, indexing, distributable or local storage, etc.
The Default cache should be ISPN. But I am not sold on a ISPN store.
ISPN File Cache Store instead of IDM FileStore Implementation is a
option. But JDBCCacheStore probably not.
_______________________________________________
security-dev mailing list
security-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/security-dev
2:53 a.m.
Pedro Igor Silva wrote:
Hi All,
Would like to know your opnion about how we're storing identity information
using the File-based Identity Store and discuss possible alternatives.
Just for background, the motivation behind the file-based store is to provide a
fast, ready-to-use and simple store, ideally for test and development
scenarios/environments. The configuration is minimal and requires the file system.
Today we're basically serializing objects (JDK Serialization API) and storing
them into files. The layout is quite simple:
/tmp/pl-idm/:
total 4
drwxrwxr-x. 2 pedroigor pedroigor 140 Jan 18 15:20
65d62693-953c-43a6-ac43-4b655174bbb4 ----> Each Partitions has its own directory
-rw-rw-r--. 1 pedroigor pedroigor 554 Jan 18 15:20 pl-idm-partitions.db ---->
Serialized data for partitions
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-relationships.db
----> Serialized data for Relationships
/tmp/pl-idm/65d62693-953c-43a6-ac43-4b655174bbb4: ----> Partition
directory.
total 8
-rw-rw-r--. 1 pedroigor pedroigor 789 Jan 18 15:20 pl-idm-agents.db ---->
Serialized data for Agents
-rw-rw-r--. 1 pedroigor pedroigor 1134 Jan 18 15:20 pl-idm-credentials.db
----> Serialized data for Credentials
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-groups.db ---->
Serialized data for Groups
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-roles.db ---->
Serialized data for Roles
Serialization provides us a fast way to store data, but I have some concerns that I
want to share:
- As we're serializing objects, we may have to ensure compatibility with
prior versions. I think Version Control is a option here (btw, Stuart Douglas gave me some
tips about that).
- Is better to use JBoss Marshalling instead of using the JDK Serialization API
directly ? Mainly considering the JBoss ecosystem ?
- Is there a better format to store data ? Such as XML ...
I think that long term XML or a custom file format will give better
results than just using JDK serialization to dump objects to disk, as it
gives you much more flexibility in terms of versioning.
The flip side is of course that it does involve more work. A potential
compromise could be to use serialization, but only to write primitive
fields, for example instead of something like:
for(User user: users) {
out.writeObject(user);
}
Do something like:
out.write(1); //file version number
for(User user: users) {
out.writeObject(user.getUsername());
out.writeObject(user.getPassword());
//etc
}
This is still very quick to code, and does not reference any Picketlink
classes in the serialized data.
Stuart
- I had some discussions with Shane about using Infinispan. We agreed that the
IDM cache will be ISPN-based, that is fine. But maybe a ISPN-based store can fits well
too. ISPN allows to store data using different CacheStore implementations, transaction
support, indexing, distributable or local storage, etc.
Wdyt ?
Regards,
Pedro Igor
_______________________________________________
security-dev mailing list
security-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/security-dev
1:03 p.m.
+1. XML involves more work, that is why I'm not sure if we should use it given the
nature of this store.
Regarding the serialization of only primitives/properties, I think that would bring some
flexibility and maintain a simple implementation. I started to use that for some types
(using a Map to store properties and attributes for each instance), maybe I should apply
the same for others.
Thanks.
Pedro Igor
----- Original Message -----
From: "Stuart Douglas" <sdouglas(a)redhat.com>
To: "Pedro Igor Silva" <psilva(a)redhat.com>
Cc: security-dev(a)lists.jboss.org
Sent: Sunday, January 20, 2013 11:53:28 PM
Subject: Re: [security-dev] [PicketLink IDM] - File-based Identity Store
Pedro Igor Silva wrote:
Hi All,
Would like to know your opnion about how we're storing identity information
using the File-based Identity Store and discuss possible alternatives.
Just for background, the motivation behind the file-based store is to provide a
fast, ready-to-use and simple store, ideally for test and development
scenarios/environments. The configuration is minimal and requires the file system.
Today we're basically serializing objects (JDK Serialization API) and storing
them into files. The layout is quite simple:
/tmp/pl-idm/:
total 4
drwxrwxr-x. 2 pedroigor pedroigor 140 Jan 18 15:20
65d62693-953c-43a6-ac43-4b655174bbb4 ----> Each Partitions has its own directory
-rw-rw-r--. 1 pedroigor pedroigor 554 Jan 18 15:20 pl-idm-partitions.db ---->
Serialized data for partitions
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-relationships.db
----> Serialized data for Relationships
/tmp/pl-idm/65d62693-953c-43a6-ac43-4b655174bbb4: ----> Partition
directory.
total 8
-rw-rw-r--. 1 pedroigor pedroigor 789 Jan 18 15:20 pl-idm-agents.db ---->
Serialized data for Agents
-rw-rw-r--. 1 pedroigor pedroigor 1134 Jan 18 15:20 pl-idm-credentials.db
----> Serialized data for Credentials
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-groups.db ---->
Serialized data for Groups
-rw-rw-r--. 1 pedroigor pedroigor 0 Jan 18 15:20 pl-idm-roles.db ---->
Serialized data for Roles
Serialization provides us a fast way to store data, but I have some concerns that I
want to share:
- As we're serializing objects, we may have to ensure compatibility with
prior versions. I think Version Control is a option here (btw, Stuart Douglas gave me some
tips about that).
- Is better to use JBoss Marshalling instead of using the JDK Serialization API
directly ? Mainly considering the JBoss ecosystem ?
- Is there a better format to store data ? Such as XML ...
I think that long term XML or a custom file format will give better
results than just using JDK serialization to dump objects to disk, as it
gives you much more flexibility in terms of versioning.
The flip side is of course that it does involve more work. A potential
compromise could be to use serialization, but only to write primitive
fields, for example instead of something like:
for(User user: users) {
out.writeObject(user);
}
Do something like:
out.write(1); //file version number
for(User user: users) {
out.writeObject(user.getUsername());
out.writeObject(user.getPassword());
//etc
}
This is still very quick to code, and does not reference any Picketlink
classes in the serialized data.
Stuart
- I had some discussions with Shane about using Infinispan. We agreed that the
IDM cache will be ISPN-based, that is fine. But maybe a ISPN-based store can fits well
too. ISPN allows to store data using different CacheStore implementations, transaction
support, indexing, distributable or local storage, etc.
Wdyt ?
Regards,
Pedro Igor
_______________________________________________
security-dev mailing list
security-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/security-dev
4177
days inactive
4180
days old
4 comments
4 participants
participants (4)
-
Anil Saldhana -
Darran Lofthouse -
Pedro Igor Silva -
Stuart Douglas