----- "Larry O'Leary" <loleary(a)redhat.com&gt; wrote: > On Mon, 2009-07-13 at 16:11 -0500, Van Halbert wrote: > > You are correct, no data source connection information will be in > the > > open. The product side will maintain its own data source > > information. Part of the challenge to this new framework will be > to > > enable a contributor to run their tests against only the data > > source(s) they have. The contributor most likely will not have > all > > the flavors. > > Which is why it is important for us to figure out the best way to > organize the tests within the test framework. The idea would be that > I > can specify that I want the Salesforcce connector tests to be > executed > and by doing so, I must provide Salesforce credentials that can be > used > with the connector. Furthermore, it would be very nice if the > schemas > used with the Salesforce connector could be the same as the ones used > with the other connectors so that a basic/common test case suite > would > apply. For example, creating a script that could be responsible for > populating Salesforce with equivalent schema and data such as BQT so > that type testing can be executed in the same manner that it is > against > PostgreSQL. This of course shouldn't be a requirement but a goal so > that new schemas don't have to be created and maintained across each > connector. > I agree with this as a goal, but I don't think it's going to be possible in this particular case. To the best of my knowledge there is no way to programmatically modify the metadata of a Salesforce app. Views built on top of the physical model of the default free developer Salesforce app are one possible way to to the common schema you're describing. Then the contributor would be able to specify his credentials and that should be enough.

Another limitation of this particular case that probably won't inform the larger framework, but I mention it anyway, is that modifications of the default Salesforce app were made to expose some of their non-standard types for testing that are not used in the default app. So no community user would be able to run all of the Salesforce tests unless they made manual modifications to their default app. This limitation could be clearly specified in the failure messages of the tests depending upon the modification, and it's not a wrinkle that I think we should be spending time on.

But like I said at the top, I agree with the goal you propose.