[windup-dev] [JBoss Migration Community of Practice] New message: "JBoss EAP Migration: discovering unsupported application libraries"

Tue May 31 11:08:38 EDT 2016

Hi Jason,

I have looked closer at Víctims.
I have few questions/issues. Could you please help resolving those?

Note: I'm adding a PUBLIC mailing list, Windup developers. Feel free to 
add some Victims list (is there one?)

1) Hashes are not real checksums
As someone wrote in https://github.com/victims/victims-cve-db/issues/45
the hashes used by Victims are not just SHA512 hashes of the file 
content, but something else.
I'd like to be able to either find CVE's by a normal file content hash, 
or create the Victims hash.
a) Is there a Java impl?
b) Could you add the plain SHA512 (or other, I'm okay with just CRC32) 
hash to the data?

2) Victims Java client API

The Java API doesn't match the needs much.
 From what I can see, it can
   a) Sync with the server
   b) Give me a list of CVE for given SHA512 hash.

What I would like to have is:
* Have some offline data distributed with our app, provide these data
* Search the database by Maven coordinates, classes,
* Get a short description of the CVE and date of appearance and 
how/where it was fixed

Is there a plan for extending the Java API?
Also I guess not all these are covered in the Victims database, right?

3) Configuration
The configuration is done through system properties, that's not too 
fortunate.
For instance it doesn't allow to run multiple clients at once in the 
same JVM.
Could that be done through an API?

4) Data structure
The data structure of the JSON is not obvious. Is there some docs for it?

5) Data storage
The data are only stored in a database. Could it be simply stored in a 
JSON or XML file? The file is just 165 KB and not growing too fast, so I 
think rather than bringing an embedded DB as a dependency, I'd prefer to 
process a XML file into a HashMap or a Lucene index and use that.

Thanks,
Ondra

On 4.4.2016 02:16, Jason Shepherd wrote:
> Hi Ondra,
>
> The architecture of Victims is such that you should never need to
> 'download' the database. The client is designed to connect to the
> central http://victi.ms API to get the latest vulnerabilities.
>
> That being said, the authors also have a 'backup' of the data in the
> form of a Github repository, [1]. In fact some members of the
> community have built a tool which just uses this repository, and does
> not use the API at all. Recently we've built a tool to rebuild the
> database from the Github repository, but it still needs some work,
> [3].
>
>     [1] https://github.com/victims/victims-cve-db
>     [2] https://github.com/h3xstream/maven-security-versions
>     [3] https://github.com/jasinner/victims-db-builder
>
> Let me know if you need any further information.
> Regards,
> Jason Shepherd
>
> On Fri, Apr 1, 2016 at 1:38 AM, Ondrej Zizka <ozizka at redhat.com> wrote:
>> Great to know it goes on, last time I talked to someone (I think djorm), he
>> said the development was stagnant.
>>
>> Jason, is there a way to download a single big file with all data in the
>> database?
>>
>> Thanks,
>> Ondra
>>
>>
>>
>>
>> On 10.3.2016 03:49, Jason Shepherd wrote:
>>
>> Hello Rodney,
>>
>> The Product Security team are still maintaining that project. It's part of
>> our process to add new CVEs to the Victims Database when they are found.
>>
>> Also, if we've missed anything, you can add a library to the database
>> yourself using the web interface at https://victim.ms
>>
>> Regards,
>> Jason Shepherd
>>
>> On Thu, Mar 10, 2016 at 1:18 AM, Marek Novotny <mnovotny at redhat.com> wrote:
>>> Why do you think nobody picked it up?
>>> I can see there at least Stephen Milner, Jason Shepherd from RH. Are
>>> they responsible for the data? I added them to CC.
>>>
>>> Also even from non-redhatter member there is the latest commit from 4th
>>> January this year https://github.com/victims/victims-cve-db so it is not
>>> really dead ;)
>>>
>>>
>>> On 9.3.2016 16:07, Rodney Russ wrote:
>>>> So, interestingly enough, this project was started from the security
>>>> team within Red Hat.  I believe the guy who started left Red Hat and no
>>>> one has picked it up since.
>>>>
>>>> On 8 Mar 2016, at 19:41, Ondrej Zizka wrote:
>>>>
>>>>> Right, I think the lack of fresh data is the issue with it. I think it
>>>>> needs someone at Red Hat adopting it and feeding with CVE's. Maybe GSS
>>>>> could take care? Try to push it higher, that's a company-wide thing.
>>>>>
>>>>> Ondra
>>>>>
>>>>>
>>>>> On 8.3.2016 00:06, Rodney Russ wrote:
>>>>>>
>>>>>> On 7 Mar 2016, at 8:35, Ondrej Zizka wrote:
>>>>>>
>>>>>>> We could use the same mechanism as with the Victims addon. Currently
>>>>>>> it's not in the distribution, though.
>>>>>> Was victims support something we wanted to move forward with?  It
>>>>>> doesn't look as active as it once was.
>>>>>>
>>>>>>> Ondra
>>>>>>>
>>>>>>>
>>>>>>> On 3.3.2016 18:54, Jess Sightler wrote:
>>>>>>>> We have a condition that can add application level messages for
>>>>>>>> this. I think that would be better than a generic org.apache.axis
>>>>>>>> catchall rule. For example, I think Axis2 uses the same packages
>>>>>>>> but would not necessarily have the same issues.
>>>>>>>>
>>>>>>>> On 03/03/2016 03:38 AM, Rodney Russ wrote:
>>>>>>>>> I agree that for the specific issue of Axis discussed, a rule
>>>>>>>>> should be added to the catch-all rules. What I outlined below was
>>>>>>>>> an answer to the question posed by Benjamin:
>>>>>>>>>
>>>>>>>>> "Aside from updating Windup with a new rule to scan for Axis in
>>>>>>>>> particular, how can we discover these unsupported libraries up
>>>>>>>>> front on future applications?".
>>>>>>>>>
>>>>>>>>> -Rodney
>>>>>>>>>
>>>>>>>>> On 2 Mar 2016, at 18:05, Robb Greathouse wrote:
>>>>>>>>>
>>>>>>>>>> We should add AXIS to the blacklist.  Do the packages in Axis have
>>>>>>>>>> a
>>>>>>>>>> signature (such as .axis.) that we could add to the Catch-All
>>>>>>>>>> rules?
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 2, 2016 at 4:12 PM, Rodney Russ <rruss at redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I believe the conclusion we came to on irc earlier today was
>>>>>>>>>>> that these
>>>>>>>>>>> types of situations are what we need the field to contribute
>>>>>>>>>>> back to the
>>>>>>>>>>> Windup project through:
>>>>>>>>>>>
>>>>>>>>>>> 1) creating rules themselves
>>>>>>>>>>> 2) creating a JIRA in the WINDUPRULE project
>>>>>>>>>>> 3) providing feedback through the link in the reports
>>>>>>>>>>>
>>>>>>>>>>> Does this seem like a reasonable approach?  I'm not sure there
>>>>>>>>>>> is anything
>>>>>>>>>>> we can realistically do to identify all unsupported libraries
>>>>>>>>>>> unless
>>>>>>>>>>> someone is aware of a comprehensive list.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -Rodney
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 29 Feb 2016, at 0:50, Tobias Hartwig wrote:
>>>>>>>>>>>
>>>>>>>>>>> See below -
>>>>>>>>>>>> Begin forwarded message:
>>>>>>>>>>>>
>>>>>>>>>>>> From: Benjamin Meiseles <mojo-notify at redhat.com>
>>>>>>>>>>>>> Date: 26. Februar 2016 um 20:58:30 MEZ
>>>>>>>>>>>>> To: Tobias Hartwig <thartwig at redhat.com>
>>>>>>>>>>>>> Subject: [JBoss Migration Community of Practice] New message:
>>>>>>>>>>>>> "JBoss EAP
>>>>>>>>>>>>> Migration: discovering unsupported application libraries"
>>>>>>>>>>>>> Reply-To:
>>>>>>>>>>>>> jive-1245489557-6qh-2-kv7c at redhatinc.hosted.jivesoftware.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mojo
>>>>>>>>>>>>>
>>>>>>>>>>>>> JBoss EAP Migration: discovering unsupported application
>>>>>>>>>>>>> libraries
>>>>>>>>>>>>> created by Benjamin Meiseles in JBoss Migration Community of
>>>>>>>>>>>>> Practice -
>>>>>>>>>>>>> View the full discussion
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello all,
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On a migration project at The Hartford, Satish Irrinki, Guy
>>>>>>>>>>>>> Bianco and I
>>>>>>>>>>>>> ran into an issue in which our Support Relationship Manager
>>>>>>>>>>>>> luckily spotted
>>>>>>>>>>>>> our use of an unsupported library (Axis 1.4). We went through
>>>>>>>>>>>>> a minor
>>>>>>>>>>>>> ordeal breaking this news to the customer and working to
>>>>>>>>>>>>> maintain our
>>>>>>>>>>>>> project deadlines without putting the application at risk. We
>>>>>>>>>>>>> were under
>>>>>>>>>>>>> the impression that Windup would have flagged any unsupported
>>>>>>>>>>>>> libraries in
>>>>>>>>>>>>> the application, but there was no trace of Axis in the report,
>>>>>>>>>>>>> so this came
>>>>>>>>>>>>> as a surprise to us.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Aside from updating Windup with a new rule to scan for Axis in
>>>>>>>>>>>>> particular, how can we discover these unsupported libraries up
>>>>>>>>>>>>> front on
>>>>>>>>>>>>> future applications?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Our initial thought is to couple our automated Windup analysis
>>>>>>>>>>>>> with a
>>>>>>>>>>>>> manual dive into the application, and ensure that all
>>>>>>>>>>>>> libraries are on the
>>>>>>>>>>>>> list of supported configurations. I am curious if anyone else
>>>>>>>>>>>>> can suggest
>>>>>>>>>>>>> alternate approaches.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ben Meiseles
>>>>>>>>>>>>>
>>>>>>>>>>>>> Reply to this message by replying to this email, or go to the
>>>>>>>>>>>>> message on
>>>>>>>>>>>>> Mojo
>>>>>>>>>>>>> Start a new discussion in JBoss Migration Community of
>>>>>>>>>>>>> Practice by email
>>>>>>>>>>>>> or at Mojo
>>>>>>>>>>>>> Following JBoss Migration Community of Practice in these
>>>>>>>>>>>>> streams: Email
>>>>>>>>>>>>> Watches
>>>>>>>>>>>>> Put Mojo in your pocket! Get Jive for iOS or Jive for Android.
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Robb Greathouse
>>>>>>>>>> Middleware BU
>>>>>>>>>> 505-507-4906
>>>
>>> --
>>> Marek Novotny
>>> --
>>> Windup team member and Seam Project Lead
>>>
>>> Red Hat Czech s.r.o.
>>> Purkynova 99
>>> 612 45 Brno
>>
>>
>>
>> --
>> Regards,
>> Jason Shepherd
>> Product Security
>>
>>
>
>