[windup-dev] Fwd: Victims Java API, data, features

[Ondrej Zizka]

Hi Jason,

Ondrej Zizka, Red Hat Migration Toolkit
On 12.6.2017 01:30, Jason Shepherd wrote:
> Hi Ondrej,
>
> Sorry for the late reply on this. Stephen and I have been discussing 
> the Victims project lately and I realised I hadn't forwarded his 
> feedback to him, so please see his replies to your feedback below.
>
> However I think we are going to refocus our efforts a bit on Victims. 
> OWASP dependency check has become a very popular project for 
> vulnerability tracking. It's being used by Fabric8 for vulnerability 
> scanning in Openshift.IO at the moment. The OWASP project is willing 
> to add Victims as a datasource, so I think we should focus our efforts 
> on that in order to get their features, and also have some influence 
> on the data in that tool.
Good news!
>
> One thing that the community are asking for is a change in licence for 
> the victims-cve-db part of the project, so that's something that we'll 
> definitely we looking at. Some members have suggested a CC BY-SA 
> licence. What do you think of that? Read the discussion here:
>
> https://github.com/victims/victims-cve-db/issues/25
I don't know much about licenses, maybe Marek will be able to tell more. 
I remember Windup would have a problem with the Java client lib being 
licensed under AGPL. Eclipse license would fit.
Regarding the db part and CC BY-SA, I guess someone (Tobias?) would have 
to consider.

Ondra
>
> Regards,
> Jason Shepherd
> Product Security
>
> ---------- Forwarded message ----------
> From: *Stephen Milner* <smilner at redhat.com <mailto:smilner at redhat.com>>
> Date: Sat, Jun 10, 2017 at 4:45 AM
> Subject: Re: Victims Java API, data, features
> To: Jason Shepherd <jshepher at redhat.com <mailto:jshepher at redhat.com>>
>
>
> Replying back to you with details. In the response please do loop my 
> address :-)
>
> Inline ...
>
> On Fri, Jun 9, 2017 at 2:04 AM, Jason Shepherd <jshepher at redhat.com 
> <mailto:jshepher at redhat.com>> wrote:
> >
> > ---------- Forwarded message ----------
> > From: Ondrej Zizka <ozizka at redhat.com <mailto:ozizka at redhat.com>>
> > Date: Wed, Jun 1, 2016 at 3:09 AM
> > Subject: Victims Java API, data, features
> > To: Jason Shepherd <jshepher at redhat.com 
> <mailto:jshepher at redhat.com>>, Windup-dev List
> > <windup-dev at lists.jboss.org <mailto:windup-dev at lists.jboss.org>>
> >
> >
> >
> >
> >
> > Hi Jason,
> >
> > (I'm seding 2nd mail to start a new thread, please ignore the 
> previous one.)
> >
> > I have looked closer at Víctims.
> > I have few questions/issues. Could you please help resolving those?
> >
> > Note: I'm adding a PUBLIC mailing list, Windup developers. Feel free 
> to add
> > some Victims list (is there one?)
> >
> > 1) Hashes are not real checksums
> > As someone wrote in 
> https://github.com/victims/victims-cve-db/issues/45 
> <https://github.com/victims/victims-cve-db/issues/45>
> > the hashes used by Victims are not just SHA512 hashes of the file 
> content,
> > but something else.
> > I'd like to be able to either find CVE's by a normal file content 
> hash, or
> > create the Victims hash.
>
> That's a fair request. For some background, the reason we recreate a
> specific hash
> is that different Java implementations create different bytecode,
> resulting in different
> hashes. Our hash creator strips out implementation specific items for 
> creating
> and scanning.
>
> > a) Is there a Java impl?
>
> Client side there is via https://github.com/victims/victims-lib-java 
> <https://github.com/victims/victims-lib-java>
>
> > b) Could you add the plain SHA512 (or other, I'm okay with just 
> CRC32) hash
> > to the data?
>
> We could do so. I assume this would be the SHA512 of the vulnerable 
> jar file.
>
> > 2) Victims Java client API
> >
> > The Java API doesn't match the needs much.
> >  From what I can see, it can
> >    a) Sync with the server
> >    b) Give me a list of CVE for given SHA512 hash.
> >
> > What I would like to have is:
> > * Have some offline data distributed with our app, provide these data
> > * Search the database by Maven coordinates, classes,
> > * Get a short description of the CVE and date of appearance and 
> how/where it
> > was fixed
> >
> > Is there a plan for extending the Java API?
> > Also I guess not all these are covered in the Victims database, right?
>
> You're correct. There is a disconnect between the victims-cve-db and 
> the hash
> database. Folks have been pretty great at submitting items to the 
> victims-cve-db
> but we've gotten very little submissions for the hash db. Part of me 
> wonders
> if it would be more beneficial to combine the two in the 
> victims-cve-db. Syncing
> would then be a ``git pull`` rather than API call. It would also let
> people do PR's
> for data inclusion which may be more submitter friendly. Thoughts?
>
> > 3) Configuration
> > The configuration is done through system properties, that's not too
> > fortunate.
> > For instance it doesn't allow to run multiple clients at once in the 
> same
> > JVM.
> > Could that be done through an API?
>
> I don't see why not. However, I think we would need some help to do that.
>
> > 4) Data structure
> > The data structure of the JSON is not obvious. Is there some docs 
> for it?
>
> No, but there should be. Here is some pointers I threw together:
>
> https://gist.github.com/ashcrow/df238b1cc1e8a2f4bba94a6bb310805e 
> <https://gist.github.com/ashcrow/df238b1cc1e8a2f4bba94a6bb310805e>
>
> > 5) Data storage
> > The data are only stored in a database over JDBC. Could it be simply 
> stored
> > in a JSON or XML file? The file is just 165 KB and not growing too 
> fast, so
> > I think rather than bringing an embedded DB as a dependency, I'd 
> prefer to
> > process a XML file into a HashMap or a Lucene index and use that.
>
> I added a possible replacement at
> https://gist.github.com/ashcrow/df238b1cc1e8a2f4bba94a6bb310805e 
> <https://gist.github.com/ashcrow/df238b1cc1e8a2f4bba94a6bb310805e>.
> Essentially we'd move to a yaml format which is a combined version of
> the victims-cve-db and the hash
> database (which currently sits behind the api). Instead of syncing
> with the API one would sync via git
> and pull down the latest changes. PTAL and let me know what you think.
>
>
> > On 4.4.2016 02:16, Jason Shepherd wrote:
> >>
> >> Hi Ondra,
> >>
> >> The architecture of Victims is such that you should never need to
> >> 'download' the database. The client is designed to connect to the
> >> central http://victi.ms API to get the latest vulnerabilities.
> >>
> >> That being said, the authors also have a 'backup' of the data in the
> >> form of a Github repository, [1]. In fact some members of the
> >> community have built a tool which just uses this repository, and does
> >> not use the API at all. Recently we've built a tool to rebuild the
> >> database from the Github repository, but it still needs some work,
> >> [3].
> >>
> >>     [1] https://github.com/victims/victims-cve-db 
> <https://github.com/victims/victims-cve-db>
> >>     [2] https://github.com/h3xstream/maven-security-versions 
> <https://github.com/h3xstream/maven-security-versions>
> >>     [3] https://github.com/jasinner/victims-db-builder 
> <https://github.com/jasinner/victims-db-builder>
> >>
> >> Let me know if you need any further information.
> >> Regards,
> >> Jason Shepherd
> >>
> >> On Fri, Apr 1, 2016 at 1:38 AM, Ondrej Zizka <ozizka at redhat.com 
> <mailto:ozizka at redhat.com>> wrote:
> >>>
> >>> Great to know it goes on, last time I talked to someone (I think 
> djorm),
> >>> he
> >>> said the development was stagnant.
> >>>
> >>> Jason, is there a way to download a single big file with all data 
> in the
> >>> database?
> >>>
> >>> Thanks,
> >>> Ondra
> >
> >
>
>
>
> --
> Thanks,
> Steve Milner
>
> Atomic | Red Hat | http://projectatomic.io/ | http://commissaire.io
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/windup-dev/attachments/20170612/ecd0ddbb/attachment.html