Secure testing team - Jun 2007 - Mini-meeting at DebConf - minutes

Hi all,

At a ''working lunch'' during DebConf about security in Debian,
we came up with
the following bits and pieces. Hopefully this should help get a quicker and
more efficient response to security in Debian.

Increase the visibility of testing security
-------------------------------------------
It was pointed out (I think by me) that a lot of the work isn''t
visible, as it
often involves making sure things get pushed through unstable, rather than
issuing a DTSA. This is especially true at release time.

One possibility would be to use debsecan to automatically work out what has
been moved where, and generate reports based on this. However, the number of
false positives, and email format of debsecan needs improvement before this can
occur.

More bugs should be filed against packages with open security vulnerabilities.
It seems that the maintainers are often (somehow) unaware that their package
has an open issue.

For unstable fixes, we should just NMU with no delay or notification (making
sure that it''s just the security fix though) to ensure that migration
can start
to happen as soon as possible.

How to improve the tracker[0]
-----------------------------
Public submission to the tracker would help a lot, we lack the current
man-power to effectively track everything. Part of the problem involved with
this is the way the advisory and code system is in the same repository as the
tracking data. This should be split up.

It would be nice if the tracker data was somehow integrated with the PTS as
well.

How to get maintainers to help us
---------------------------------
It seems that maintainers are a little confused as to what should happen if
there''s a security issue, mostly due to the documentation being out of
date in
various places (dev-ref etc). It was also pointed out that the vast majority of
security uploads should be done with a high priority. See [Documentation]
below.

An additional NM question could be introduced, along the lines of "How long
should you support your package for security issues?" as some maintainers
seem
to be resistant to fixing bugs in their packages that no longer exist in
unstable.

Lenny plans
-----------
Some packages were released with etch that are very hard to support by the SST.
It would be good to try and avoid these getting into testing or even unstable
all together.

Embedded code copies are still causing problems. There''s a policy bug
open
(http://bugs.debian.org/392362), and Neil McGovern will ask the RMs to make
''no
embedded code copies'' a release goal.

Documentation
-------------
Documentation should be written/updated for the following groups:
Maintainers: how to be nice to the security team.
Tracker submitters: how to help us (one for DDs, and one for non-DDs).
dev-ref: it could do with a lot of improvement to reflect current practice.

A ''bits from the security team'' to d-d-a may be useful once
this is done.

Disto agnostic patch/tracking/magic system/conference/mad-idea
--------------------------------------------------------------
A mad idea was formed to look at the possibility of creating a more harmonised
approach to security in Linux generally. A patch repository and/or tracking
system that could be used for all distributions may be useful. This may help
create a more unified approach to security, and stop work being duplicated.

Additionally, a global Linux distro security teams summit may be something that
would help kick-start this approach.


I think this is fairly accurate. Any comments?

Cheers,
Neil
[0] http://security-tracker.debian.net/
-- 
<hermanr> 10 people enough for a Debconf?  If they were all Germans,
maybe...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url :
http://lists.alioth.debian.org/pipermail/secure-testing-team/attachments/20070625/4005dde5/attachment.pgp

Hi all,

my name is Florian Maier and i'' m the responsible security guy for the
City of Munich''s "LiMux" distribution, which is based on
Debian Sarge
with lots of backports and will be migrated to Debian Etch this autumn.

I had the chance to talk to Stefan Fritsch and Micah Anderson (i was the
Florian Weimer double ;-) at DebConf and offered to help out with Debian
security, or testing-security.

Things like checking CVE''s on a regular basis are already a part of my
daytime job and of course i''m motivated because i want to give back
something to Debian, the distribution without which everything in Munich
would be different.

As i''m (at least at the moment) a non-DD, writing the documentation for
Tracker submitters (Neil McGovern mentioned it in his last E-Mail) could
be a good place to start.

But what do you guys think?

Regards,


Florian

Florian Maier on 2007-06-26 22:55:34 +0200:
> Things like checking CVE''s on a regular basis are already a part
of my
> daytime job and of course i''m motivated because i want to give
back
> something to Debian, the distribution without which everything in Munich
> would be different.
>
> As i''m (at least at the moment) a non-DD, writing the
documentation for
> Tracker submitters (Neil McGovern mentioned it in his last E-Mail) could
> be a good place to start.
I''m not a DD.  My suggestion is to grab a copy of the tracker data from
svn [0], pop open a list of items that still need work [1], and knock
off a few.  The tracker data is generated from data/CVE/list in svn; if
you start sending good patches, chances are people will get tired of
applying them and one of the Alioth project admins will give you commit
access.  Most of the CVE issues aren''t relevant to Debian, and we mark
them as such, but when you find a real one, submit a bug to get the ball
rolling.  Maintainers love patches and test cases.

[0] http://svn.debian.org/wsvn/secure-testing/data
[1] http://security-tracker.debian.net/tracker/status/todo

Well, I couldn''t make it to debconf, but I probably should contribute
a few notes anyway.

Status of the tracker software
------------------------------

As most of you probably know, the web service
(<http://security-tracker.debian.net/>,
<http://idssi.enyo.de/tracker/>)
works by watching for Subversion commits and updates its internal
database accordingly.  The data used by debsecan to display
vulnerability information is computed as well.

Processing a single commit currently takes more than ten minutes.
Incorporating Package and Source file updates takes even longer.  It''s
not necessary to process every commit individually, that''s why the
system can keep up with the commits.  I suppose that switching to
faster hardware could bring it down to 40 seconds or something like
that, which still isn''t acceptable IMO.  (This number is an estimate
for top-of-the-line amd64 hardware with oodles of RAM, just to put
that number into perspective.) It makes editing the data files
unnecessarily hard if you can''t run this tool locally--even though the
plain syntax check performed by "make check" is quick.

Part of the problem is that all data is pre-computed.  One obvious
change would be to pre-compute the data for the overview pages (for
unstable/testing/stable) only, and recompute the per-package
vulnerability status as needed for the per-package/per-bug pages.
However, the logic which decides which package is vulnerable and which
is not is a series of SQL statements, not Python code[*], which makes
this somewhat difficult.  It would also help a lot if the code used
hashes to detect which data/CVE/list entries have actually changed,
and re-compute only the affected package and bug data.  Again, this
appears to be difficult to implement in the current framework.

[*] Actually, there are two different SQL implementations and two
    Python implementations (one of them in debsecan), if I recall
    correctly.  It seems that they are mostly in sync, but it''s not a
    nice design.

I''ve experimenting with a GUI editor for data/CVE/list recently which
simplifies creating NOT-FOR-US entries.  The Bayesian learning module
didn''t work as well as I had hoped, but I still think that such a tool
can be helpful.  If actually used, it would streamline the process of
rating CVEs and avoid stupid mistakes (like missing JFFNMS when
there''s actually a package called "jffnms").  This still
needs
signifcant amounts of work, though.  The current prototype depends on
non-free software; I also used it as an exercise to become familiar
with this particular way of GUI development.  It might also make sense
to re-cast this as a web application.  Whatever we do, we should share
the infrastructure code (file parsing etc.) with the code that feeds
the database.

Right now, I fear that it''s not feasible for anyone except me to work
on the software.  But my next step will be to document the actual
algorithm used to derive exact per-bug, per-package vulnerability data
from the data/CVE/list.  Unfortunately, it''s a bit of black art in a
some corners.  This document will also make clear why addressing bug
#357942 is so hard, and perhaps even show what changes are needed for
fix.  Based on that documentation, it should be possible to change the
Python code, or rewrite it from scratch, if anyone is actually
interested.  I also plan to improve matters by writing actual code,
but the amount of time I can put into this depends on a factor I can
only partially contorl (there''s one fairly important hobby project of
mine which I really need to bring forward, but my employer might
actually developer resources at that one, so I wouldn''t have to do it
in my spare time, and all by myself).

No progress whatsoever has been made on the "document the likely level
of security support packages will receive" front, I''m afraid.

Hi Florian,

On Dienstag, 26. Juni 2007, Florian Maier wrote:
> As i''m (at least at the moment) a non-DD, writing the
documentation
> for Tracker submitters (Neil McGovern mentioned it in his last
> E-Mail) could be a good place to start.
yes, I think trying to do some outstanding TODO items and then 
updating and extending the documentation [1] accordingly would be a 
good idea.

Some hints
- you should subscribe to the commits list [2]. Error messages will be 
sent there and you can get a feeling how things work
- you can check the syntax with "make check" in the top level svn 
directory
- there is a script bin/check-new-issues that helps a bit with sorting 
out NOT-FOR-US issues
- joining #debian-security on irc.debian.org is probably also a good 
idea
- the thread at [3] might be interesting, too

If you have questions, just ask.

Cheers,
Stefan

[1] 
http://svn.debian.org/wsvn/secure-testing/doc/narrative_introduction?op=file&rev=0&sc=0

[2]  
http://lists.alioth.debian.org/mailman/listinfo/secure-testing-commits

[3] 
http://lists.alioth.debian.org/pipermail/secure-testing-team/2007-January/001023.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url :
http://lists.alioth.debian.org/pipermail/secure-testing-team/attachments/20070627/eb296fbb/attachment.pgp

Florian Maier wrote:
> my name is Florian Maier and i'' m the responsible security guy for
the
> City of Munich''s "LiMux" distribution, which is based on
Debian Sarge
> with lots of backports and will be migrated to Debian Etch this autumn.
> 
 Things like checking CVE''s on a regular basis are already a part of
my
> daytime job and of course i''m motivated because i want to give
back
> something to Debian, the distribution without which everything in Munich
> would be different.
There are two things of special interest to Debian:

- Verify the Sarge status of vulnerabilities:
http://idssi.enyo.de/tracker/status/release/oldstable?hide_nodsa=1 

They are derived from the unstable data and should be checked/verified
if really all of these apply to Sarge. (e.g. sometimes older versions
don''t include vulnerable code)

- In the short/mid-term I''m planning to work on a better QA process
with more
external participants. There''s a delay of up to a couple of days
between
the time, when a package is initially built and the release of the fixed
package. Large installations like Munich could receive the packages prior
to release and provide testing/QA feedback in return. So, participating
in this would be very much appreciated once the infrastructure is in place.
(This would be limited to publicly known vulnerabilities, which is > 80%)

Cheers,
        Moritz

Moritz Muehlenhoff wrote:
> There are two things of special interest to Debian:
>
> - Verify the Sarge status of vulnerabilities:
> http://idssi.enyo.de/tracker/status/release/oldstable?hide_nodsa=1 
>
> They are derived from the unstable data and should be checked/verified
> if really all of these apply to Sarge. (e.g. sometimes older versions
> don''t include vulnerable code)
>
> - In the short/mid-term I''m planning to work on a better QA
process with more
> external participants. There''s a delay of up to a couple of days
between
> the time, when a package is initially built and the release of the fixed
> package. Large installations like Munich could receive the packages prior
> to release and provide testing/QA feedback in return. So, participating
> in this would be very much appreciated once the infrastructure is in place.
> (This would be limited to publicly known vulnerabilities, which is >
80%)
>   
Sounds good. Although we do only use a small subset of the sarge
repositories, i can definitely do this. A collaboration would be very
worthwile for all parties involved!

Is there already a certain timeframe you are thinking of?

Regards,


Florian

On Fri, Jun 29, 2007 at 12:01:15AM +0200, Florian Maier
wrote:
> Moritz Muehlenhoff wrote:
> > There are two things of special interest to Debian:
> >
> > - Verify the Sarge status of vulnerabilities:
> > http://idssi.enyo.de/tracker/status/release/oldstable?hide_nodsa=1 
> >
> > They are derived from the unstable data and should be checked/verified
> > if really all of these apply to Sarge. (e.g. sometimes older versions
> > don''t include vulnerable code)
1)

> > - In the short/mid-term I''m planning to work on a better QA
process with more
> > external participants. There''s a delay of up to a couple of
days between
> > the time, when a package is initially built and the release of the
fixed
> > package. Large installations like Munich could receive the packages
prior
> > to release and provide testing/QA feedback in return. So,
participating
> > in this would be very much appreciated once the infrastructure is in
place.
> > (This would be limited to publicly known vulnerabilities, which is
> 80%)
2)
> Sounds good. Although we do only use a small subset of the sarge
> repositories, i can definitely do this. A collaboration would be very
> worthwile for all parties involved!
> 
> Is there already a certain timeframe you are thinking of?
You can start with 1) right-away, 2) will need some infrastructure improvements,
which will likely take a couple months.

Cheers,
        Moritz