Author: fw Date: 2010-05-07 18:51:15 +0000 (Fri, 07 May 2010) New Revision: 14624 Added: doc/python-format.txt Log: Add documentation for the upcoming external data formats Added: doc/python-format.txt ==================================================================--- doc/python-format.txt (rev 0) +++ doc/python-format.txt 2010-05-07 18:51:15 UTC (rev 14624) @@ -0,0 +1,116 @@ +NOTE: THIS DOES NOT DESCRIBE THE CURRENT IMPLEMENTATION + +# Layout of major internal data structures + +Most data structures use named tuples, as provided by +xcollections.namedtuples (they are not available in Python 2.5, but +the implementation from Python 2.6 works on Python 2.5, too). + +Due to the way unpickling works, you need to import the "parsers" +package. + +The data structures described here are supposed to be fairly stable, +except for the addition of additional attributes and changes in the +internal order of named tuples (so you really should not rely on +that). + +# Individual bug information + +The data/*/list files are parsed as lists of bugs. A line which does +not start with whitespace is called a "header", and the following +intended lines are called "annotations". + +The top-level named tuple contains two elements: + +* list: the list of bug objects (see below) + +* messages: the list of messages from the parser (see below) + +All lists are sorted by file position of the contained objects. + +## bug objects + +* bug.file: path to the file containing this bug + +* bug.header: header object (see below) + +* bug.annotations: list of all annotations of this bug (see below) + +## header objects + +* header.line: line number + +* header.name: bug name (auto-generated for temporary issues) + +* header.description: string, can be empty or None + +## message objects + +* msg.file: file name + +* msg.line: line number + +* msg.level: "error" or "warning" + +* msg.contents: free-text message + +## Errors + +## annotation objects + +All annotation objects have these fields: + +* ann.line: the line number + +* ann.type: code value to determine the structure + +Additional fields are described below, depending on the ann.type +value. + +### types "NOT-FOR-US", "NOTE", "TODO" + +* ann.description: user-supplied string + +### types "RESERVED", "REJECTED" + +These act just as flags; no additional data is present. + +### type "xref" + +* ann.bugs: list of bugs being referenced + +### type "package" + +* ann.release: applicable release, or None for unstable + +* ann.package: name of the package + +* ann.kind: one of "fixed" (version was supplied), "unfixed", "removed", + "itp", "no-dsa", "not-affected", "undetermined" + +* ann.version: fixed version number, or "None" for unfixed/not applicable + +* ann.urgency: one of None, undetermined, low, medium, high + +* ann.debian_bugs: set of numbers of Debian bugs + +* ann.description: free-text information, or None if not applicable + +# Derived vulnerability information + +These are contained in a list of info objects: + +* info.bug: name of the bug (potentially auto-generated) + +* info.package: name of the package + +* info.fixed: fixed version in unstable (a string), or None (no fix + available) or True (all versions fixed) + +* info.fixed_other: a tuple, containing other fixed versions (which + are less than the unfixed unstable version, but nevertheless known + not to be vulnerable) + +In itself, this data is not very illuminating, but comparision with +other information sources can be used to detect vulnerable installed +packages, generate bug and distribution overview pages etc.