freebsd security - May 2007 - (Security Regression Testsuites)Request for comments

Dear All,
   I am a student enrolled google summer code 2007. My job is to write
security regression testsuites for FreeBSD under the guidance of my mentor
Dr. Robert Watson. Under his encourage, I write following request for comments
RFC :-) 
//////////////////////////////////////////////////////////////
   What I plan to do:
1) to test the stability of Mandatory Access Control and Audit Subsystem
for FreeBSD and TrustedBSD.
Backgroud: 
  a) there are many other modules in FreeBSD such as PF?IPFW and 
IPSec and VIMAGE have had ignored the existance of Mandatory Access Control,
they
generate mbuf without a tag for Mandatory Access Control. Many of these has been
corrected.
  b) The audit subsystem's handling of auditing disk full is wrong in
locking
vnodes

2) to test the correct enforement of various of access control (Mandatory Access
Control,
ACL, and priviledges in jail).
Goal:
   To prevent the access right violation of the designer's intension

3) the consistency between the Mandatory Access Control Label generated by
userland
application and the label kernel actually handles.


4) to test the various of Firewalls and IPSec

///////////////////////////////////////////////////////////////
   What I have done:
1) investigate the Linux Test Project, especially for SeLinux
2) investigate the stress2 package for FreeBSD
3) summary the reason and the settlement of the confliction between Mandatory
Access Control
and PF, IPFW, IPSEC and VIMAGE
4) write a pair of pseudo ethernet pairs following the idea of another Socer Dr.
Nanjun Li and
Oreilly's <Linux Device Driver>, so that the network tests can be done
in a single machine

///////////////////////////////////////////////////////////////
   Where I am still confused:
1) Which area and direction should I focus. The security subsystem in FreeBSD is
large, which
area deserves a testsuite in higher priority.
2) The general structure of the testsuite:
Will it be a userland application package like stress2, or include a kernel
module cooperation
(like security/mac_test)
3) How to write a testsuite that will prevent the furthor violation of security
instead of test
the cases which are already corrected.
PF?IPFW and IPSec have already corrected their confliction with Mandatory Access
Control, I think
the testcases for the already corrected problems will not discover the newly
generated problems,
for example: 
test case  for the PF's synproxy state rule only verify PF have correctly
add a correct tag for Mandatory
access control in function pf_send_tcp, how we discover a problem which may
create in the future
by means of create a mbuf without a correct tag for Mandatory access control in
a new function?

///////////////////////////////////////////////////////////////////

Finally 
I owe greatly thanks for
various kind of suggestions not limited to above

Sincerely yours
Zhouyi Zhou
Insitute of Software
Chinese Academy of Sciences

On Tue, 29 May 2007, zhouyi zhou wrote:
>  Where I am still confused:
>
> 1) Which area and direction should I focus. The security subsystem in 
> FreeBSD is large, which area deserves a testsuite in higher priority.
Off-hand, my feeling is I'd like us to consider three areas of testing:

- Correctness testing, in which we test general and edge cases in access
   control to make sure both the infrastructure and policies operate correctly.
   For example, using test policies to validate that the MAC Framework is
   correctly implemented, and using tests on real policies to determine that
   the real policies implement the desired access control.  Likewise, for
   audit, making sure that the right records are generated with the right
   tokens for the right events, that the selection model works, etc.

- Stability testing, in which we test general and edge cases to make sure the
   system is stable, especially under load, etc.

- ABI/API life cycle testing, in which we confirm that key data structures in
   the ABI remain stable over time, and that the interpretation remains
   consistent.  For example, the persisting data structures in on-disk labels,
   audit record formats, etc.

Obviously, doing all this in one summer is well out of scope, but I think some 
useful in-roads can be made in testing key areas, such as making sure that 
file system protections with MLS and Biba are correct, tests for audit, and so 
on.  You may want to look at Pawel's and my existing file system test tools 
(src/tools/regression in 7-CURRENT) to see some areas and approaches to 
testing.
> 2) The general structure of the testsuite: Will it be a userland
application
> package like stress2, or include a kernel module cooperation (like 
> security/mac_test) 3) How to write a testsuite that will prevent the
furthor
> violation of security instead of test the cases which are already
corrected.
> PF?IPFW and IPSec have already corrected their confliction with Mandatory 
> Access Control, I think the testcases for the already corrected problems 
> will not discover the newly generated problems, for example: test case for 
> the PF's synproxy state rule only verify PF have correctly add a
correct tag
> for Mandatory access control in function pf_send_tcp, how we discover a 
> problem which may create in the future by means of create a mbuf without a 
> correct tag for Mandatory access control in a new function?
I would suggest starting with a small set of test projects to evaluate 
approaches.  For example:

(1) Consider adding a new test policy couple with userland tools to make sure
     that access control checks occur as required for each system call.

(2) Add a set of user space tests that confirm that MLS is properly
     implemented for each system call.

(3) Add a set of user space tests that confirm that, for each system call, the
     right audit records are generated with the right tokens.

(4) Add a set of user space tests to confirm that audit record preselection is
     properly implemented.

These are a bit more bounded in scope, and should start to bring out common 
aspects to testing across security functions (i.e., "foreach system
call").

Robert N M Watson
Computer Laboratory
University of Cambridge