CentOS - Nov 2015 - After reboot of web-server accessing website shows "Forbidden", restarting httpd all is fine

Hi.

I am stuck with this one and I do not know where and how to search for this
problem nor do I know how to fix it.

When I reboot one of our servers (CentOS 6.7, selinux target, yum fully updated)
the http server loads fine (no erros) but when accessing one of the server's
websites it displays "Forbidden", restarting the httpd server (command
line) will give full access and all is fine.

What troubles me that a simple restart of the daemon fixes everything but it
does not come up on reboot.

[Sat Nov 07 13:02:44 2015] [notice] caught SIGTERM, shutting down
[Sat Nov 07 13:02:45 2015] [notice] SELinux policy enabled; httpd running as
context unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[Sat Nov 07 13:02:45 2015] [notice] suEXEC mechanism enabled (wrapper:
/usr/sbin/suexec)
[Sat Nov 07 13:02:45 2015] [notice] Digest: generating secret for digest
authentication ...
[Sat Nov 07 13:02:45 2015] [notice] Digest: done
[Sat Nov 07 13:02:45 2015] [notice] Apache/2.2.15 (Unix) PHP/5.4.45
mod_ssl/2.2.15 OpenSSL/1.0.1e-fips mod_wsgi/3.2 Python/2.6.6 mod_perl/2.0.4
Perl/v5.10.1 configured -- resuming normal operations

I started it with an strace but I cant find anything that looks suspicious ...

Putting selinux into permissive mode starts the server right from boot.
Looking at all the logs I cannot see anything.

Any ideas?


Jobst


-- 
Though the pen IS mightier than the sword, the sword is mightier at any given
moment.

  | |0| |   Jobst Schmalenbach, jobst at barrett.com.au, General Manager
  | | |0|   Barrett Consulting Group P/L & The Meditation Room P/L
  |0|0|0|   +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia

On 11/06/2015 06:30 PM, Jobst Schmalenbach wrote:
> What troubles me that a simple restart of the daemon fixes everything but
it does not come up on reboot.
Running the service script manually may not give you the same selinux 
context as on boot.  Services should be started using "run_init" to 
ensure they get the correct context.

I think this is legitimately the most confusing aspect of SELinux, and 
it's one of the things that systemd fixed properly.

On Fri, Nov 06, 2015 at 07:23:59PM -0800, Gordon Messmer
wrote:
> On 11/06/2015 06:30 PM, Jobst Schmalenbach wrote:
> >What troubles me that a simple restart of the daemon fixes everything
but it does not come up on reboot.
> 
> Running the service script manually may not give you the same
> selinux context as on boot.  Services should be started using
> "run_init" to ensure they get the correct context.
How long has this been the case? I have never heard of this before,
it seems a very well-kept secret!
> 
> I think this is legitimately the most confusing aspect of SELinux,
> and it's one of the things that systemd fixed properly.
-- 
---- Fred Smith -- fredex at fcshome.stoneham.ma.us
-----------------------------
                      The eyes of the Lord are everywhere, 
                    keeping watch on the wicked and the good.
----------------------------- Proverbs 15:3 (niv) -----------------------------

On 11/06/2015 06:30 PM, Jobst Schmalenbach wrote:
> Putting selinux into permissive mode starts the server right from boot.
> Looking at all the logs I cannot see anything.
Which logs?  You should see AVC denies logged in 
/var/log/audit/audit.log, unless you've disabled audit logging.

The AVCs should indicate which files are labeled incorrectly, and what 
their current label is.  You probably need to fix the tree from which 
you're serving files.

Could be just "restorecon -r -v /var/www" if you're using the
default
paths.  Otherwise, you should use semanage to fix whatever paths you're 
using:

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security-Enhanced_Linux/sect-Security-Enhanced_Linux-SELinux_Contexts_Labeling_Files-Persistent_Changes_semanage_fcontext.html

sorry late reply.
thanks, will look into it.

On Fri, Nov 06, 2015 at 07:23:59PM -0800, Gordon Messmer (gordon.messmer at
gmail.com) wrote:
> On 11/06/2015 06:30 PM, Jobst Schmalenbach wrote:
> >What troubles me that a simple restart of the daemon fixes everything
but it does not come up on reboot.
> 
> Running the service script manually may not give you the same selinux
> context as on boot.  Services should be started using "run_init"
to ensure
> they get the correct context.
> 
> I think this is legitimately the most confusing aspect of SELinux, and
it's
> one of the things that systemd fixed properly.
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
-- 
Never share a foxhole with anyone braver than yourself.

  | |0| |   Jobst Schmalenbach, jobst at barrett.com.au, General Manager
  | | |0|   Barrett Consulting Group P/L & The Meditation Room P/L
  |0|0|0|   +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia

sorry, late reply.
thanks, one part was a path.

jobst


On Mon, Nov 09, 2015 at 09:40:44AM -0800, Gordon Messmer (gordon.messmer at
gmail.com) wrote:
> On 11/06/2015 06:30 PM, Jobst Schmalenbach wrote:
> >Putting selinux into permissive mode starts the server right from boot.
> >Looking at all the logs I cannot see anything.
> 
> Which logs?  You should see AVC denies logged in /var/log/audit/audit.log,
> unless you've disabled audit logging.
> 
> The AVCs should indicate which files are labeled incorrectly, and what
their
> current label is.  You probably need to fix the tree from which you're
> serving files.
> 
> Could be just "restorecon -r -v /var/www" if you're using the
default paths.
> Otherwise, you should use semanage to fix whatever paths you're using:
> 
>
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security-Enhanced_Linux/sect-Security-Enhanced_Linux-SELinux_Contexts_Labeling_Files-Persistent_Changes_semanage_fcontext.html
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
-- 
while ( !sorted ) { do_nothing ( ) ; }

  | |0| |   Jobst Schmalenbach, jobst at barrett.com.au, General Manager
  | | |0|   Barrett Consulting Group P/L & The Meditation Room P/L
  |0|0|0|   +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia