CentOS - Apr 2016 - CentOS-announce Digest, Vol 134, Issue 9

Send CentOS-announce mailing list submissions to
	centos-announce at centos.org

To subscribe or unsubscribe via the World Wide Web, visit
	https://lists.centos.org/mailman/listinfo/centos-announce
or, via email, send a message with subject or body 'help' to
	centos-announce-request at centos.org

You can reach the person managing the list at
	centos-announce-owner at centos.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of CentOS-announce digest..."


Today's Topics:

   1. *correction* Notification of outage : mirrorlist.centos.org
      (ipv4) 2016-04-16 08:58 UTC (Karanbir Singh)


----------------------------------------------------------------------

Message: 1
Date: Sat, 16 Apr 2016 23:16:12 +0100
From: Karanbir Singh <kbsingh at centos.org>
To: centos-announce at centos.org
Subject: [CentOS-announce] *correction* Notification of outage :
	mirrorlist.centos.org (ipv4) 2016-04-16 08:58 UTC
Message-ID: <5712B9AC.10600 at centos.org>
Content-Type: text/plain; charset=windows-1252

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Apologies for this, the correct date should be : 2016-04-16 Sat. The
timestamps remain accurate.


On 16/04/16 10:09, Karanbir Singh wrote:
> Hi,
> 
> At 05:00 UTC 2016-05-16 an automated component update caused the 
> mirrorlist service for CentOS to go down for all IPv4 based
> services. The IPv6 service for mirrorlist.centos.org was
> unaffected.
> 
> By 08:11 UTC 2016-05-16 I rolled back the impacted components, 
> disabled the update mechanics, and restarted services.
> 
> By 08:15 UTC 2016-05-16 Services were returning to normal.
> 
> By 08:23 UTC 2016-05-16 we had multiple confirmations from around
> the world that services were restored.
> 
> --------------- system wide followup:
> 
> - We will work to extend and add where needed, better testing
> around each component involved in such roles.
> 
> - I will work with Fabian and make sure that all our automated 
> component and system changes that impact a production service are
> only run during regular working hours for the team.
> 
> --------------- root cause:
> 
> all the components involved in the downtime are backed up and we
> will start looking at the root cause of why services went down, for
> now the immediate focus was to restore services which was done by a
> rollback.
> 
> --------------- reporting issues:
> 
> Note that for real time, time sensitive issues always drop into 
> #centos-devel on irc.freenode.net and let us know, along with
> filing a bug report at bugs.centos.org - For any non time critical
> issues, please report them at bugs.centos.org against the
> 'Infrastructure' project, and we will aim to address them as soon
> as possible.
> 

- -- 
Karanbir Singh, Project Lead, The CentOS Project
+44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS
GnuPG Key : http://www.karan.org/publickey.asc
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJXErmsAAoJEI3Oi2Mx7xbtJdgIAI1+4KTYCdI4O+kFlrJ5IlMy
TwFGYt7O1WyGDfXC27VLehDxNP0lFiZOd1TMMSWy0/P64TYcBr5GLa4Vc7WLpLef
ywgjvgNE9VY69z4D1Lkgdxlgnhg31tPtW8kA1Py24nKrEFMYYi7/2tUGZn43kE8D
SRn0CywFizCFIScB0xI7uSJRE2PzFFwixIG4LaEdqppxcId6hsKb80hzpIP35pPU
eXlagl5CMad2XD0aVqKrqoU5W9vS9swKv9rc6ASsUHB28W7KcT0mq0YsIojhS81L
w8hiSsTpW3b62uBb55MIvkg1Y11H0qlkW306ZaZAXjTpRsvNaz82+qRmgcVVJvc=Uvpc
-----END PGP SIGNATURE-----


------------------------------

_______________________________________________
CentOS-announce mailing list
CentOS-announce at centos.org
https://lists.centos.org/mailman/listinfo/centos-announce


End of CentOS-announce Digest, Vol 134, Issue 9
***********************************************