Hello, I''m running puppet on a bunch of Red Hat Enterprise Linux 5 machines. I am seeing an issue where puppetd will freeze in the middle of a run, regularly. If I run with --debug, the last line of output I see is usually a command that completes quickly, or a notification that something has succeeded - so I don''t think it''s related to the freeze. The strange thing is this problem only occurs on one class of machine that I am running puppet on - none of the other machines (which have different configurations under puppet) have this problem. I''m not sure how to get more information to debug what puppet is getting hung up on. Can anyone point me in the right direction? Thanks! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Michael DeHaan
2010-Mar-18 16:41 UTC
Re: [Puppet Users] Puppet freezes in the middle of runs
On Thu, Mar 18, 2010 at 12:36 PM, Thomas <tommyfisher@gmail.com> wrote:> Hello, > > I''m running puppet on a bunch of Red Hat Enterprise Linux 5 machines. > I am seeing an issue where puppetd will freeze in the middle of a run, > regularly. If I run with --debug, the last line of output I see is > usually a command that completes quickly, or a notification that > something has succeeded - so I don''t think it''s related to the freeze. > The strange thing is this problem only occurs on one class of machine > that I am running puppet on - none of the other machines (which have > different configurations under puppet) have this problem. > > I''m not sure how to get more information to debug what puppet is > getting hung up on. Can anyone point me in the right direction? > > Thanks! >For starters, what is the class and what versions of Puppet are you using? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Puppet is version 0.25.4 The machines are all running under similar hardware, sorry if the word "class" was misleading. The database machines are failing to run puppet, while all of the other servers (mostly application servers) have no problem. On Mar 18, 12:41 pm, Michael DeHaan <mich...@reductivelabs.com> wrote:> On Thu, Mar 18, 2010 at 12:36 PM, Thomas <tommyfis...@gmail.com> wrote: > > Hello, > > > I''m running puppet on a bunch of Red Hat Enterprise Linux 5 machines. > > I am seeing an issue where puppetd will freeze in the middle of a run, > > regularly. If I run with --debug, the last line of output I see is > > usually a command that completes quickly, or a notification that > > something has succeeded - so I don''t think it''s related to the freeze. > > The strange thing is this problem only occurs on one class of machine > > that I am running puppet on - none of the other machines (which have > > different configurations under puppet) have this problem. > > > I''m not sure how to get more information to debug what puppet is > > getting hung up on. Can anyone point me in the right direction? > > > Thanks! > > For starters, what is the class and what versions of Puppet are you using?-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Mark Plaksin
2010-Mar-18 17:42 UTC
[Puppet Users] Re: Puppet freezes in the middle of runs
Thomas <tommyfisher@gmail.com> writes:> Hello, > > I''m running puppet on a bunch of Red Hat Enterprise Linux 5 machines. > I am seeing an issue where puppetd will freeze in the middle of a run, > regularly. If I run with --debug, the last line of output I see is > usually a command that completes quickly, or a notification that > something has succeeded - so I don''t think it''s related to the freeze. > The strange thing is this problem only occurs on one class of machine > that I am running puppet on - none of the other machines (which have > different configurations under puppet) have this problem. > > I''m not sure how to get more information to debug what puppet is > getting hung up on. Can anyone point me in the right direction?I''ve seen this happen when puppet is running a command that hangs. So pstree might help you find the problem. In our case it was trying to do an NFS mount and the ''mount'' command itself was hanging. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Michael DeHaan
2010-Mar-18 17:57 UTC
Re: [Puppet Users] Re: Puppet freezes in the middle of runs
On Thu, Mar 18, 2010 at 12:45 PM, Thomas <tommyfisher@gmail.com> wrote:> Puppet is version 0.25.4 > > The machines are all running under similar hardware, sorry if the word > "class" was misleading. The database machines are failing to run > puppet, while all of the other servers (mostly application servers) > have no problem. > >Can you also check http://projects.reductivelabs.com/search/index/puppet?q=hangs&scope=&all_words=1&issues=1&submit=Submit for any open items that may apply to your case? --Michael -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
pstree doesn''t show anything interesting:
├─sshd,1354
│ ├─sshd,2008
│ │ └─bash,2010
│ ├─sshd,4942
│ │ └─bash,4944
│ │ └─puppetd,5459 /usr/sbin/puppetd --onetime --no-
daemonize --test --pluginsync --factpath=/var/lib/puppet/lib/facter --
server puppet1.sermo.dev
│ │ └─{puppetd},5505
When I do strace, however, I get the following. Seems like it''s stuck
on something:
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = 4096
read(6, "90BD46D9DB1'',''145D4600-3186-11DF"..., 4096)
= 4096
read(6, "1sL21hcnNoYWx1bm1hcnNoYWwvcGx1Z2"..., 4096) = 4096
read(6, "F89CA8A12257'',''106FB820-3186-11D"..., 4096)
= 4096
read(6, "Nlc2IiPnF1ZXVlPC9qYm9zc2VzYjpkZX"..., 4096) = 4096
read(6, "VsYzJJdWJXVnpjMkZuWlM1MGNtRnVjM0"..., 4096) = 4096
read(6, "53B15'',''0F809D30-3186-11DF-9D30-"..., 4096)
= 4096
read(6, "y'' ORDER BY M.NAME DESC,M.LAST_U"..., 4096) = 4096
read(6, "cGVydHk+PEtleT5iM0puTG1waWIzTnpM"..., 4096) = 4096
read(6, "_SERVICE_CATEGORY C WHERE C.SERV"..., 4096) = 4096
read(6, "46amJvc3MvZXNiL2Vwci90eXBlL2ptcz"..., 4096) = 4096
read(6, "((UPPER(NAME) = ''PANELSSNOOPSERV"..., 4096) = 4096
read(6, "_UPDATE FROM JUDDI_BUSINESS_SERV"..., 4096) = 4096
read(6, "Lmpib3NzLmNvbS93cy8yMDA3LzAxL2pi"..., 4096) = 4096
read(6, "MlZSZFdWMVpTQSs8L1ZhbHVlPjwvUHJv"..., 4096) = 4096
read(6, "-3186-11DF-BF20-B7B2D7785F29'',''1"..., 4096)
= 4096
read(6, "JXMWhibVF0WlhabGJuUSs8L21hcnNoYW"..., 4096) = 4096
read(6, "53B5A7C523B'',''0EC51560-3186-11DF"..., 4096)
= 4096
read(6, "B3B0-3186-11DF-B3B0-F2756F2630FE"..., 4096) = 4096
read(6, "ib3NzZXNiIj51cm46amJvc3MvZXNiL2V"..., 4096) = 4096
read(6, "122062 Query\tcommit\n\t\t122062 Que"..., 4096) = 4096
read(6, "7'',''0FBCE2E0-3186-11DF-A2E0-92D5"..., 4096)
= 4096
read(6, "Wx1bm1hcnNoYWwvcGx1Z2luL3Nlcmlhb"..., 4096) = 4096
read(6, "N (''CBA178E0-3186-11DF-B8E0-C39C"..., 4096) = 4096
read(6, "A6Ly9zY2hlbWFzLmpib3NzLmNvbS93cy"..., 4096) = 4096
read(6, "ZoWkUxbGMzTmhaMlZSZFdWMVpTQSs8L1"..., 4096) = 4096
read(6, "N.NAME FROM JUDDI_BUSINESS_SERVI"..., 4096) = 4096
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = 4096
read(6, "''1516AAF0-3186-11DF-AAF0-F9DB38C"..., 4096) = 4096
read(6, "XNiOnBlcnNpc3RlbnQ+PGpib3NzZXNiO"..., 4096) = 4096
read(6, "UE'',''RDLVR'')\n\t\t122063
Query\tcomm"..., 4096) = 4096
read(6, "51cm46amJvc3MvZXNiL2Vwci90eXBlL2"..., 4096) = 4096
read(6, "BY M.NAME DESC,M.LAST_UPDATE DES"..., 4096) = 4096
read(6, "6-11DF-B3B0-F2756F2630FE'',''10F5F"..., 4096)
= 4096
read(6, "1DF-AD30-E4944AD03A09'',''0F3F9C90"..., 4096)
= 4096
read(6, "lPnJPMEFCWFZ5QUFKYlFxenpGL2dHQ0Z"..., 4096) = 4096
read(6, "1DF-9DA0-B2D00126D089'',''15719C30"..., 4096)
= 4096
read(6, "lVZFdhR051VW1sYVYwWXdURzVTY0ZveV"..., 4096) = 4096
read(6, "'',''1354FDC0-3186-11DF-BDC0-9CD52"..., 4096)
= 4096
read(6, "F6EF0896BCFD'',''1516AAF0-3186-11D"..., 4096)
= 4096
read(6, "pib3NzZXNiOmphdmEubmFtaW5nLnByb3"..., 4096) = 4096
read(6, "lYZ3VhbTF6TG0xbGMzTmhaMlV1Y21Wa1"..., 4096) = 4096
read(6, " from message where uuid=''urn:jb"..., 4096) = 4096
read(6, "uTG1waWIzTnpMbk52WVM1bGMySXVaR1Z"..., 4096) = 4096
read(6, "THORIZED_NAME,OPERATOR,NAME,LANG"..., 4096) = 4096
read(6, "85F29'',''10A59530-3186-11DF-9530-"..., 4096)
= 4096
read(6, "nRlbnQ+PEtleT5iM0puTG1waWIzTnpMb"..., 4096) = 4096
read(6, "ATE,N.NAME FROM JUDDI_BUSINESS_S"..., 4096) = 4096
read(6, "SERVICE_NAME N WHERE N.SERVICE_K"..., 4096) = 4096
read(6, "bGJuUWdkR0Z5WjJWMFBTSndZVzVsYkhN"..., 4096) = 4096
read(6, "DF-8FA0-9C0B5EA0244B'',''12F55190-"..., 4096)
= 4096
read(6, "ZXh0RmFjdG9yeTwvamJvc3Nlc2I6amF2"..., 4096) = 4096
read(6, "YldWemMyRm5aUzV6YjNWeVkyVT08L0tl"..., 4096) = 4096
read(6, "1E0-3186-11DF-A1E0-B52AF3A5E0C9''"..., 4096) = 4096
read(6, "3186-11DF-9C90-F7F201853B15'',''0F"..., 4096)
= 4096
read(6, "V1YxWlM1dVlXMWw8L0tleT48VmFsdWU+"..., 4096) = 4096
read(6, "TMODEL_KEY,M.LAST_UPDATE,M.NAME,"..., 4096) = 4096
read(6, "tczovLzEwLjE4LjIuNTM6MTA5OS9xdWV"..., 4096) = 4096
read(6, "9PTwvS2V5PjxWYWx1ZT5yTzBBQlhOeUF"..., 4096) = 4096
read(6, "E = ''org.jboss.soa.esb.:category"..., 4096) = 4096
read(6, "mFtaW5nLnByb3ZpZGVyLnVybD48amJvc"..., 4096) = 4096
read(6, "mhaMlV1Y21Wa1pXeHBkbVZ5WldRPTwvS"..., 4096) = 4096
read(6, "90-C8786D5E36C7'',''129C0E00-3186-"..., 4096)
= 4096
read(6, "ZXM+PFByb3BlcnR5PjxLZXk+YjNKbkxt"..., 4096) = 4096
read(6, "0-3186-11DF-A1E0-DC63C5C390D7'',''"..., 4096)
= 4096
read(6, "86-11DF-9530-F89CA8A12257'',''106F"..., 4096)
= 4096
read(6, "11DF-9D30-F53B5A7C523B'',''0EC5156"..., 4096)
= 4096
read(6, "WhMbVZ6WWk1VFpYSjJhV05sQUFBQUFBQ"..., 4096) = 4096
read(6, "PmptczovLzEwLjE4LjIuNTM6MTA5OS9x"..., 4096) = 4096
read(6, "ZEE9PTwvS2V5PjxWYWx1ZT5yTzBBQlhO"..., 4096) = 4096
read(6, "DI_TMODEL WHERE TMODEL_KEY=''uuid"..., 4096) = 4096
read(6, ".NAME DESC,M.LAST_UPDATE DESC\n\t\t"..., 4096) = 4096
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = 4096
read(6, "bWFzLmpib3NzLmNvbS93cy8yMDA3LzAx"..., 4096) = 4096
read(6, ",''14AF5DA0-3186-11DF-9DA0-F90BD4"..., 4096) = 4096
read(6, "-F90BD46D9DB1'',''145D4600-3186-11"..., 4096)
= 4096
read(6, "EZNRWxGVmtWV1FUMDlDbDFkUGp3dlkyO"..., 4096) = 4096
read(6, "9D544D328F'',''1151B3B0-3186-11DF-"..., 4096)
= 4096
read(6, "774506'',''102FA1E0-3186-11DF-A1E0"..., 4096)
= 4096
read(6, "leT48VmFsdWU+ck8wQUJYUUFGM0YxWlh"..., 4096) = 4096
read(6, "uery\tcommit\n\t\t111462 Query\tdelet"..., 4096) = 4096
read(6, "ec86-411f-b507-aa95d7dafbab'',''ur"..., 4096)
= 4096
read(6, "xbExtUnZZZz09PC9LZXk+PFZhbHVlPnJ"..., 4096) = 4096
read(6, "6245C4E79A1'' AND DELETED IS NULL"..., 4096) = 4096
read(6, "jpqYXZhLm5hbWluZy5wcm92aWRlci51c"..., 4096) = 4096
read(6, "TJVZ1BDQnFiWE02THk4eE1DNHhPQzR5T"..., 4096) = 4096
read(6, "USINESS_SERVICE S,JUDDI_SERVICE_"..., 4096) = 4096
read(6, "tYXMuamJvc3MuY29tL3dzLzIwMDcvMDE"..., 4096) = 4096
read(6, "wZXJ0eT48S2V5PmIzSm5MbXBpYjNOekx"..., 4096) = 4096
read(6, "\t\t122062 Query\tSELECT S.SERVICE_"..., 4096) = 4096
read(6, "aW9uLXZlcnNpb24geG1sbnM6amJvc3Nl"..., 4096) = 4096
read(6, "QUM5dmNtY3VhbUp2YzNNdWMyOWhMbVZ6"..., 4096) = 4096
read(6, "3186-11DF-9560-9BE8ED6D3173'') OR"..., 4096) = 4096
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = 4096
read(6, "cm9wZXJ0eT48UHJvcGVydHk+PEtleT5i"..., 4096) = 4096
read(6, "W_URL,DELETED FROM JUDDI_TMODEL "..., 4096) = 4096
read(6, "22063 Query\tSELECT S.SERVICE_KEY"..., 4096) = 4096
read(6, "ZXNiPSJodHRwOi8vc2NoZW1hcy5qYm9z"..., 4096) = 4096
read(6, "DF-B8E0-C39CC8CB2DC1'',''CB2E1DA0-"..., 4096)
= 4096
read(6, "dHA6Ly9zY2hlbWFzLmpib3NzLmNvbS93"..., 4096) = 4096
read(6, "R1ZoWkUxbGMzTmhaMlZSZFdWMVpTQSs8"..., 4096) = 4096
read(6, "E1DA0-3186-11DF-9DA0-B2D00126D08"..., 4096) = 4096
read(6, "BA90-3186-11DF-BA90-929D544D328F"..., 4096) = 4096
read(6, "xWcFRFaFNjRm95Vm5sTmFUVjZXbGhLZE"..., 4096) = 4096
read(6, "ER BY N.NAME DESC,S.LAST_UPDATE "..., 4096) = 4096
read(6, "MS9qYm9zc2VzYiI+Q29ubmVjdGlvbkZh"..., 4096) = 4096
read(6, "dVpHVnNhWFpsY2k1cGMxSmxaR1ZzYVha"..., 4096) = 4096
read(6, "PDATE DESC\n\t\t122062 Query\tSELECT"..., 4096) = 4096
read(6, "values(''urn:jboss/esb/message/UI"..., 4096) = 4096
read(6, "cGliM056TG5OdllTNWxjMkl1YldWemMy"..., 4096) = 4096
read(6, "(UPPER(NAME) = ''PANELSSNOOPSERVI"..., 4096) = 4096
read(6, "enpGL2dHQ0ZUZ0FnQUFlSEFBQUFGclBI"..., 4096) = 4096
read(6, "71'',''1354FDC0-3186-11DF-BDC0-9CD"..., 4096)
= 4096
read(6, "= S.SERVICE_KEY AND ((UPPER(NAME"..., 4096) = 4096
read(6, "L2pib3NzZXNiOmphdmEubmFtaW5nLnBy"..., 4096) = 4096
read(6, "MllYZ3VhbTF6TG0xbGMzTmhaMlV1Y21W"..., 4096) = 4096
read(6, "\n\t\t111469 Query\tinsert into mess"..., 4096) = 4096
read(6, "VzVsYkhOVGJtOXZjRk5sY25acFkyVT08"..., 4096) = 4096
read(6, "ation=''RDLVR''\n\t\t122063 Query\tcom"...,
4096) = 4096
read(6, "E,M.NAME,M.DELETED FROM JUDDI_TM"..., 4096) = 4096
read(6, "001490127e99'',''urn:jboss/esb/mes"..., 4096)
= 4096
read(6, "9LZXk+PFZhbHVlPnJPMEFCWFFBSEZkbF"..., 4096) = 4096
read(6, "czwvamJvc3Nlc2I6amF2YS5uYW1pbmcu"..., 4096) = 4096
read(6, "bXBpYjNOekxuTnZZUzVsYzJJdWJXVnpj"..., 4096) = 4096
read(6, "'',''15719C30-3186-11DF-9C30-F6EF0"..., 4096)
= 4096
read(6, "12F55190-3186-11DF-9190-C8786D5E"..., 4096) = 4096
read(6, "RKYWNHSkhWVFpEVjBaNVdqTk9Za0o1U1"..., 4096) = 4096
read(6, "''0F0B6D30-3186-11DF-AD30-E4944AD"..., 4096) = 4096
read(6, "VzdGluYXRpb24tdHlwZSB4bWxuczpqYm"..., 4096) = 4096
read(6, "x1ZT5yTzBBQlhRQUJEUXhOemM9PC9WYW"..., 4096) = 4096
read(6, "Y M.NAME DESC,M.LAST_UPDATE DESC"..., 4096) = 4096
read(6, "4tZmFjdG9yeT48amJvc3Nlc2I6cGVyc2"..., 4096) = 4096
read(6, "lBQkZxWVhaaExteGhibWN1UW05dmJHVm"..., 4096) = 4096
read(6, "sb.:category'' ORDER BY M.NAME DE"..., 4096) = 4096
read(6, "aW5nLmZhY3RvcnkuaW5pdGlhbCB4bWxu"..., 4096) = 4096
read(6, "c2RXVjRjZ0FRYW1GMllTNXNZVzVuTGs1"..., 4096) = 4096
read(6, "UPPER(NAME) = ''PANELSSNOOPSERVIC"..., 4096) = 4096
read(6, "xtYXJzaGFsdW5tYXJzaGFsPjxwbHVnaW"..., 4096) = 4096
read(6, ",''10A59530-3186-11DF-9530-F89CA8"..., 4096) = 4096
read(6, "NOOPSERVICE'')) AND S.SERVICE_KEY"..., 4096) = 4096
read(6, "lbWFzLmpib3NzLmNvbS93cy8yMDA3LzA"..., 4096) = 4096
read(6, "TMODEL M WHERE M.DELETED IS NULL"..., 4096) = 4096
read(6, "5DDCCFBD2EF'',''120461E0-3186-11DF"..., 4096)
= 4096
read(6, "A6Ly9zY2hlbWFzLmpib3NzLmNvbS93cy"..., 4096) = 4096
read(6, "MODEL_KEY_REF = ''uuid:C7536010-F"..., 4096) = 4096
read(6, "bD48amJvc3Nlc2I6amF2YS5uYW1pbmcu"..., 4096) = 4096
read(6, "WldRPTwvS2V5PjxWYWx1ZT5yTzBBQlhO"..., 4096) = 4096
read(6, "ommit\n\t\t122063 Query\tSELECT S.SE"..., 4096) = 4096
read(6, "2063 Query\tSELECT S.SERVICE_KEY,"..., 4096) = 4096
read(6, "5qbnAuaW50ZXJmYWNlczwvamJvc3Nlc2"..., 4096) = 4096
read(6, "J0eT48S2V5PmIzSm5MbXBpYjNOekxuTn"..., 4096) = 4096
read(6, "pc3RlbnQ+PGpib3NzZXNiOmFja25vd2x"..., 4096) = 4096
read(6, "'')\n\t\t111469 Query\tcommit\n\t\t11146"..., 4096) =
4096
read(6, "F-BA90-929D544D328F'',''1151B3B0-3"..., 4096)
= 4096
read(6, "Y N.NAME DESC,S.LAST_UPDATE DESC"..., 4096) = 4096
read(6, "Nlc2IiPnRydWU8L2pib3NzZXNiOnBlcn"..., 4096) = 4096
read(6,
"M+PC9FbnZlbG9wZT4='',''TRUE'',''RDLV"...,
4096) = 4096
read(6, "elpYTnphVzl1WDJsa1BTSWlJR1JoZEdV"..., 4096) = 4096
read(6, "7536010-F786-11DB-A010-E6245C4E7"..., 4096) = 4096
read(6, "'')) AND S.SERVICE_KEY IN (''CBA17"..., 4096)
= 4096
read(6, "1DF-8A20-9D8418679D71'',''1354FDC0"..., 4096)
= 4096
read(6, "REpXZWxscE9XaGpNMngxV1hrNWIxcFhS"..., 4096) = 4096
read(6, "11DF-A1E0-B52AF3A5E0C9'',''0FED8FD"..., 4096)
= 4096
read(6, "UlhabGJuUkhZWFJsZDJGNTwvVmFsdWU+"..., 4096) = 4096
read(6, "9C8'',''13A2CFA0-3186-11DF-8FA0-9C"..., 4096)
= 4096
read(6, "11DF-B3B0-F2756F2630FE'',''10F5FF2"..., 4096)
= 4096
read(6, "F2756F2630FE'',''10F5FF20-3186-11D"..., 4096)
= 4096
read(6, "nZUlEPjwvSGVhZGVyPjxDb250ZXh0IC8"..., 4096) = 4096
read(6, "aWRlci51cmwgeG1sbnM6amJvc3Nlc2I9"..., 4096) = 4096
read(6, "NHhPQzR5TGpVek9qRXdPVGt2Y1hWbGRX"..., 4096) = 4096
read(6, "rmo'')) ORDER BY S.LAST_UPDATE DE"..., 4096) = 4096
read(6, "B2DC1'',''CB2E1DA0-3186-11DF-9DA0-"..., 4096)
= 4096
read(6, "C8'',''13A2CFA0-3186-11DF-8FA0-9C0"..., 4096)
= 4096
read(6, "+PC9WYWx1ZT48L0NvbnRlbnQ+PC9Cb2R"..., 4096) = 4096
read(6, "MDcvMDEvamJvc3Nlc2IiPnRydWU8L2pi"..., 4096) = 4096
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = 4096
On Mar 18, 1:57 pm, Michael DeHaan <mich...@reductivelabs.com>
wrote:> On Thu, Mar 18, 2010 at 12:45 PM, Thomas <tommyfis...@gmail.com>
wrote:
> > Puppet is version 0.25.4
>
> > The machines are all running under similar hardware, sorry if the word
> > "class" was misleading. The database machines are failing to
run
> > puppet, while all of the other servers (mostly application servers)
> > have no problem.
>
> Can you also
checkhttp://projects.reductivelabs.com/search/index/puppet?q=hangs&scope=&...
> for any open items that may apply to your case?
>
> --Michael
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Looking into it more, I think I found the root of the problem. The
following is a snippet from my puppet configuration:
# create /var/log/mysql/archived_logs
file { "archived_logs":
path => "/var/log/mysql/archived_logs",
ensure => directory,
mode => 775,
owner => "mysql",
group => "mysql",
recurse => "true",
require => File[mysql_log]
}
What I see happening in strace is puppet trying to read in data from
one of the files in the archived_logs directory. Did I make a mistake
in my configuration here?
On Mar 18, 2:18 pm, Thomas <tommyfis...@gmail.com>
wrote:> pstree doesn''t show anything interesting:
>
> ├─sshd,1354
> │ ├─sshd,2008
> │ │ └─bash,2010
> │ ├─sshd,4942
> │ │ └─bash,4944
> │ │ └─puppetd,5459 /usr/sbin/puppetd --onetime --no-
> daemonize --test --pluginsync --factpath=/var/lib/puppet/lib/facter --
> server puppet1.sermo.dev
> │ │ └─{puppetd},5505
>
> When I do strace, however, I get the following. Seems like it''s
stuck
> on something:
>
> --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
> rt_sigreturn(0x1a) = 4096
> read(6, "90BD46D9DB1'',''145D4600-3186-11DF"...,
4096) = 4096
> read(6, "1sL21hcnNoYWx1bm1hcnNoYWwvcGx1Z2"..., 4096) = 4096
> read(6, "F89CA8A12257'',''106FB820-3186-11D"...,
4096) = 4096
> read(6, "Nlc2IiPnF1ZXVlPC9qYm9zc2VzYjpkZX"..., 4096) = 4096
> read(6, "VsYzJJdWJXVnpjMkZuWlM1MGNtRnVjM0"..., 4096) = 4096
> read(6, "53B15'',''0F809D30-3186-11DF-9D30-"...,
4096) = 4096
> read(6, "y'' ORDER BY M.NAME DESC,M.LAST_U"..., 4096) =
4096
> read(6, "cGVydHk+PEtleT5iM0puTG1waWIzTnpM"..., 4096) = 4096
> read(6, "_SERVICE_CATEGORY C WHERE C.SERV"..., 4096) = 4096
> read(6, "46amJvc3MvZXNiL2Vwci90eXBlL2ptcz"..., 4096) = 4096
> read(6, "((UPPER(NAME) = ''PANELSSNOOPSERV"..., 4096) =
4096
> read(6, "_UPDATE FROM JUDDI_BUSINESS_SERV"..., 4096) = 4096
> read(6, "Lmpib3NzLmNvbS93cy8yMDA3LzAxL2pi"..., 4096) = 4096
> read(6, "MlZSZFdWMVpTQSs8L1ZhbHVlPjwvUHJv"..., 4096) = 4096
> read(6, "-3186-11DF-BF20-B7B2D7785F29'',''1"...,
4096) = 4096
> read(6, "JXMWhibVF0WlhabGJuUSs8L21hcnNoYW"..., 4096) = 4096
> read(6, "53B5A7C523B'',''0EC51560-3186-11DF"...,
4096) = 4096
> read(6, "B3B0-3186-11DF-B3B0-F2756F2630FE"..., 4096) = 4096
> read(6, "ib3NzZXNiIj51cm46amJvc3MvZXNiL2V"..., 4096) = 4096
> read(6, "122062 Query\tcommit\n\t\t122062 Que"..., 4096) = 4096
> read(6, "7'',''0FBCE2E0-3186-11DF-A2E0-92D5"...,
4096) = 4096
> read(6, "Wx1bm1hcnNoYWwvcGx1Z2luL3Nlcmlhb"..., 4096) = 4096
> read(6, "N (''CBA178E0-3186-11DF-B8E0-C39C"..., 4096) =
4096
> read(6, "A6Ly9zY2hlbWFzLmpib3NzLmNvbS93cy"..., 4096) = 4096
> read(6, "ZoWkUxbGMzTmhaMlZSZFdWMVpTQSs8L1"..., 4096) = 4096
> read(6, "N.NAME FROM JUDDI_BUSINESS_SERVI"..., 4096) = 4096
> --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
> rt_sigreturn(0x1a) = 4096
> read(6, "''1516AAF0-3186-11DF-AAF0-F9DB38C"..., 4096) =
4096
> read(6, "XNiOnBlcnNpc3RlbnQ+PGpib3NzZXNiO"..., 4096) = 4096
> read(6, "UE'',''RDLVR'')\n\t\t122063
Query\tcomm"..., 4096) = 4096
> read(6, "51cm46amJvc3MvZXNiL2Vwci90eXBlL2"..., 4096) = 4096
> read(6, "BY M.NAME DESC,M.LAST_UPDATE DES"..., 4096) = 4096
> read(6, "6-11DF-B3B0-F2756F2630FE'',''10F5F"...,
4096) = 4096
> read(6, "1DF-AD30-E4944AD03A09'',''0F3F9C90"...,
4096) = 4096
> read(6, "lPnJPMEFCWFZ5QUFKYlFxenpGL2dHQ0Z"..., 4096) = 4096
> read(6, "1DF-9DA0-B2D00126D089'',''15719C30"...,
4096) = 4096
> read(6, "lVZFdhR051VW1sYVYwWXdURzVTY0ZveV"..., 4096) = 4096
> read(6, "'',''1354FDC0-3186-11DF-BDC0-9CD52"...,
4096) = 4096
> read(6, "F6EF0896BCFD'',''1516AAF0-3186-11D"...,
4096) = 4096
> read(6, "pib3NzZXNiOmphdmEubmFtaW5nLnByb3"..., 4096) = 4096
> read(6, "lYZ3VhbTF6TG0xbGMzTmhaMlV1Y21Wa1"..., 4096) = 4096
> read(6, " from message where uuid=''urn:jb"..., 4096) =
4096
> read(6, "uTG1waWIzTnpMbk52WVM1bGMySXVaR1Z"..., 4096) = 4096
> read(6, "THORIZED_NAME,OPERATOR,NAME,LANG"..., 4096) = 4096
> read(6, "85F29'',''10A59530-3186-11DF-9530-"...,
4096) = 4096
> read(6, "nRlbnQ+PEtleT5iM0puTG1waWIzTnpMb"..., 4096) = 4096
> read(6, "ATE,N.NAME FROM JUDDI_BUSINESS_S"..., 4096) = 4096
> read(6, "SERVICE_NAME N WHERE N.SERVICE_K"..., 4096) = 4096
> read(6, "bGJuUWdkR0Z5WjJWMFBTSndZVzVsYkhN"..., 4096) = 4096
> read(6, "DF-8FA0-9C0B5EA0244B'',''12F55190-"...,
4096) = 4096
> read(6, "ZXh0RmFjdG9yeTwvamJvc3Nlc2I6amF2"..., 4096) = 4096
> read(6, "YldWemMyRm5aUzV6YjNWeVkyVT08L0tl"..., 4096) = 4096
> read(6, "1E0-3186-11DF-A1E0-B52AF3A5E0C9''"..., 4096) =
4096
> read(6, "3186-11DF-9C90-F7F201853B15'',''0F"...,
4096) = 4096
> read(6, "V1YxWlM1dVlXMWw8L0tleT48VmFsdWU+"..., 4096) = 4096
> read(6, "TMODEL_KEY,M.LAST_UPDATE,M.NAME,"..., 4096) = 4096
> read(6, "tczovLzEwLjE4LjIuNTM6MTA5OS9xdWV"..., 4096) = 4096
> read(6, "9PTwvS2V5PjxWYWx1ZT5yTzBBQlhOeUF"..., 4096) = 4096
> read(6, "E = ''org.jboss.soa.esb.:category"..., 4096) =
4096
> read(6, "mFtaW5nLnByb3ZpZGVyLnVybD48amJvc"..., 4096) = 4096
> read(6, "mhaMlV1Y21Wa1pXeHBkbVZ5WldRPTwvS"..., 4096) = 4096
> read(6, "90-C8786D5E36C7'',''129C0E00-3186-"...,
4096) = 4096
> read(6, "ZXM+PFByb3BlcnR5PjxLZXk+YjNKbkxt"..., 4096) = 4096
> read(6, "0-3186-11DF-A1E0-DC63C5C390D7'',''"...,
4096) = 4096
> read(6, "86-11DF-9530-F89CA8A12257'',''106F"...,
4096) = 4096
> read(6, "11DF-9D30-F53B5A7C523B'',''0EC5156"...,
4096) = 4096
> read(6, "WhMbVZ6WWk1VFpYSjJhV05sQUFBQUFBQ"..., 4096) = 4096
> read(6, "PmptczovLzEwLjE4LjIuNTM6MTA5OS9x"..., 4096) = 4096
> read(6, "ZEE9PTwvS2V5PjxWYWx1ZT5yTzBBQlhO"..., 4096) = 4096
> read(6, "DI_TMODEL WHERE TMODEL_KEY=''uuid"..., 4096) =
4096
> read(6, ".NAME DESC,M.LAST_UPDATE DESC\n\t\t"..., 4096) = 4096
> --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
> rt_sigreturn(0x1a) = 4096
> read(6, "bWFzLmpib3NzLmNvbS93cy8yMDA3LzAx"..., 4096) = 4096
> read(6, ",''14AF5DA0-3186-11DF-9DA0-F90BD4"..., 4096) =
4096
> read(6, "-F90BD46D9DB1'',''145D4600-3186-11"...,
4096) = 4096
> read(6, "EZNRWxGVmtWV1FUMDlDbDFkUGp3dlkyO"..., 4096) = 4096
> read(6, "9D544D328F'',''1151B3B0-3186-11DF-"...,
4096) = 4096
> read(6, "774506'',''102FA1E0-3186-11DF-A1E0"...,
4096) = 4096
> read(6, "leT48VmFsdWU+ck8wQUJYUUFGM0YxWlh"..., 4096) = 4096
> read(6, "uery\tcommit\n\t\t111462 Query\tdelet"..., 4096) = 4096
> read(6, "ec86-411f-b507-aa95d7dafbab'',''ur"...,
4096) = 4096
> read(6, "xbExtUnZZZz09PC9LZXk+PFZhbHVlPnJ"..., 4096) = 4096
> read(6, "6245C4E79A1'' AND DELETED IS NULL"..., 4096) =
4096
> read(6, "jpqYXZhLm5hbWluZy5wcm92aWRlci51c"..., 4096) = 4096
> read(6, "TJVZ1BDQnFiWE02THk4eE1DNHhPQzR5T"..., 4096) = 4096
> read(6, "USINESS_SERVICE S,JUDDI_SERVICE_"..., 4096) = 4096
> read(6, "tYXMuamJvc3MuY29tL3dzLzIwMDcvMDE"..., 4096) = 4096
> read(6, "wZXJ0eT48S2V5PmIzSm5MbXBpYjNOekx"..., 4096) = 4096
> read(6, "\t\t122062 Query\tSELECT S.SERVICE_"..., 4096) = 4096
> read(6, "aW9uLXZlcnNpb24geG1sbnM6amJvc3Nl"..., 4096) = 4096
> read(6, "QUM5dmNtY3VhbUp2YzNNdWMyOWhMbVZ6"..., 4096) = 4096
> read(6, "3186-11DF-9560-9BE8ED6D3173'') OR"..., 4096) =
4096
> --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
> rt_sigreturn(0x1a) = 4096
> read(6, "cm9wZXJ0eT48UHJvcGVydHk+PEtleT5i"..., 4096) = 4096
> read(6, "W_URL,DELETED FROM JUDDI_TMODEL "..., 4096) = 4096
> read(6, "22063 Query\tSELECT S.SERVICE_KEY"..., 4096) = 4096
> read(6, "ZXNiPSJodHRwOi8vc2NoZW1hcy5qYm9z"..., 4096) = 4096
> read(6, "DF-B8E0-C39CC8CB2DC1'',''CB2E1DA0-"...,
4096) = 4096
> read(6, "dHA6Ly9zY2hlbWFzLmpib3NzLmNvbS93"..., 4096) = 4096
> read(6, "R1ZoWkUxbGMzTmhaMlZSZFdWMVpTQSs8"..., 4096) = 4096
> read(6, "E1DA0-3186-11DF-9DA0-B2D00126D08"..., 4096) = 4096
> read(6, "BA90-3186-11DF-BA90-929D544D328F"..., 4096) = 4096
> read(6, "xWcFRFaFNjRm95Vm5sTmFUVjZXbGhLZE"..., 4096) = 4096
> read(6, "ER BY N.NAME DESC,S.LAST_UPDATE "..., 4096) = 4096
> read(6, "MS9qYm9zc2VzYiI+Q29ubmVjdGlvbkZh"..., 4096) = 4096
> read(6, "dVpHVnNhWFpsY2k1cGMxSmxaR1ZzYVha"..., 4096) = 4096
> read(6, "PDATE DESC\n\t\t122062 Query\tSELECT"..., 4096) = 4096
> read(6, "values(''urn:jboss/esb/message/UI"..., 4096) =
4096
> read(6, "cGliM056TG5OdllTNWxjMkl1YldWemMy"..., 4096) = 4096
> read(6, "(UPPER(NAME) = ''PANELSSNOOPSERVI"..., 4096) =
4096
> read(6, "enpGL2dHQ0ZUZ0FnQUFlSEFBQUFGclBI"..., 4096) = 4096
> read(6, "71'',''1354FDC0-3186-11DF-BDC0-9CD"...,
4096) = 4096
> read(6, "= S.SERVICE_KEY AND ((UPPER(NAME"..., 4096) = 4096
> read(6, "L2pib3NzZXNiOmphdmEubmFtaW5nLnBy"..., 4096) = 4096
> read(6, "MllYZ3VhbTF6TG0xbGMzTmhaMlV1Y21W"..., 4096) = 4096
> read(6, "\n\t\t111469 Query\tinsert into mess"..., 4096) = 4096
> read(6, "VzVsYkhOVGJtOXZjRk5sY25acFkyVT08"..., 4096) = 4096
> read(6, "ation=''RDLVR''\n\t\t122063
Query\tcom"..., 4096) = 4096
> read(6, "E,M.NAME,M.DELETED FROM JUDDI_TM"..., 4096) = 4096
> read(6, "001490127e99'',''urn:jboss/esb/mes"...,
4096) = 4096
> read(6, "9LZXk+PFZhbHVlPnJPMEFCWFFBSEZkbF"..., 4096) = 4096
> read(6, "czwvamJvc3Nlc2I6amF2YS5uYW1pbmcu"..., 4096) = 4096
> read(6, "bXBpYjNOekxuTnZZUzVsYzJJdWJXVnpj"..., 4096) = 4096
> read(6, "'',''15719C30-3186-11DF-9C30-F6EF0"...,
4096) = 4096
> read(6, "12F55190-3186-11DF-9190-C8786D5E"..., 4096) = 4096
> read(6, "RKYWNHSkhWVFpEVjBaNVdqTk9Za0o1U1"..., 4096) = 4096
> read(6, "''0F0B6D30-3186-11DF-AD30-E4944AD"..., 4096) =
4096
> read(6, "VzdGluYXRpb24tdHlwZSB4bWxuczpqYm"..., 4096) = 4096
> read(6, "x1ZT5yTzBBQlhRQUJEUXhOemM9PC9WYW"..., 4096) = 4096
> read(6, "Y M.NAME DESC,M.LAST_UPDATE DESC"..., 4096) = 4096
> read(6, "4tZmFjdG9yeT48amJvc3Nlc2I6cGVyc2"..., 4096) = 4096
> read(6, "lBQkZxWVhaaExteGhibWN1UW05dmJHVm"..., 4096) = 4096
> read(6, "sb.:category'' ORDER BY M.NAME DE"..., 4096) =
4096
> read(6, "aW5nLmZhY3RvcnkuaW5pdGlhbCB4bWxu"..., 4096) = 4096
> read(6, "c2RXVjRjZ0FRYW1GMllTNXNZVzVuTGs1"..., 4096) = 4096
> read(6, "UPPER(NAME) = ''PANELSSNOOPSERVIC"..., 4096) =
4096
> read(6, "xtYXJzaGFsdW5tYXJzaGFsPjxwbHVnaW"..., 4096) = 4096
> read(6, ",''10A59530-3186-11DF-9530-F89CA8"..., 4096) =
4096
> read(6, "NOOPSERVICE'')) AND S.SERVICE_KEY"..., 4096) =
4096
> read(6, "lbWFzLmpib3NzLmNvbS93cy8yMDA3LzA"..., 4096) = 4096
> read(6, "TMODEL M WHERE M.DELETED IS NULL"..., 4096) = 4096
> read(6, "5DDCCFBD2EF'',''120461E0-3186-11DF"...,
4096) = 4096
> read(6, "A6Ly9zY2hlbWFzLmpib3NzLmNvbS93cy"..., 4096) = 4096
> read(6, "MODEL_KEY_REF = ''uuid:C7536010-F"..., 4096) =
4096
> read(6, "bD48amJvc3Nlc2I6amF2YS5uYW1pbmcu"..., 4096) = 4096
> read(6, "WldRPTwvS2V5PjxWYWx1ZT5yTzBBQlhO"..., 4096) = 4096
> read(6, "ommit\n\t\t122063 Query\tSELECT S.SE"..., 4096) = 4096
> read(6, "2063 Query\tSELECT S.SERVICE_KEY,"..., 4096) = 4096
> read(6, "5qbnAuaW50ZXJmYWNlczwvamJvc3Nlc2"..., 4096) = 4096
> read(6, "J0eT48S2V5PmIzSm5MbXBpYjNOekxuTn"..., 4096) = 4096
> read(6, "pc3RlbnQ+PGpib3NzZXNiOmFja25vd2x"..., 4096) = 4096
> read(6, "'')\n\t\t111469 Query\tcommit\n\t\t11146"...,
4096) = 4096
> read(6, "F-BA90-929D544D328F'',''1151B3B0-3"...,
4096) = 4096
> read(6, "Y N.NAME DESC,S.LAST_UPDATE DESC"..., 4096) = 4096
> read(6, ...
>
> read more »
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
On Mar 18, 2010, at 12:24 PM, Thomas wrote:> Looking into it more, I think I found the root of the problem. The > following is a snippet from my puppet configuration: > > # create /var/log/mysql/archived_logs > file { "archived_logs": > path => "/var/log/mysql/archived_logs", > ensure => directory, > mode => 775, > owner => "mysql", > group => "mysql", > recurse => "true", > require => File[mysql_log] > } > > What I see happening in strace is puppet trying to read in data from > one of the files in the archived_logs directory. Did I make a mistake > in my configuration here? >What exactly are you trying to do here? Are you trying to push logs from the server to the clients? If not, why do you have "recurse => true"? If you are trying to push permissions, this isn''t a good way to do it. Sadly, Puppet is slow enough that an exec with "chown -R mysql:mysql /var/log/mysql/archived_logs" and a "chmod -R ..." would probably be much faster. One wild guess would be that a file in that folder is locked, and puppet is blocking on trying to do something to that file. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Peter Meier
2010-Mar-18 22:09 UTC
Re: [Puppet Users] Re: Puppet freezes in the middle of runs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1> What exactly are you trying to do here? Are you trying to push logs > from the server to the clients? If not, why do you have "recurse => > true"? If you are trying to push permissions, this isn''t a good way > to do it. Sadly, Puppet is slow enough that an exec with "chown -R > mysql:mysql /var/log/mysql/archived_logs" and a "chmod -R ..." would > probably be much faster.yeah have a look at: http://groups.google.com/group/puppet-users/browse_thread/thread/84ab151c4935524f and http://groups.google.com/group/puppet-dev/browse_thread/thread/2297b1e36c590d2e/ba9ab386bedf3f35> One wild guess would be that a file in that folder is locked, and > puppet is blocking on trying to do something to that file.which might additionally be a problem. cheers pete -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkuipLEACgkQbwltcAfKi39wGQCeMtIo7WoIyoVCl6eNTaJv8JnN VU0AoLC9RtAAylpFikfkB1jU7kw5eYJt =cwJc -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.