Hello all. I find myself in unfamiliar territory by posting to this list but here we go. I am doing some experimentation regarding logging in JSON formats for integration with systems like LogStash. So far things are reasonably good but there is one thing in rails which is making things a lot harder than they need to be. At the moment rails has a large number of LogSubscribers such as ActiveRecord::ExplainSubscriber and ActionController::LogSubscriber which are automatically subscribed to the Notifications system. What I would like to be able to do is stop this automatic subscribing from happening (in effect turning off the rails default logging system). Currently this is not possible as far as I can tell, it is possible to go through all the various notifications and remove subscribers but this is a pain, is not very granular and is generally a very hack''ish situation. Now, messing around with the internals of rails is something I do not have a huge amount of experience with so I would like to ask the core members: 1. Before I start would there be any non-technical objections to a pull request to allow for us to disable all the default LogSubscribers? 2. If not would anyone be willing to offer advice / guidance / checking the design as I put this together? cheers Jeff -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.
On Fri, Sep 14, 2012 at 05:51:13PM +0900, Jeffrey Jones wrote:> Hello all. > > I find myself in unfamiliar territory by posting to this list but > here we go. > > I am doing some experimentation regarding logging in JSON formats > for integration with systems like LogStash. So far things are > reasonably good but there is one thing in rails which is making > things a lot harder than they need to be. > > At the moment rails has a large number of LogSubscribers such as > ActiveRecord::ExplainSubscriber and ActionController::LogSubscriber > which are automatically subscribed to the Notifications system. > > What I would like to be able to do is stop this automatic > subscribing from happening (in effect turning off the rails default > logging system). > > Currently this is not possible as far as I can tell, it is possible > to go through all the various notifications and remove subscribers > but this is a pain, is not very granular and is generally a very > hack''ish situation. > > Now, messing around with the internals of rails is something I do > not have a huge amount of experience with so I would like to ask the > core members: > > 1. Before I start would there be any non-technical objections to a > pull request to allow for us to disable all the default > LogSubscribers?I''d rather we don''t have something specific to *log* subscribers. You''re right though, there is no way to find the *types* of listeners listening on each channel. Maybe just being able to access the list of subscribers would be enough?> 2. If not would anyone be willing to offer advice / guidance / > checking the design as I put this together?Yes. Put together an example of an API you would like (you don''t need to actually make it work), and I''ll help you figure something out. The more you can share about the problem you''re trying to solve, the more we can help! -- Aaron Patterson http://tenderlovemaking.com/
On 15/09/12 15:41, Aaron Patterson wrote:> On Fri, Sep 14, 2012 at 05:51:13PM +0900, Jeffrey Jones wrote: >> Hello all. >> >> I find myself in unfamiliar territory by posting to this list but >> here we go. >> >> I am doing some experimentation regarding logging in JSON formats >> for integration with systems like LogStash. So far things are >> reasonably good but there is one thing in rails which is making >> things a lot harder than they need to be. >> >> At the moment rails has a large number of LogSubscribers such as >> ActiveRecord::ExplainSubscriber and ActionController::LogSubscriber >> which are automatically subscribed to the Notifications system. >> >> What I would like to be able to do is stop this automatic >> subscribing from happening (in effect turning off the rails default >> logging system). >> >> Currently this is not possible as far as I can tell, it is possible >> to go through all the various notifications and remove subscribers >> but this is a pain, is not very granular and is generally a very >> hack''ish situation. >> >> Now, messing around with the internals of rails is something I do >> not have a huge amount of experience with so I would like to ask the >> core members: >> >> 1. Before I start would there be any non-technical objections to a >> pull request to allow for us to disable all the default >> LogSubscribers? > I''d rather we don''t have something specific to *log* subscribers. > You''re right though, there is no way to find the *types* of listeners > listening on each channel. > > Maybe just being able to access the list of subscribers would be enough?I think there is the possibility for two things here. First is adding a few functions to make it easier to see all / types of subscribers to the notifications system which would be a good thing. For my purposes though what I would really like to do is be able to stop the default Rails subscribers from subscribing in the first place. At the moment they all automatically subscribe as soon as they are loaded (Subscribing is done automatically in the same file as they are defined.).>> 2. If not would anyone be willing to offer advice / guidance / >> checking the design as I put this together? > Yes. Put together an example of an API you would like (you don''t need > to actually make it work), and I''ll help you figure something out. > > The more you can share about the problem you''re trying to solve, the > more we can help! >The problem I have is that I want to write JSON formatted log files that contain the same information as you get from the basic rails ones (to start with), something like the following: https://gist.github.com/3694987 but it is not as easy as it should be to silence the default rails logging system. I could of course kludge things and just use separate logfile for the JSON stuff and redirect the default loggers to /dev/null or something but that feels wrong. I do actually have a plan for dealing with people using the logger with plain strings but I would still like to get rid of all the noise from the rails default subscribers. I could also do something like this in a gem''s railtie: https://gist.github.com/3740558 but this is imprecise as you can get. What I would love to be able to do is something along the lines of config.default_subscibers.disable = true in application.rb (or a railtie) to stop all of them from subscribing themselves in the first place, it just seems like the cleaner approach. Cheers Jeff -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.
I''ve also just found a use case where I''d like to unsubscribe the default subscribers. I''d like to turn on INFO level logging for my application to get some better visibility into the running production application. However, the application gets a ton of traffic (100''s of requests/second) and with the default subscribers the logs will fill up with gigabytes of data in a matter of minutes. What I think would make sense is to remove the static subscribe code from the LogSubscriber classes and instead have it be application config option and simply have the default subscribers be configured by default. On Monday, September 17, 2012 5:24:44 PM UTC-7, Jeffrey Jones wrote:> > > On 15/09/12 15:41, Aaron Patterson wrote: > > On Fri, Sep 14, 2012 at 05:51:13PM +0900, Jeffrey Jones wrote: > >> Hello all. > >> > >> I find myself in unfamiliar territory by posting to this list but > >> here we go. > >> > >> I am doing some experimentation regarding logging in JSON formats > >> for integration with systems like LogStash. So far things are > >> reasonably good but there is one thing in rails which is making > >> things a lot harder than they need to be. > >> > >> At the moment rails has a large number of LogSubscribers such as > >> ActiveRecord::ExplainSubscriber and ActionController::LogSubscriber > >> which are automatically subscribed to the Notifications system. > >> > >> What I would like to be able to do is stop this automatic > >> subscribing from happening (in effect turning off the rails default > >> logging system). > >> > >> Currently this is not possible as far as I can tell, it is possible > >> to go through all the various notifications and remove subscribers > >> but this is a pain, is not very granular and is generally a very > >> hack''ish situation. > >> > >> Now, messing around with the internals of rails is something I do > >> not have a huge amount of experience with so I would like to ask the > >> core members: > >> > >> 1. Before I start would there be any non-technical objections to a > >> pull request to allow for us to disable all the default > >> LogSubscribers? > > I''d rather we don''t have something specific to *log* subscribers. > > You''re right though, there is no way to find the *types* of listeners > > listening on each channel. > > > > Maybe just being able to access the list of subscribers would be enough? > I think there is the possibility for two things here. First is adding a > few functions to make it easier to see all / types of subscribers > to the notifications system which would be a good thing. > > For my purposes though what I would really like to do is be able to stop > the default Rails subscribers from subscribing in the first place. > At the moment they all automatically subscribe as soon as they are > loaded (Subscribing is done automatically in the same file as they are > defined.). > > >> 2. If not would anyone be willing to offer advice / guidance / > >> checking the design as I put this together? > > Yes. Put together an example of an API you would like (you don''t need > > to actually make it work), and I''ll help you figure something out. > > > > The more you can share about the problem you''re trying to solve, the > > more we can help! > > > The problem I have is that I want to write JSON formatted log files that > contain the same information as you get from the basic rails ones (to > start with), something like the following: > https://gist.github.com/3694987 but it is not as easy as it should be to > silence the default rails logging system. > > I could of course kludge things and just use separate logfile for the > JSON stuff and redirect the default loggers to /dev/null or something > but that feels wrong. I do actually have a plan for dealing with people > using the logger with plain strings but I would still like to get rid of > all the noise from the rails default subscribers. > > I could also do something like this in a gem''s railtie: > https://gist.github.com/3740558 but this is imprecise as you can get. > > What I would love to be able to do is something along the lines of > > config.default_subscibers.disable = true > > in application.rb (or a railtie) to stop all of them from subscribing > themselves in the first place, it just seems like the cleaner approach. > > Cheers > > Jeff > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-core/-/F228yt_XpmwJ. To post to this group, send email to rubyonrails-core@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.
On Wed, Sep 19, 2012 at 11:00:32AM -0700, Brian Durand wrote:> I''ve also just found a use case where I''d like to unsubscribe the default > subscribers. I''d like to turn on INFO level logging for my application to > get some better visibility into the running production application. > However, the application gets a ton of traffic (100''s of requests/second) > and with the default subscribers the logs will fill up with gigabytes of > data in a matter of minutes. > > What I think would make sense is to remove the static subscribe code from > the LogSubscriber classes and instead have it be application config option > and simply have the default subscribers be configured by default.This makes sense to me. I think it would deal with Jeffrey''s use case as well. Can you put together a patch and we can discuss on a PR? -- Aaron Patterson http://tenderlovemaking.com/
Groovy, I shall give it a go. Cheers Jeff On 24/09/12 05:25, Aaron Patterson wrote:> On Wed, Sep 19, 2012 at 11:00:32AM -0700, Brian Durand wrote: >> I''ve also just found a use case where I''d like to unsubscribe the default >> subscribers. I''d like to turn on INFO level logging for my application to >> get some better visibility into the running production application. >> However, the application gets a ton of traffic (100''s of requests/second) >> and with the default subscribers the logs will fill up with gigabytes of >> data in a matter of minutes. >> >> What I think would make sense is to remove the static subscribe code from >> the LogSubscriber classes and instead have it be application config option >> and simply have the default subscribers be configured by default. > This makes sense to me. I think it would deal with Jeffrey''s use case > as well. Can you put together a patch and we can discuss on a PR? >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.