Hi, All! I have been looking for a way to eliminate the contents of certain large (e.g. pdf files) database columns from my development (and production) logs using RoR 3 with no luck. I found many posts, etc. about eliminating parameters (e.g. password) from the log, but nothing about the contents of the db. I have two reasons for this, 1) the verbosity and size of the log slow down my development/debugging, and 2) writing these columns definitely slows the operation of the site itself. Can anyone help? TIA Donz -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
I believe you''re looking for config.filter_parameters in application.rb. Note that filter_parameters is an array, so you can add to it with << Hope this helps, Garrett Lancaster> ------------------------------------------------------------------------ > > donz <mailto:donald-FWDIqG1pcu4dnm+yROfE0A@public.gmane.org> > January 10, 2011 6:30 PM > > > Hi, All! > > I have been looking for a way to eliminate the contents of certain > large (e.g. pdf files) database columns from my development (and > production) logs using RoR 3 with no luck. I found many posts, etc. > about eliminating parameters (e.g. password) from the log, but nothing > about the contents of the db. > > I have two reasons for this, 1) the verbosity and size of the log slow > down my development/debugging, and 2) writing these columns definitely > slows the operation of the site itself. > > Can anyone help? > > TIA > > Donz >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Don Ziesig wrote in post #973810:> Hi, All! > > I have been looking for a way to eliminate the contents of certain > large (e.g. pdf files) database columns from my development (and > production) logs using RoR 3 with no luck.Wait, you''re storing PDF files in your DB? Stop doing that. Data like that belongs in the filesystem. Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org Sent from my iPhone -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Donald R. Ziesig
2011-Jan-11 12:45 UTC
Re: [ruby_on_rails] Re: How to filter db column(s) from log.
Filesystem is write-only. On 1/11/2011 4:57 AM, Marnen Laibow-Koser wrote:> Don Ziesig wrote in post #973810: >> Hi, All! >> >> I have been looking for a way to eliminate the contents of certain >> large (e.g. pdf files) database columns from my development (and >> production) logs using RoR 3 with no luck. > Wait, you''re storing PDF files in your DB? Stop doing that. Data like > that belongs in the filesystem. > > Best, > -- > Marnen Laibow-Koser > http://www.marnen.org > marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org > > Sent from my iPhone >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Donald R. Ziesig
2011-Jan-11 12:56 UTC
Re: [ruby_on_rails] Re: How to filter db column(s) from log.
Garrett, I have config.filter_parameters = [:password, :document_file] in application.rb. :document_file is the name of the big column. I "puts" the array.inspect and confirmed that both password and document_file are there. The contents of the big column are still showing up in the log. BTW, I was advised to store these data in the file system instead of the db, Unfortunately, the file system will be read-only once I deploy and I want to test in the environment that I will be using, so I set the rails tree to read only during some of my development and test operations (which is how I discovered that I needed to use the database for all dynamic data in the first place). Thanks, Donz On 1/10/2011 10:33 PM, Garrett Lancaster wrote:> I believe you''re looking for config.filter_parameters in > application.rb. Note that filter_parameters is an array, so you can > add to it with << > > Hope this helps, > Garrett Lancaster > >> ------------------------------------------------------------------------ >> >> donz <mailto:donald-FWDIqG1pcu4dnm+yROfE0A@public.gmane.org> >> January 10, 2011 6:30 PM >> >> >> Hi, All! >> >> I have been looking for a way to eliminate the contents of certain >> large (e.g. pdf files) database columns from my development (and >> production) logs using RoR 3 with no luck. I found many posts, etc. >> about eliminating parameters (e.g. password) from the log, but nothing >> about the contents of the db. >> >> I have two reasons for this, 1) the verbosity and size of the log slow >> down my development/debugging, and 2) writing these columns definitely >> slows the operation of the site itself. >> >> Can anyone help? >> >> TIA >> >> Donz >> > -- > You received this message because you are subscribed to the Google > Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to > rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > For more options, visit this group at > http://groups.google.com/group/rubyonrails-talk?hl=en.-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Donald R. Ziesig
2011-Jan-11 12:57 UTC
Re: [ruby_on_rails] Re: How to filter db column(s) from log.
OOPS.... Filesystem is READ-only :-[ On 1/11/2011 7:45 AM, Donald R. Ziesig wrote:> Filesystem is write-only. > On 1/11/2011 4:57 AM, Marnen Laibow-Koser wrote: >> Don Ziesig wrote in post #973810: >>> Hi, All! >>> >>> I have been looking for a way to eliminate the contents of certain >>> large (e.g. pdf files) database columns from my development (and >>> production) logs using RoR 3 with no luck. >> Wait, you''re storing PDF files in your DB? Stop doing that. Data like >> that belongs in the filesystem. >> >> Best, >> -- >> Marnen Laibow-Koser >> http://www.marnen.org >> marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org >> >> Sent from my iPhone >> >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Walter Lee Davis
2011-Jan-11 14:25 UTC
Re: [ruby_on_rails] Re: How to filter db column(s) from log.
On Jan 11, 2011, at 7:57 AM, Donald R. Ziesig wrote:> OOPS.... Filesystem is READ-only :-[So use Paperclip and the S3 plugin, and have at it. You can write anything to S3, probably more files than you will ever have patience for. Walter> > On 1/11/2011 7:45 AM, Donald R. Ziesig wrote: >> Filesystem is write-only. >> On 1/11/2011 4:57 AM, Marnen Laibow-Koser wrote: >>> Don Ziesig wrote in post #973810: >>>> Hi, All! >>>> >>>> I have been looking for a way to eliminate the contents of certain >>>> large (e.g. pdf files) database columns from my development (and >>>> production) logs using RoR 3 with no luck. >>> Wait, you''re storing PDF files in your DB? Stop doing that. Data >>> like >>> that belongs in the filesystem. >>> >>> Best, >>> -- >>> Marnen Laibow-Koser >>> http://www.marnen.org >>> marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org >>> >>> Sent from my iPhone >>> >> > > -- > You received this message because you are subscribed to the Google > Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails- > talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > . > For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en > . >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Donald R. Ziesig
2011-Jan-11 15:27 UTC
Re: [ruby_on_rails] Re: How to filter db column(s) from log.
Hi All (again)!
I started this thread with a set of requirements and conditions that I
did not state in my original post. Among these are:
1) The ultimate host will have a read-only file system which
precludes use of paperclip''s default filesystem storage, and;
2) My client''s business/legal reasons preclude the use of
paperclip''s S3 option.
3) I have added the name(s) of the big columns in
config.filter_parameters but that doesn''t seem to change anything.
Pat Shaughnessy''s attachment-in-database-column(s) update to paperclip
(which works great, incidentally) does the job except for the cruft in
the log when storing large files.
The cruft tends to make my life difficult when analyzing the log and
seems to impact the speed during the time that the logfile is being written.
If I have to, I''ll continue with the current implementation and put up
with the aggravation hoping that production mode will stop most of it
(have to figure out where to put the log file, too, due to the read-only
fs).
Thanks,
Don Ziesig
On 1/11/2011 9:25 AM, Walter Lee Davis wrote:>
> On Jan 11, 2011, at 7:57 AM, Donald R. Ziesig wrote:
>
>> OOPS.... Filesystem is READ-only :-[
>
> So use Paperclip and the S3 plugin, and have at it. You can write
> anything to S3, probably more files than you will ever have patience for.
>
> Walter
>
>>
>> On 1/11/2011 7:45 AM, Donald R. Ziesig wrote:
>>> Filesystem is write-only.
>>> On 1/11/2011 4:57 AM, Marnen Laibow-Koser wrote:
>>>> Don Ziesig wrote in post #973810:
>>>>> Hi, All!
>>>>>
>>>>> I have been looking for a way to eliminate the contents of
certain
>>>>> large (e.g. pdf files) database columns from my development
(and
>>>>> production) logs using RoR 3 with no luck.
>>>> Wait, you''re storing PDF files in your DB? Stop doing
that. Data
>>>> like
>>>> that belongs in the filesystem.
>>>>
>>>> Best,
>>>> --
>>>> Marnen Laibow-Koser
>>>> http://www.marnen.org
>>>> marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org
>>>>
>>>> Sent from my iPhone
>>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Ruby on Rails: Talk" group.
>> To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To unsubscribe from this group, send email to
>>
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> For more options, visit this group at
>> http://groups.google.com/group/rubyonrails-talk?hl=en.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
>> donz >> January 10, 2011 6:30 PM >> >> Hi, All! >> >> I have been looking for a way to eliminate the contents of certain >> large (e.g. pdf files) database columns from my development (and >> production) logs using RoR 3 with no luck. I found many posts, etc. >> about eliminating parameters (e.g. password) from the log, but nothing >> about the contents of the db. >> >> I have two reasons for this, 1) the verbosity and size of the log slow >> down my development/debugging, and 2) writing these columns definitely >> slows the operation of the site itself. >> >> Can anyone help?You want it removed from the "SQL (1.2ms) INSERT...." line in the logs? filter_params will remove it from the "Parameters: {...." line. I think you''re only option for removing it from the SQL in the log itself is to silence the active record logger whenever you do something that generates those queries. It''s not an issue in production as production doesn''t log the sql queries... -philip -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
After a long interlude, I found my problem with config.filter_parameters. I can''t count how many hours I have wasted due to a typo :( The original line was: config.filter_parameters = [:password, :document_file] the correct line is: config.filter_parameters = [:password, :documents_file] This isn''t the first time that I have gotten burned by rails'' pluralization. Many thanks for all who commented and pointed me in the right direction. Don Z. -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.