Dennis Hoppe
2012-Mar-15 10:49 UTC
[Puppet Users] Problem with stored configs / Invalid unicode escaping
Hello,
i have a problem with stored configs since the migration from "sqlite"
to "postgresql".
dho@appelbaum:~$ sudo puppetd --test --verbose
info: Retrieving plugin
info: Loading facts in disks
info: Loading facts in users
info: Loading facts in mountpoints
info: Loading facts in disks
info: Loading facts in users
info: Loading facts in mountpoints
err: Could not retrieve catalog from remote server: wrong header line format
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run
I took a closer look and found the following error message at my
"postgresql.log".
2012-03-15 11:15:01 CET FEHLER: ungültiges Unicode-Escape bei Zeichen 184
2012-03-15 11:15:01 CET TIPP: Unicode-Escapes müssen \uXXXX oder
\UXXXXXXXX sein.
2012-03-15 11:15:01 CET ANWEISUNG: INSERT INTO "param_values"
("created_at", "line", "resource_id",
"updated_at", "value",
"param_name_id") VALUES(''2012-03-15
11:15:01.040833'', NULL, 154,
''2012-03-15 11:15:01.040833'', E''#
# THIS FILE IS MANAGED BY PUPPET
# /etc/puppet/modules/production/bash/templates/squeeze/etc/skel/bashrc.erb
#
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
# If not running interactively, don''''t do anything
[ -z "$PS1" ] && return
# don''''t put duplicate lines in the history. See bash(1) for
more options
# don''''t overwrite GNU Midnight
Commander''''s setting of `ignorespace''''.
HISTCONTROL=$HISTCONTROL${HISTCONTROL+:}ignoredups
# ... or force ignoredups and ignorespace
HISTCONTROL=ignoreboth
# append to the history file, don''''t overwrite it
shopt -s histappend
# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=1000
HISTFILESIZE=2000
HISTTIMEFORMAT="%F %T "
# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize
# make less more friendly for non-text input files, see lesspipe(1)
#[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"
# set variable identifying the chroot you work in (used in the prompt
below)
if [ -z "$debian_chroot" ] && [ -r /etc/debian_chroot ]; then
debian_chroot=$(cat /etc/debian_chroot)
fi
# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
xterm-color) color_prompt=yes;;
esac
# uncomment for a colored prompt, if the terminal has the capability;
turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
force_color_prompt=yes
if [ -n "$force_color_prompt" ]; then
if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
# We have color support; assume it''''s compliant with Ecma-48
# (ISO/IEC-6429). (Lack of such support is extremely rare, and such
# a case would tend to support setf rather than setaf.)
color_prompt=yes
else
color_prompt fi
fi
if [ "$color_prompt" = yes ]; then
PS1=''''${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$
''''
else
PS1=''''${debian_chroot:+($debian_chroot)}\u@\h:\w\$
''''
fi
unset color_prompt force_color_prompt
# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
;;
*)
;;
esac
# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
test -r ~/.dircolors && eval "$(dircolors -b
~/.dircolors)" || eval
"$(dircolors -b)"
#alias ls=''''ls --color=auto''''
#alias dir=''''dir --color=auto''''
#alias vdir=''''vdir --color=auto''''
alias grep=''''grep --color=auto''''
alias fgrep=''''fgrep --color=auto''''
alias egrep=''''egrep --color=auto''''
alias cl=''''clear''''
alias l=''''ls --color -l''''
alias lh=''''ls --color -lh''''
alias ll=''''ls --color -la''''
alias ls=''''ls --color=auto''''
fi
# some more ls aliases
#alias ll=''''ls -l''''
#alias la=''''ls -A''''
#alias l=''''ls -CF''''
# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
# enable programmable completion features (you don''''t need to
enable
# this, if it''''s already enabled in /etc/bash.bashrc and
/etc/profile
# sources /etc/bash.bashrc).
if [ -f /etc/bash_completion ] && ! shopt -oq posix; then
. /etc/bash_completion
fi
'', 26) RETURNING "id"
If i disable the include for my BASH module
(https://github.com/dhoppe/puppet-bash), everything is working as expected.
Does anyone know wich component of puppet is responsible for the wrong
unicode escaping? I would like to do some more debugging before i file a
bug report.
BTW. I am using the following package versions:
dho@appelbaum:~$ dpkg -l | egrep ''(puppet|postgresql)''
ii postgresql-9.1 9.1.3-1~bpo60+1
object-relational SQL database, version 9.1 server
ii postgresql-client-9.1 9.1.3-1~bpo60+1
front-end programs for PostgreSQL 9.1
ii postgresql-client-common 128~bpo60+1
manager for multiple PostgreSQL client versions
ii postgresql-common 128~bpo60+1
PostgreSQL database-cluster manager
ii postgresql-server-dev-9.1 9.1.3-1~bpo60+1
development files for PostgreSQL 9.1 server-side programming
ii puppet 2.6.2-5+squeeze4
Centralized configuration management - agent startup and compatibility
scripts
ii puppet-common 2.6.2-5+squeeze4
Centralized configuration management
ii puppetmaster
dho@appelbaum:~$ gem list
*** LOCAL GEMS ***
activemodel (3.0.11)
activerecord (3.0.11)
activesupport (3.0.11)
arel (2.0.10)
builder (2.1.2)
hiera (0.3.0)
hiera-puppet (0.3.0)
i18n (0.5.0)
pg (0.13.2)
tzinfo (0.3.32)
Regards, Dennis
Dennis Hoppe
2012-Mar-18 14:53 UTC
Re: [Puppet Users] Problem with stored configs / Invalid unicode escaping
Hello, Am 15.03.2012 11:49, schrieb Dennis Hoppe:> i have a problem with stored configs since the migration from "sqlite" > to "postgresql". > > dho@appelbaum:~$ sudo puppetd --test --verbose > info: Retrieving plugin > info: Loading facts in disks > info: Loading facts in users > info: Loading facts in mountpoints > info: Loading facts in disks > info: Loading facts in users > info: Loading facts in mountpoints > err: Could not retrieve catalog from remote server: wrong header line format > warning: Not using cache on failed catalog > err: Could not retrieve catalog; skipping run > > I took a closer look and found the following error message at my > "postgresql.log". > > 2012-03-15 11:15:01 CET FEHLER: ungültiges Unicode-Escape bei Zeichen 184 > 2012-03-15 11:15:01 CET TIPP: Unicode-Escapes müssen \uXXXX oder > \UXXXXXXXX sein. > 2012-03-15 11:15:01 CET ANWEISUNG: INSERT INTO "param_values" > ("created_at", "line", "resource_id", "updated_at", "value", > "param_name_id") VALUES(''2012-03-15 11:15:01.040833'', NULL, 154, > ''2012-03-15 11:15:01.040833'', E''# > # THIS FILE IS MANAGED BY PUPPET > # /etc/puppet/modules/production/bash/templates/squeeze/etc/skel/bashrc.erb > # > > # ~/.bashrc: executed by bash(1) for non-login shells. > # see /usr/share/doc/bash/examples/startup-files (in the package bash-doc) > # for examples > > # If not running interactively, don''''t do anything > [ -z "$PS1" ] && return > > # don''''t put duplicate lines in the history. See bash(1) for more options > # don''''t overwrite GNU Midnight Commander''''s setting of `ignorespace''''. > HISTCONTROL=$HISTCONTROL${HISTCONTROL+:}ignoredups > # ... or force ignoredups and ignorespace > HISTCONTROL=ignoreboth > > # append to the history file, don''''t overwrite it > shopt -s histappend > > # for setting history length see HISTSIZE and HISTFILESIZE in bash(1) > HISTSIZE=1000 > HISTFILESIZE=2000 > HISTTIMEFORMAT="%F %T " > > # check the window size after each command and, if necessary, > # update the values of LINES and COLUMNS. > shopt -s checkwinsize > > # make less more friendly for non-text input files, see lesspipe(1) > #[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)" > > # set variable identifying the chroot you work in (used in the prompt > below) > if [ -z "$debian_chroot" ] && [ -r /etc/debian_chroot ]; then > debian_chroot=$(cat /etc/debian_chroot) > fi > > # set a fancy prompt (non-color, unless we know we "want" color) > case "$TERM" in > xterm-color) color_prompt=yes;; > esac > > # uncomment for a colored prompt, if the terminal has the capability; > turned > # off by default to not distract the user: the focus in a terminal window > # should be on the output of commands, not on the prompt > force_color_prompt=yes > > if [ -n "$force_color_prompt" ]; then > if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then > # We have color support; assume it''''s compliant with Ecma-48 > # (ISO/IEC-6429). (Lack of such support is extremely rare, and such > # a case would tend to support setf rather than setaf.) > color_prompt=yes > else > color_prompt> fi > fi > > if [ "$color_prompt" = yes ]; then > > PS1=''''${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ > '''' > else > PS1=''''${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '''' > fi > unset color_prompt force_color_prompt > > # If this is an xterm set the title to user@host:dir > case "$TERM" in > xterm*|rxvt*) > PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1" > ;; > *) > ;; > esac > > # enable color support of ls and also add handy aliases > if [ -x /usr/bin/dircolors ]; then > test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval > "$(dircolors -b)" > #alias ls=''''ls --color=auto'''' > #alias dir=''''dir --color=auto'''' > #alias vdir=''''vdir --color=auto'''' > > alias grep=''''grep --color=auto'''' > alias fgrep=''''fgrep --color=auto'''' > alias egrep=''''egrep --color=auto'''' > > alias cl=''''clear'''' > alias l=''''ls --color -l'''' > alias lh=''''ls --color -lh'''' > alias ll=''''ls --color -la'''' > alias ls=''''ls --color=auto'''' > fi > > # some more ls aliases > #alias ll=''''ls -l'''' > #alias la=''''ls -A'''' > #alias l=''''ls -CF'''' > > # Alias definitions. > # You may want to put all your additions into a separate file like > # ~/.bash_aliases, instead of adding them here directly. > # See /usr/share/doc/bash-doc/examples in the bash-doc package. > > if [ -f ~/.bash_aliases ]; then > . ~/.bash_aliases > fi > > # enable programmable completion features (you don''''t need to enable > # this, if it''''s already enabled in /etc/bash.bashrc and /etc/profile > # sources /etc/bash.bashrc). > if [ -f /etc/bash_completion ] && ! shopt -oq posix; then > . /etc/bash_completion > fi > '', 26) RETURNING "id" > > If i disable the include for my BASH module > (https://github.com/dhoppe/puppet-bash), everything is working as expected. > > Does anyone know wich component of puppet is responsible for the wrong > unicode escaping? I would like to do some more debugging before i file a > bug report. > > BTW. I am using the following package versions: > > dho@appelbaum:~$ dpkg -l | egrep ''(puppet|postgresql)'' > ii postgresql-9.1 9.1.3-1~bpo60+1 > object-relational SQL database, version 9.1 server > ii postgresql-client-9.1 9.1.3-1~bpo60+1 > front-end programs for PostgreSQL 9.1 > ii postgresql-client-common 128~bpo60+1 > manager for multiple PostgreSQL client versions > ii postgresql-common 128~bpo60+1 > PostgreSQL database-cluster manager > ii postgresql-server-dev-9.1 9.1.3-1~bpo60+1 > development files for PostgreSQL 9.1 server-side programming > ii puppet 2.6.2-5+squeeze4 > Centralized configuration management - agent startup and compatibility > scripts > ii puppet-common 2.6.2-5+squeeze4 > Centralized configuration management > ii puppetmaster > > dho@appelbaum:~$ gem list > > *** LOCAL GEMS *** > > activemodel (3.0.11) > activerecord (3.0.11) > activesupport (3.0.11) > arel (2.0.10) > builder (2.1.2) > hiera (0.3.0) > hiera-puppet (0.3.0) > i18n (0.5.0) > pg (0.13.2) > tzinfo (0.3.32)i did some debugging and found out that with PostgreSQL 8.4 everything works as expected. If i use PostgreSQL 9.1, the following lines are responsible for the error message. PS1=''${debian_chroot:+($debian_chroot)}\[<%= t_color -%>\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '' PS1=''${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '' PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1" I spoke to our DBA and he told me that with PostgreSQL 9.1 the parameter "standard_conforming_strings" was introduced. If i set this parameter to off, everyhting works as expected again. Regards, Dennis
jcbollinger
2012-Mar-19 13:57 UTC
[Puppet Users] Re: Problem with stored configs / Invalid unicode escaping
On Mar 18, 9:53 am, Dennis Hoppe <dennis.ho...@debian-solutions.de> wrote:> i did some debugging and found out that with PostgreSQL 8.4 everything > works as expected. > > If i use PostgreSQL 9.1, the following lines are responsible for the > error message. > > PS1=''${debian_chroot:+($debian_chroot)}\[<%= t_color > -%>\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '' > PS1=''${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '' > PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1" > > I spoke to our DBA and he told me that with PostgreSQL 9.1 the parameter > "standard_conforming_strings" was introduced. If i set this parameter to > off, everyhting works as expected again.I am glad you discovered a solution. Your DBA is incorrect that the standard_conforming_strings parameter is new in PG 9, however. It is documented at least as far back as version 8.2, and with the same default value (off). It appears that with standard_conforming_strings on, Puppet was attempting to insert your file content as a PG ''escaped'' string instead of as a literal string. Perhaps it did so because of the backslashes in the value, but whatever the reason, that looks like a bug to me, and I encourage you to file an issue about it. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Dennis Hoppe
2012-Mar-21 09:48 UTC
Re: [Puppet Users] Re: Problem with stored configs / Invalid unicode escaping
Hello John, Am 19.03.2012 14:57, schrieb jcbollinger:> On Mar 18, 9:53 am, Dennis Hoppe <dennis.ho...@debian-solutions.de> > wrote: > >> i did some debugging and found out that with PostgreSQL 8.4 everything >> works as expected. >> >> If i use PostgreSQL 9.1, the following lines are responsible for the >> error message. >> >> PS1=''${debian_chroot:+($debian_chroot)}\[<%= t_color >> -%>\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '' >> PS1=''${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '' >> PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1" >> >> I spoke to our DBA and he told me that with PostgreSQL 9.1 the parameter >> "standard_conforming_strings" was introduced. If i set this parameter to >> off, everyhting works as expected again. > > I am glad you discovered a solution. Your DBA is incorrect that the > standard_conforming_strings parameter is new in PG 9, however. It is > documented at least as far back as version 8.2, and with the same > default value (off).that was my fault. The parameter "standard_conforming_strings" is not new, but since PostgreSQL 9.x the value "on" is default.> It appears that with standard_conforming_strings on, Puppet was > attempting to insert your file content as a PG ''escaped'' string > instead of as a literal string. Perhaps it did so because of the > backslashes in the value, but whatever the reason, that looks like a > bug to me, and I encourage you to file an issue about it.The backslashes are not the problem, but \u will be handled as unicode escapes which should look like "\uXXXX" or "\UXXXXXXXX". I hoped that somebody would know the component which is creating the insert statement, because is wanted to do some more debugging before i file a bug. Regards, Dennis